Don't Fear the Implicits: Everything You Need to Know About Typeclasses

Slides from Scala Days Berlin 2016

Developers who are new to Scala often shy away from coming into contact with implicits, and by extension, understanding typeclasses. In big organizations that have been adopting Scala at scale, you sometimes even come across hard rules that put a ban on the use of implicits because that language feature is considered to be too advanced and not understood by a lot of developers. On the other hand, implicits and typeclasses are used heavily not only by a lot of the most important Scala frameworks and libraries, but also in the standard library. Given the fact that it is so hard to evade them when writing real world Scala code, I would like to encourage developers adopting Scala to overcome their fear of implicits and instead embrace the typeclass pattern. In this talk, as an intermediate Scala developer, you will learn everything you really need to know about typeclasses: What they are good for and how they compare to what you are familiar with from object-oriented languages, when you should and should not use them, how the pattern can be encoded in Scala and how to write your own typeclasses, how to provide instances of typeclasses for your own or existing types, and how to do all of this with minimal boilerplate. Throughout the talk, you will see numerous examples of typeclasses used in the Scala ecosystem and the standard library, and you'll see that you don't need to know anything about category theory to benefit from embracing typeclasses.

Transcript

1. Don’t Fear the Implicits
   Everything You Need to Know About
   Typeclasses
   Daniel Westheide
   

   View Slide

2. View Slide

3. About me
   > consultant at innoQ Germany
   > author of The Neophyte’s Guide to Scala
   > Twitter: @kaffeecoder
   > Website: danielwestheide.com
   

   View Slide

4. Implicits
   

   View Slide

5. def map[S](f: T => S)
   (implicit executor: ExecutionContext): Future[S]
   def ?(message: Any)(implicit timeout: Timeout)
   def !(message: Any)
   (implicit sender: ActorRef = Actor.noSender)
   def errorsAsJson(implicit lang: Lang): JsValue
   dependencies
   configuration
   context
   context
   

   View Slide

6. Typeclasses
   

   View Slide

7. Magic Button(c) Jake Rust, https://www.flickr.com/photos/jakerust/16846017825/, CC-BY 2.0, www.gotcredit.com
   

   View Slide

8. @Configuration
   @EnableAuthorizationServer
   public class AuthorizationServerConfiguration
   extends AuthorizationServerConfigurerAdapter {
   ...
   }
   

   View Slide

9. La la la, I can’t see
   you, darn implicits!
   I can’t see you (c) Newtown grafitti
   https://www.flickr.com/photos/newtown_grafitti/5225673037
   CC-BY 2.0
   

   View Slide

10. val xs = Vector(1, 2).map(_ * 2)
    val sum = xs.sum
    CanBuildFrom
    Numeric
    

    View Slide

11. View Slide

12. Arcane (c) Bart, https://www.flickr.com/photos/cayusa/2651494125, CC-BY 2.0
    

    View Slide

13. Example
    > given: big data framework for processing
    distributed data sets
    > need: processing of various types of
    quantities
    

    View Slide

14. DistributedDataset
    trait DistributedDataset[A] {
    def foldLeft[B](z: B)(op: (B, A) => B): B
    def count: Int
    }
    

    View Slide

15. case class Kilograms(value: BigDecimal) {
    def +(y: Kilograms): Kilograms = Kilograms(value + y.value)
    def -(y: Kilograms): Kilograms = Kilograms(value - y.value)
    def *(y: BigDecimal): Kilograms = Kilograms(value * y)
    def /(y: BigDecimal): Kilograms = Kilograms(value / y)
    }
    object Kilograms {
    val zero: Kilograms = Kilograms(BigDecimal(0))
    def sum(xs: DistributedDataset[Kilograms]): Kilograms =
    xs.foldLeft(zero)(_ + _)
    def mean(xs: DistributedDataset[Kilograms]): Kilograms =
    sum(xs) / xs.count
    }
    

    View Slide

16. case class Kilometers(value: BigDecimal) {
    def +(y: Kilometers): Kilometers = Kilometers(value + y.value)
    def -(y: Kilometers): Kilometers = Kilometers(value - y.value)
    def *(y: BigDecimal): Kilometers = Kilometers(value * y)
    def /(y: BigDecimal): Kilometers = Kilometers(value / y)
    }
    object Kilometers {
    val zero: Kilometers = Kilometers(BigDecimal(0))
    def sum(xs: DistributedDataset[Kilometers]): Kilometers =
    xs.foldLeft(zero)(_ + _)
    def mean(xs: DistributedDataset[Kilometers]): Kilometers =
    sum(xs) / xs.count
    }
    

    View Slide

17. I know! Let’s use
    inheritance!
    paris rodin thinker (c) Mark B. Schlemmer
    https://www.flickr.com/photos/mbschlemmer/3122967980
    CC-BY 2.0
    

    View Slide

18. trait Quantity[A <: Quantity[A]] {
    def value: BigDecimal
    def unit(x: BigDecimal): A
    def +(y: A): A = unit(value + y.value)
    def -(y: A): A = unit(value - y.value)
    def *(y: BigDecimal): A = unit(value * y)
    def /(y: BigDecimal): A = unit(value / y)
    }
    object Quantity {
    def sum[A <: Quantity[A]](xs: DistributedDataset[A], zero: A): A =
    xs.foldLeft(zero)(_ + _)
    def mean[A <: Quantity[A]](xs: DistributedDataset[A], zero: A): A =
    sum(xs, zero) / xs.count
    }
    

    View Slide

19. case class Kilograms(value: BigDecimal) extends Quantity[Kilograms] {
    override def unit(x: BigDecimal): Kilograms = Kilograms(x)
    }
    case class Kilometers(value: BigDecimal) extends Quantity[Kilometers] {
    override def unit(x: BigDecimal): Kilometers = Kilometers(x)
    }
    

    View Slide

20. Maybe I should use
    an adapter…
    USB ethernet adapter (c) Ash Kyd
    https://www.flickr.com/photos/ashkyd/14359608157
    CC-BY 2.0
    

    View Slide

21. import org.joda.time.Duration
    /* Adapter pattern */
    case class Milliseconds(underlying: Duration) extends
    Quantity[Milliseconds] {
    override def value: BigDecimal = underlying.getMillis
    override def unit(x: BigDecimal): Milliseconds =
    Milliseconds(Duration.millis(x.toLong))
    }
    

    View Slide

22. val durations: DistributedDataset[Milliseconds] =
    DistributedDataset.distribute(Seq(
    Milliseconds(Duration.standardMinutes(3)),
    Milliseconds(Duration.standardSeconds(17)),
    Milliseconds(Duration.standardSeconds(5)),
    Milliseconds(Duration.standardHours(1))))
    val meanDuration = Quantity.mean(durations, Milliseconds(Duration.ZERO))
    Lots of new objects
    boilerplate
    losing the original type
    

    View Slide

23. Try it differently!
    old woman in the woods (c) Matt Wiebe
    https://www.flickr.com/photos/mattwieve/17721503894
    CC-BY 2.0
    

    View Slide

24. import org.joda.time.Duration
    case class Kilograms(value: BigDecimal)
    case class Kilometers(value: BigDecimal)
    trait Quantity[A] {
    def value(x: A): BigDecimal
    def unit(x: BigDecimal): A
    def zero: A = unit(BigDecimal(0))
    def plus(x: A, y: A): A = unit(value(x) + value(y))
    def minus(x: A, y: A): A = unit(value(x) - value(y))
    def times(x: A, y: BigDecimal): A = unit(value(x) * y)
    def div(x: A, y: BigDecimal): A = unit(value(x) / y)
    }
    

    View Slide

25. import org.joda.time.Duration
    object Quantity {
    val kilogramQuantity: Quantity[Kilograms] = new Quantity[Kilograms] {
    override def value(x: Kilograms): BigDecimal = x.value
    override def unit(x: BigDecimal): Kilograms = Kilograms(x)
    }
    val kilometerQuantity: Quantity[Kilometers] = new Quantity[Kilometers] {
    override def value(x: Kilometers): BigDecimal = x.value
    override def unit(x: BigDecimal): Kilometers = Kilometers(x)
    }
    // to be continued
    }
    

    View Slide

26. import org.joda.time.Duration
    object Quantity {
    // continued
    val durationQuantity: Quantity[Duration] = new Quantity[Duration] {
    override val zero: Duration = Duration.ZERO
    override def value(x: Duration): BigDecimal = x.getMillis
    override def plus(x: Duration, y: Duration): Duration = x.plus(y)
    override def minus(x: Duration, y: Duration): Duration = x.minus(y)
    override def unit(x: BigDecimal): Duration = Duration.millis(x.toLong)
    }
    // to be continued
    }
    

    View Slide

27. import org.joda.time.Duration
    object Quantity {
    // continued
    def sum[A](xs: DistributedDataset[A], quantity: Quantity[A]): A =
    xs.foldLeft(quantity.zero)(quantity.plus)
    def mean[A](xs: DistributedDataset[A], quantity: Quantity[A]): A =
    quantity.div(sum(xs, quantity), xs.count)
    }
    

    View Slide

28. val durations: DistributedDataset[Duration] =
    DistributedDataset.distribute(Seq(
    Duration.standardMinutes(3),
    Duration.standardSeconds(17),
    Duration.standardSeconds(5),
    Duration.standardHours(1)))
    val meanDuration = Quantity.mean(durations, Quantity.durationQuantity)
    

    View Slide

29. def sum[A](xs: DistributedDataset[A], quantity: Quantity[A]): A =
    xs.foldLeft(quantity.zero)(quantity.plus)
    def mean[A](xs: DistributedDataset[A], quantity: Quantity[A]): A =
    quantity.div(sum(xs, quantity), xs.count)
    

    View Slide

30. def sum[A](xs: DistributedDataset[A])(quantity: Quantity[A]): A =
    xs.foldLeft(quantity.zero)(quantity.plus)
    def mean[A](xs: DistributedDataset[A])(quantity: Quantity[A]): A =
    quantity.div(sum(xs)(quantity), xs.count)
    separate parameter list
    separate parameter list
    

    View Slide

31. import org.joda.time.Duration
    val durations: DistributedDataset[Duration] =
    DistributedDataset.distribute(Seq(
    Duration.standardMinutes(3),
    Duration.standardSeconds(17),
    Duration.standardSeconds(5),
    Duration.standardHours(1)))
    val meanDuration = Quantity.mean(durations, Quantity.durationQuantity)
    

    View Slide

32. import org.joda.time.Duration
    val durations: DistributedDataset[Duration] =
    DistributedDataset.distribute(Seq(
    Duration.standardMinutes(3),
    Duration.standardSeconds(17),
    Duration.standardSeconds(5),
    Duration.standardHours(1)))
    val meanDuration = Quantity.mean(durations)(Quantity.durationQuantity)
    separate parameter list
    

    View Slide

33. def sum[A](xs: DistributedDataset[A])
    (implicit quantity: Quantity[A]): A =
    xs.foldLeft(quantity.zero)(quantity.plus)
    def mean[A](xs: DistributedDataset[A])
    (implicit quantity: Quantity[A]): A =
    quantity.div(sum(xs), xs.count)
    

    View Slide

34. import org.joda.time.Duration
    object Quantity {
    implicit val kilogramQuantity: Quantity[Kilograms] = ???
    implicit val kilometerQuantity: Quantity[Kilometers] = ???
    implicit val durationQuantity: Quantity[Duration] = ???
    }
    

    View Slide

35. import org.joda.time.Duration
    val durations: DistributedDataset[Duration] =
    DistributedDataset.distribute(Seq(
    Duration.standardMinutes(3),
    Duration.standardSeconds(17),
    Duration.standardSeconds(5),
    Duration.standardHours(1)))
    val meanDuration = Quantity.mean(durations)(Quantity.durationQuantity)
    

    View Slide

36. import org.joda.time.Duration
    val durations: DistributedDataset[Duration] =
    DistributedDataset.distribute(Seq(
    Duration.standardMinutes(3),
    Duration.standardSeconds(17),
    Duration.standardSeconds(5),
    Duration.standardHours(1)))
    val meanDuration = Quantity.mean(durations) implicits filled in by compiler
    

    View Slide

37. And this is how we
    discovered typeclasses…
    

    View Slide

38. Context bounds
    def sum[A: Quantity](xs: DistributedDataset[A]): A = {
    val quantity = implicitly[Quantity[A]]
    xs.foldLeft(quantity.zero)(quantity.plus)
    }
    def mean[A: Quantity](xs: DistributedDataset[A]): A = {
    val quantity = implicitly[Quantity[A]]
    quantity.div(sum(xs), xs.count)
    }
    

    View Slide

39. Implicit resolution
    Imported
    Inherited
    in package object
    Local
    Explicit
    Implicit scope
    > companion object of typeclass
    > companion object of A
    > companion objects of super
    types
    

    View Slide

40. Local implicit
    import org.joda.time.Duration
    implicit val durationQuantity: Quantity[Duration] = ???
    val durations: DistributedDataset[Duration] =
    DistributedDataset.distribute(Seq(
    Duration.standardMinutes(3),
    Duration.standardSeconds(17),
    Duration.standardSeconds(5),
    Duration.standardHours(1)))
    val meanDuration = Quantity.mean(durations)
    

    View Slide

41. Imported implicits
    import org.joda.time.Duration
    trait LowPriorityImplicits {
    implicit val durationQuantity: Quantity[Duration] = ???
    }
    object Implicits extends LowPriorityImplicits
    

    View Slide

42. Imported implicits
    import org.joda.time.Duration
    import Implicits._
    val durations: DistributedDataset[Duration] = ???
    val meanDuration = Quantity.mean(durations)
    imported implicit Quantity[Duration]
    

    View Slide

43. Inherited implicits
    import org.joda.time.Duration
    object MixinExample extends App
    with LowPriorityImplicits {
    val durations: DistributedDataset[Duration] = ???
    val meanDuration = Quantity.mean(durations)
    println(meanDuration)
    }
    mixed in implicit Quantity[Duration]
    

    View Slide

44. Overriding implicits
    import org.joda.time.Duration
    object OverridingExample extends App with LowPriorityImplicits {
    implicit val myDurationQuantity: Quantity[Duration] = ???
    val durations: DistributedDataset[Duration] = ???
    val meanDuration = Quantity.mean(durations)
    println(meanDuration)
    }
    

    View Slide

45. Operators
    package quantities
    package object lib {
    implicit class QuantityOps[A](a: A)(implicit quantity: Quantity[A]) {
    def +(a2: A): A = quantity.plus(a, a2)
    def -(a2: A): A = quantity.minus(a, a2)
    def *(y: BigDecimal): A = quantity.times(a, y)
    def /(y: BigDecimal): A = quantity.div(a, y)
    }
    }
    

    View Slide

46. Operators
    import quantities.v10.lib._
    val nineKilos = Kilograms(3) * 3
    val twelveKilos = nineKilos + Kilograms(3)
    

    View Slide

47. Usability
    

    View Slide

48. Error messages
    import scala.annotation.implicitNotFound
    @implicitNotFound("No instance of typeclass Quantity found for type ${A}.")
    trait Quantity[A] {
    def value(x: A): BigDecimal
    def unit(x: BigDecimal): A
    def zero: A = unit(BigDecimal(0))
    def plus(x: A, y: A): A = unit(value(x) + value(y))
    def minus(x: A, y: A): A = unit(value(x) - value(y))
    def times(x: A, y: BigDecimal): A = unit(value(x) * y)
    def div(x: A, y: BigDecimal): A = unit(value(x) / y)
    }
    

    View Slide

49. Shortcut for implicitly
    object Quantity {
    def apply[A](implicit quantity: Quantity[A]): Quantity[A] = quantity
    def sum[A : Quantity](xs: DistributedDataset[A]): A =
    xs.foldLeft(Quantity[A].zero)(Quantity[A].plus)
    def mean[A : Quantity](xs: DistributedDataset[A]): A =
    Quantity[A].div(sum(xs), xs.count)
    }
    

    View Slide

50. Typeclass constructors
    object Quantity {
    def simple[A](unitF: BigDecimal => A)
    (valueF: A => BigDecimal): Quantity[A] = new Quantity[A] {
    override def value(x: A): BigDecimal = valueF(x)
    override def unit(x: BigDecimal): A = unitF(x)
    }
    implicit val kilogramQuantity = Quantity.simple(Kilograms)(_.value)
    implicit val kilometerQuantity = Quantity.simple(Kilometers)(_.value)
    }
    

    View Slide

51. Typeclass constructors: Ordering
    import org.joda.time.{DateTime, Duration, LocalDate}
    implicit lazy val defaultDurationOrdering: Ordering[Duration] =
    Ordering.by(_.getMillis)
    implicit lazy val defaultDateTimeOrdering: Ordering[DateTime] =
    Ordering.fromLessThan(_ isBefore _)
    

    View Slide

52. Combinators
    

    View Slide

53. Combinators: Circe
    import io.circe.{Encoder, Json}
    import io.circe.Encoder._
    case class BookId(value: Int) extends AnyVal
    object BookId {
    implicit val bookIdEncoder: Encoder[BookId] = Encoder[Int].contramap(_.value)
    }
    package io.circe
    trait Encoder[A] extends Serializable { self =>
    def apply(a: A): Json
    final def contramap[B](f: B => A): Encoder[B] = new Encoder[B] {
    final def apply(a: B) = self(f(a))
    }
    }
    

    View Slide

54. Generic typeclass instances
    

    View Slide

55. Generic instances: Circe
    package io.circe
    object Encoder {
    implicit final def encodeOption[A](implicit e: Encoder[A]):
    Encoder[Option[A]] =
    new Encoder[Option[A]] {
    final def apply(a: Option[A]): Json = a match {
    case Some(v) => e(v)
    case None => Json.Null
    }
    }
    }
    

    View Slide

56. Deriving instances of Foo[A] from
    instances of Bar[A]
    

    View Slide

57. Example: Plugging typeclass-based
    JSON libraries into Akka HTTP
    

    View Slide

58. Akka HTTP JSON
    package de.heikoseeberger.akkahttpcirce
    import akka.http.scaladsl.marshalling.{ Marshaller, ToEntityMarshaller }
    import akka.http.scaladsl.model.MediaTypes.`application/json`
    import io.circe.{ Encoder, Json, Printer }
    trait CirceSupport {
    implicit def circeToEntityMarshaller[A]
    (implicit encoder: Encoder[A], printer: Json => String =
    Printer.noSpaces.pretty): ToEntityMarshaller[A] =
    Marshaller.StringMarshaller.wrap(`application/json`)
    (printer).compose(encoder.apply)
    }
    

    View Slide

59. import io.circe.Encoder
    import io.circe.Encoder._
    import io.circe.generic.semiauto._
    case class Book(id: BookId, title: String, author: String)
    object Book {
    implicit val bookEncoder: Encoder[Book] = deriveEncoder[Book]
    }
    import akka.http.scaladsl.server.Directives._
    import de.heikoseeberger.akkahttpcirce.CirceSupport
    trait Routes extends CirceSupport {
    val bookRepository = new BookRepository
    val route = path("books" / IntNumber) { bookId =>
    get {
    complete {
    bookRepository.find(BookId(bookId))
    }
    }
    }
    }
    

    View Slide

60. Default instances
    > Provide default instances that make sense
    most of the time
    > Make it easy to opt out of default instances
    

    View Slide

61. Default instances
    > Don’t provide default instances whose
    behaviour is often undesired
    > Examples:
    > PTypeH[AnyRef]
    > EmptyValue[HttpResponse] with generic
    Marshaller[Option[A], B]
    

    View Slide

62. Serializable
    trait Quantity[A] extends Serializable {
    def value(x: A): BigDecimal
    def unit(x: BigDecimal): A
    def zero: A = unit(BigDecimal(0))
    def plus(x: A, y: A): A = unit(value(x) + value(y))
    def minus(x: A, y: A): A = unit(value(x) - value(y))
    def times(x: A, y: BigDecimal): A = unit(value(x) * y)
    def div(x: A, y: BigDecimal): A = unit(value(x) / y)
    }
    thankful Spark,
    Crunch,
    etc. users
    

    View Slide

63. Implicits FTW?
    

    View Slide

64. View Slide

65. Questions to ask
    > Is retroactive extension an important use case?
    > Can you provide good defaults?
    > How much boilerplate do you avoid? How much clarity do you
    lose?
    > Can you support two usage models, one of them not relying on
    implicits?
    

    View Slide

66. Summary
    > Typeclasses enable retroactive extension
    > Alternative to inheritance and adapters
    > Provide simple constructors and combinators
    > Powerful due to automatic derivation
    > Interactions of multiple typeclasses difficult to understand
    > Don’t throw typeclasses at every problem
    

    View Slide

67. Thank you for your attention!
    Twitter: @kaffeecoder Website: danielwestheide.com
    

    View Slide

68. View Slide

69. View Slide