Combining Scalaz and Shapeless for Great Good

Scalaz is an extension to the Scala standard library providing purely functional data structures and useful abstractions. Shapeless is a library for generic programming and pushes Scala’s type system quite far. Recently, there have been some developments in providing interoperability between both libraries, which turns out to be massively useful.

In this talk, I showcased some of the cool available techniques, including abstracting over heterogeneous data and the arity of functions, using macros to generate type class instances, and (de)serializing data with zero boilerplate.

Transcript

1. Combining Scalaz and Shapeless for Great Good
   Lars Hupel
   Technische Universität München
   October 25th, 2013
   

   View Slide

2. Agenda
   1 Deriving Instances
   2 Automatic Serialization
   3 Functions with Arbitrary Arity
   2
   

   View Slide

3. Type Classes in Scala
   Bad News
   Scala offers no native support for type classes
   3
   

   View Slide

4. Type Classes in Scala
   Bad News
   Scala offers no native support for type classes
   Good News
   ... but they can be encoded
   3
   

   View Slide

5. Type Classes in Scala
   Example: Ordering
   trait Ordering[A] {
   def compare(x: A, y: A): Order
   }
   ▶ type classes are (first-class) traits
   4
   

   View Slide

6. Type Classes in Scala
   Example: Ordering
   trait Ordering[A] {
   def compare(x: A, y: A): Order
   }
   ▶ type classes are (first-class) traits
   ▶ can use arbitrary language features
   4
   

   View Slide

7. Type Classes in Scala
   Example: Ordering
   trait Ordering[A] {
   def compare(x: A, y: A): Order
   def equal(x: A, y: A) =
   compare(x, y) == Order.EQ
   }
   ▶ type classes are (first-class) traits
   ▶ can use arbitrary language features
   e.g. default implementations
   4
   

   View Slide

8. Type Classes in Scala
   Example: Ordering
   implicit object IntHasOrdering extends Ordering[Int] {
   def compare(x: Int, y: Int) =
   if (x < y) Order.LT
   else if (x > y) Order.GT
   else Order.EQ
   }
   ▶ instances are (first-class) implicits
   5
   

   View Slide

9. Type Classes in Scala
   Example: Ordering
   implicit object IntHasOrdering extends Ordering[Int] {
   def compare(x: Int, y: Int) =
   if (x < y) Order.LT
   else if (x > y) Order.GT
   else Order.EQ
   }
   ▶ instances are (first-class) implicits
   ▶ can use arbitrary language features
   5
   

   View Slide

10. Type Classes in Scala
    Example: Ordering
    implicit object IntHasOrdering extends Ordering[Int] {
    def compare(x: Int, y: Int) =
    if (x < y) Order.LT
    else if (x > y) Order.GT
    else Order.EQ
    }
    ▶ instances are (first-class) implicits
    ▶ can use arbitrary language features
    e.g. macros
    5
    

    View Slide

11. Implementing Instances
    class Vector2D(val x: Int, val y: Int)
    class Vector3D(val x: Int, val y: Int, val z: Int)
    ▶ we want: default instances for ...
    ▶ Semigroup (pointwise addition)
    ▶ Ordering (lexicographic order)
    ▶ and more?
    6
    

    View Slide

12. Implementing Instances
    class Vector2D(val x: Int, val y: Int)
    class Vector3D(val x: Int, val y: Int, val z: Int)
    ▶ we want: default instances for ...
    ▶ Semigroup (pointwise addition)
    ▶ Ordering (lexicographic order)
    ▶ and more?
    ▶ very repetitive to implement by hand
    ▶ need a mechanism to automate that task
    6
    

    View Slide

13. Implementing Instances
    class Vector2D(val x: Int, val y: Int)
    class Vector3D(val x: Int, val y: Int, val z: Int)
    ▶ Scala already provides some automation
    6
    

    View Slide

14. Implementing Instances
    case class Vector2D(x: Int, y: Int)
    case class Vector3D(x: Int, y: Int, z: Int)
    ▶ Scala already provides some automation
    6
    

    View Slide

15. Implementing Instances
    case class Vector2D(x: Int, y: Int)
    case class Vector3D(x: Int, y: Int, z: Int)
    ▶ Scala already provides some automation
    ▶ generates:
    ▶ toString
    ▶ equals
    ▶ hashCode
    6
    

    View Slide

16. Implementing Instances
    case class Vector2D(x: Int, y: Int)
    case class Vector3D(x: Int, y: Int, z: Int)
    ▶ Scala already provides some automation
    ▶ generates:
    ▶ toString
    ▶ equals
    ▶ hashCode
    ▶ Problem: baked into the compiler
    6
    

    View Slide

17. Implementing Instances
    case class Vector2D(x: Int, y: Int)
    case class Vector3D(x: Int, y: Int, z: Int)
    ▶ Scala already provides some automation
    ▶ generates:
    ▶ toString
    ▶ equals
    ▶ hashCode
    ▶ Problem: baked into the compiler
    6
    

    View Slide

18. Goals
    ... for library designers
    1. Define a type class.
    2. Implement some base
    instances.
    3. Declare how to combine
    instances.
    7
    

    View Slide

19. Goals
    ... for library designers
    1. Define a type class.
    2. Implement some base
    instances.
    3. Declare how to combine
    instances.
    ... for library users
    1. Define some types.
    7
    

    View Slide

20. The Essence of Data Types
    ▶ we are dealing with algebraic data types
    ▶ ... consisting of one or more cases
    ▶ ... consisting of zero or more fields
    8
    

    View Slide

21. The Essence of Data Types
    Example: Lists
    sealed trait List
    case class Cons(head: Int, tail: List) extends List
    case class Nil() extends List
    9
    

    View Slide

22. The Essence of Data Types
    Example: Lists
    sealed trait List
    case class Cons(head: Int, tail: List) extends List
    case class Nil() extends List
    Cons(1, Cons(2, Nil()))
    .
    .
    Cons
    .
    1
    .
    Cons
    .
    2
    .
    Nil
    9
    

    View Slide

23. Abstract Representation
    Think: Serialization
    10
    

    View Slide

24. Abstract Representation
    Think: Serialization
    ... but not with bits, rather with products and sums
    10
    

    View Slide

25. Abstract Representation
    Examples
    Data type
    case class Vector3D(x: Int, y: Int, z: Int)
    Representation
    type Repr = (Int, Int, Int)
    11
    

    View Slide

26. Abstract Representation
    Examples
    Data type
    sealed trait Shape
    case class Circle(radius: Int) extends Shape
    case class Rectangle(height: Int, width: Int) extends Shape
    Representation
    type Repr = Either[Int, (Int, Int)]
    11
    

    View Slide

27. Type Classes as Type Constructors
    Realization: type classes are regular types
    12
    

    View Slide

28. Type Classes as Type Constructors
    Realization: type classes are regular types
    ... Scala’s encoding makes abstraction possible
    12
    

    View Slide

29. Composing Instances
    Example: Ordering
    Given
    val A: Ordering[A]
    val B: Ordering[B]
    ... we can implement
    val prod: Ordering[(A, B)]
    val sum: Ordering[Either[A, B]]
    13
    

    View Slide

30. Composing Instances
    Example: Ordering
    Given
    val A: Ordering[A]
    val B: Ordering[B]
    ... we can implement
    val prod: Ordering[(A, B)]
    val sum: Ordering[Either[A, B]]
    Question: Can we do the same for Semigroup?
    13
    

    View Slide

31. Composing Instances
    Example: Ordering
    Demo
    14
    

    View Slide

32. Putting it together
    Macros!
    Input
    Data type type T
    Type class type C[_]
    Output
    Instance val instance: C[T]
    15
    

    View Slide

33. Putting it together
    Macros!
    Input
    Data type type T
    Type class type C[_]
    Output
    Instance val instance: C[T]
    15
    

    View Slide

34. Problems
    ▶ base instances are resolved via regular implicit search
    ▶ What happens for (mutually) recursive data types?
    16
    

    View Slide

35. Problems
    ▶ base instances are resolved via regular implicit search
    ▶ What happens for (mutually) recursive data types?
    ▶ solution: make macro aware of recursion (aka tying the knot)
    ▶ self recursion: easy, use this instead of implicitly
    ▶ mutual recursion: require user setup
    ▶ indirect recursion: ongoing ...
    16
    

    View Slide

36. Library Support
    Shapeless · Scalaz · Spire
    typelevel.org
    github.com/typelevel
    17
    

    View Slide

37. Agenda
    1 Deriving Instances
    2 Automatic Serialization
    3 Functions with Arbitrary Arity
    18
    

    View Slide

38. Automatic Serialization in Java
    ... just extend Serializable, what could possibly go wrong?
    19
    

    View Slide

39. Use Case: Serialization
    type ByteString = // ...
    def encode(a: A): ByteString
    def decode(bs: ByteString): Option[(A, ByteString)]
    20
    

    View Slide

40. Use Case: Serialization
    ... for library designers
    1. Define a type class. ✓
    2. Implement some base instances. ✓
    3. Declare how to combine instances. ✓
    21
    

    View Slide

41. Use Case: Serialization
    ... for library users
    1. Define some types.
    22
    

    View Slide

42. Use Case: Serialization
    ... for library users
    1. Define some types. Demo
    22
    

    View Slide

43. Use Case: Serialization
    ... for library users
    1. Define some types. Demo
    22
    

    View Slide

44. Agenda
    1 Deriving Instances
    2 Automatic Serialization
    3 Functions with Arbitrary Arity
    23
    

    View Slide

45. Recap: Representation
    Data type
    case class Employee(name: String, years: Int, salary: Float)
    Representation
    type Repr = (String, Int, Float)
    24
    

    View Slide

46. Recap: Representation
    Data type
    case class Employee(name: String, years: Int, salary: Float)
    Representation
    type Repr = String :: Int :: Float :: HNil
    24
    

    View Slide

47. Heterogeneous Lists
    Think: tuples, but with arbitrary arity
    25
    

    View Slide

48. Heterogeneous Lists
    Think: tuples, but with arbitrary arity
    ... makes abstraction over arity possible
    25
    

    View Slide

49. Use Case: Applicative Composition
    The Problem
    ▶ You have Options of different types.
    ▶ You want to apply a function which takes plain values.
    26
    

    View Slide

50. Use Case: Applicative Composition
    The Problem
    ▶ You have Options of different types.
    ▶ You want to apply a function which takes plain values.
    The Naïve Solution
    Pattern Matching
    26
    

    View Slide

51. Use Case: Applicative Composition
    The Problem
    ▶ You have Options of different types.
    ▶ You want to apply a function which takes plain values.
    The Scalaz Solution
    (nameOpt |@| yearOpt |@| salaryOpt) { Employee(_, _, _) }
    26
    

    View Slide

52. Use Case: Applicative Composition
    The Problem
    ▶ You have Options of different types.
    ▶ You want to apply a function which takes plain values.
    The Scalaz Solution
    (nameOpt |@| yearOpt |@| salaryOpt) { Employee(_, _, _) }
    Downside: only goes up to 14
    26
    

    View Slide

53. Use Case: Applicative Composition
    The Problem
    ▶ You have Options of different types.
    ▶ You want to apply a function which takes plain values.
    The Unified Solution
    val makeEmployee = // ...
    sequence(nameOpt :: yearOpt :: salaryOpt :: HNil).
    map(makeEmployee)
    26
    

    View Slide

54. Use Case: Applicative Composition
    The Problem
    ▶ You have Options of different types.
    ▶ You want to apply a function which takes plain values.
    The Unified Solution
    val makeEmployee = // ...
    sequence(nameOpt :: yearOpt :: salaryOpt :: HNil).
    map(makeEmployee)
    26
    

    View Slide

55. Fin
    @larsr h
    larsrh.github.io
    typelevel.org/blog
    

    View Slide