Combining Scalaz and Shapeless for Great Good

Scalaz is an extension to the Scala standard library providing purely functional data structures and useful abstractions. Shapeless is a library for generic programming and pushes Scala’s type system quite far. Recently, there have been some developments in providing interoperability between both libraries, which turns out to be massively useful.

In this talk, I showcased some of the cool available techniques, including abstracting over heterogeneous data and the arity of functions, using macros to generate type class instances, and (de)serializing data with zero boilerplate.

Transcript

1. Combining Scalaz and Shapeless for Great Good Lars Hupel Technische Universität München October 25th, 2013
2. Agenda 1 Deriving Instances 2 Automatic Serialization 3 Functions with Arbitrary Arity 2
3. Type Classes in Scala Bad News Scala offers no native support for type classes 3
4. Type Classes in Scala Bad News Scala offers no native support for type classes Good News ... but they can be encoded 3
5. Type Classes in Scala Example: Ordering trait Ordering[A] { def compare(x: A, y: A): Order } ▶ type classes are (first-class) traits 4
6. Type Classes in Scala Example: Ordering trait Ordering[A] { def compare(x: A, y: A): Order } ▶ type classes are (first-class) traits ▶ can use arbitrary language features 4
7. Type Classes in Scala Example: Ordering trait Ordering[A] { def compare(x: A, y: A): Order def equal(x: A, y: A) = compare(x, y) == Order.EQ } ▶ type classes are (first-class) traits ▶ can use arbitrary language features e.g. default implementations 4
8. Type Classes in Scala Example: Ordering implicit object IntHasOrdering extends Ordering[Int] { def compare(x: Int, y: Int) = if (x < y) Order.LT else if (x > y) Order.GT else Order.EQ } ▶ instances are (first-class) implicits 5
9. Type Classes in Scala Example: Ordering implicit object IntHasOrdering extends Ordering[Int] { def compare(x: Int, y: Int) = if (x < y) Order.LT else if (x > y) Order.GT else Order.EQ } ▶ instances are (first-class) implicits ▶ can use arbitrary language features 5
10. Type Classes in Scala Example: Ordering implicit object IntHasOrdering extends Ordering[Int] { def compare(x: Int, y: Int) = if (x < y) Order.LT else if (x > y) Order.GT else Order.EQ } ▶ instances are (first-class) implicits ▶ can use arbitrary language features e.g. macros 5
11. Implementing Instances class Vector2D(val x: Int, val y: Int) class Vector3D(val x: Int, val y: Int, val z: Int) ▶ we want: default instances for ... ▶ Semigroup (pointwise addition) ▶ Ordering (lexicographic order) ▶ and more? 6
12. Implementing Instances class Vector2D(val x: Int, val y: Int) class Vector3D(val x: Int, val y: Int, val z: Int) ▶ we want: default instances for ... ▶ Semigroup (pointwise addition) ▶ Ordering (lexicographic order) ▶ and more? ▶ very repetitive to implement by hand ▶ need a mechanism to automate that task 6
13. Implementing Instances class Vector2D(val x: Int, val y: Int) class Vector3D(val x: Int, val y: Int, val z: Int) ▶ Scala already provides some automation 6
14. Implementing Instances case class Vector2D(x: Int, y: Int) case class Vector3D(x: Int, y: Int, z: Int) ▶ Scala already provides some automation 6
15. Implementing Instances case class Vector2D(x: Int, y: Int) case class Vector3D(x: Int, y: Int, z: Int) ▶ Scala already provides some automation ▶ generates: ▶ toString ▶ equals ▶ hashCode 6
16. Implementing Instances case class Vector2D(x: Int, y: Int) case class Vector3D(x: Int, y: Int, z: Int) ▶ Scala already provides some automation ▶ generates: ▶ toString ▶ equals ▶ hashCode ▶ Problem: baked into the compiler 6
17. Implementing Instances case class Vector2D(x: Int, y: Int) case class Vector3D(x: Int, y: Int, z: Int) ▶ Scala already provides some automation ▶ generates: ▶ toString ▶ equals ▶ hashCode ▶ Problem: baked into the compiler 6
18. Goals ... for library designers 1. Define a type class. 2. Implement some base instances. 3. Declare how to combine instances. 7
19. Goals ... for library designers 1. Define a type class. 2. Implement some base instances. 3. Declare how to combine instances. ... for library users 1. Define some types. 7
20. The Essence of Data Types ▶ we are dealing with algebraic data types ▶ ... consisting of one or more cases ▶ ... consisting of zero or more fields 8
21. The Essence of Data Types Example: Lists sealed trait List case class Cons(head: Int, tail: List) extends List case class Nil() extends List 9
22. The Essence of Data Types Example: Lists sealed trait List case class Cons(head: Int, tail: List) extends List case class Nil() extends List Cons(1, Cons(2, Nil())) . . Cons . 1 . Cons . 2 . Nil 9
23. Abstract Representation Think: Serialization 10
24. Abstract Representation Think: Serialization ... but not with bits, rather with products and sums 10
25. Abstract Representation Examples Data type case class Vector3D(x: Int, y: Int, z: Int) Representation type Repr = (Int, Int, Int) 11
26. Abstract Representation Examples Data type sealed trait Shape case class Circle(radius: Int) extends Shape case class Rectangle(height: Int, width: Int) extends Shape Representation type Repr = Either[Int, (Int, Int)] 11
27. Type Classes as Type Constructors Realization: type classes are regular types 12
28. Type Classes as Type Constructors Realization: type classes are regular types ... Scala’s encoding makes abstraction possible 12
29. Composing Instances Example: Ordering Given val A: Ordering[A] val B: Ordering[B] ... we can implement val prod: Ordering[(A, B)] val sum: Ordering[Either[A, B]] 13
30. Composing Instances Example: Ordering Given val A: Ordering[A] val B: Ordering[B] ... we can implement val prod: Ordering[(A, B)] val sum: Ordering[Either[A, B]] Question: Can we do the same for Semigroup? 13
31. Composing Instances Example: Ordering Demo 14
32. Putting it together Macros! Input Data type type T Type class type C[_] Output Instance val instance: C[T] 15
33. Putting it together Macros! Input Data type type T Type class type C[_] Output Instance val instance: C[T] 15
34. Problems ▶ base instances are resolved via regular implicit search ▶ What happens for (mutually) recursive data types? 16
35. Problems ▶ base instances are resolved via regular implicit search ▶ What happens for (mutually) recursive data types? ▶ solution: make macro aware of recursion (aka tying the knot) ▶ self recursion: easy, use this instead of implicitly ▶ mutual recursion: require user setup ▶ indirect recursion: ongoing ... 16
36. Library Support Shapeless · Scalaz · Spire typelevel.org github.com/typelevel 17
37. Agenda 1 Deriving Instances 2 Automatic Serialization 3 Functions with Arbitrary Arity 18
38. Automatic Serialization in Java ... just extend Serializable, what could possibly go wrong? 19
39. Use Case: Serialization type ByteString = // ... def encode(a: A): ByteString def decode(bs: ByteString): Option[(A, ByteString)] 20
40. Use Case: Serialization ... for library designers 1. Define a type class. ✓ 2. Implement some base instances. ✓ 3. Declare how to combine instances. ✓ 21
41. Use Case: Serialization ... for library users 1. Define some types. 22
42. Use Case: Serialization ... for library users 1. Define some types. Demo 22
43. Use Case: Serialization ... for library users 1. Define some types. Demo 22
44. Agenda 1 Deriving Instances 2 Automatic Serialization 3 Functions with Arbitrary Arity 23
45. Recap: Representation Data type case class Employee(name: String, years: Int, salary: Float) Representation type Repr = (String, Int, Float) 24
46. Recap: Representation Data type case class Employee(name: String, years: Int, salary: Float) Representation type Repr = String :: Int :: Float :: HNil 24
47. Heterogeneous Lists Think: tuples, but with arbitrary arity 25
48. Heterogeneous Lists Think: tuples, but with arbitrary arity ... makes abstraction over arity possible 25
49. Use Case: Applicative Composition The Problem ▶ You have Options of different types. ▶ You want to apply a function which takes plain values. 26
50. Use Case: Applicative Composition The Problem ▶ You have Options of different types. ▶ You want to apply a function which takes plain values. The Naïve Solution Pattern Matching 26
51. Use Case: Applicative Composition The Problem ▶ You have Options of different types. ▶ You want to apply a function which takes plain values. The Scalaz Solution (nameOpt |@| yearOpt |@| salaryOpt) { Employee(_, _, _) } 26
52. Use Case: Applicative Composition The Problem ▶ You have Options of different types. ▶ You want to apply a function which takes plain values. The Scalaz Solution (nameOpt |@| yearOpt |@| salaryOpt) { Employee(_, _, _) } Downside: only goes up to 14 26
53. Use Case: Applicative Composition The Problem ▶ You have Options of different types. ▶ You want to apply a function which takes plain values. The Unified Solution val makeEmployee = // ... sequence(nameOpt :: yearOpt :: salaryOpt :: HNil). map(makeEmployee) 26
54. Use Case: Applicative Composition The Problem ▶ You have Options of different types. ▶ You want to apply a function which takes plain values. The Unified Solution val makeEmployee = // ... sequence(nameOpt :: yearOpt :: salaryOpt :: HNil). map(makeEmployee) 26
55. Fin @larsr h larsrh.github.io typelevel.org/blog