Let’s Blend some Data Structures

How parametric polymorphism and Algebra can solve a real world problem

https://www.meetup.com/en-AU/AnnArborFP/events/243678261/

Transcript

1. October 2017
   @filippovitale
   Let’s Blend some
   Data Structures
   

   View Slide

2. Map(
   "k1" -> List(11, 12, 13),
   "k2" -> List(21, 22, 23),
   )
   

   View Slide

3. Map(
   "k1" -> List(11, 12, 13),
   "k2" -> List(21, 22, 23),
   )
   key value
   key value
   key value
   

   View Slide

4. Map("a" ->
   Map("aa" ->
   Map("aaa" ->
   Map("aaaa" -> List(1, 3),
   "aaab" -> List(2, 4)))))
   

   View Slide

5. View Slide

6. Map("a" ->
   Map("aa" ->
   Map("aaa" ->
   Map("aaaa" -> List(1, 3),
   "aaab" -> List(2, 4))))) ????? Map("a" ->
   Map("aa" ->
   Map("aaa" ->
   Map("aaaa" -> List(5, 7),
   "aaab" -> List(6, 8)))))
   

   View Slide

7. View Slide

8. View Slide

9. 3 Simple
   Data Structures
   

   View Slide

10. Seq Set Map
    package scala.collection
    

    View Slide

11. Seq Set Map
    package scala.collection
    v v v v
    

    View Slide

12. Seq Set Map
    package scala.collection
    valueA
    valueB
    valueC
    

    View Slide

13. Seq Set Map
    package scala.collection
    key value
    key value
    key value
    

    View Slide

14. Seq Set Map
    package scala.collection
    base trait for
    mutable & immutable
    implementations
    

    View Slide

15. “Mutability is an optimisation – perhaps premature”
    Seq Set Map
    base trait for
    mutable & immutable
    implementations
    

    View Slide

16. Let’s “Blend” something!
    

    View Slide

17. value
    value
    value
    Map Set
    key value
    key value
    key value
    List
    v v v v
    

    View Slide

18. Immutable Collection Hierarchy
    Traversable Seq List
    http://docs.scala-lang.org/tutorials/FAQ/collections.html
    def ++[B]
    (that:Traversable[B])
    :Traversable[B]
    

    View Slide

19. Blending Lists
    List(1, 2, 3) ++ List(4, 5, 6) == ???
    

    View Slide

20. Blending Appending Lists
    List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6)
    http://www.scala-lang.org/api/current/#scala.collection.immutable.List
    http://docs.scala-lang.org/overviews/collections/seqs.html
    

    View Slide

21. Immutable Collection Hierarchy
    Traversable Seq
    Set
    List
    HashSet
    http://docs.scala-lang.org/tutorials/FAQ/collections.html
    

    View Slide

22. Blending Sets
    Set(1, 2, 3) ++ Set(4, 5, 6) == ???
    

    View Slide

23. Blending Adding Sets
    http://www.scala-lang.org/api/current/#scala.collection.immutable.Set
    http://docs.scala-lang.org/overviews/collections/sets.html
    Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)
    

    View Slide

24. Blending Adding Sets
    Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)
    Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)
    

    View Slide

25. View Slide

26. View Slide

27. Immutable Collection Hierarchy
    Traversable Seq
    Set
    Map
    List
    HashSet
    HashMap
    http://docs.scala-lang.org/tutorials/FAQ/collections.html
    

    View Slide

28. Blending Adding Maps
    Map() ++ Map("a" -> 1) == Map("a" -> 1)
    http://www.scala-lang.org/api/current/#scala.collection.immutable.Map
    http://docs.scala-lang.org/overviews/collections/maps.html
    

    View Slide

29. Blending Adding Maps
    Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    http://www.scala-lang.org/api/current/#scala.collection.immutable.Map
    http://docs.scala-lang.org/overviews/collections/maps.html
    

    View Slide

30. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    1: Map("a" -> Set(1, 2))
    2: Map("a" -> Set(1, 2, 3))
    3: Map("a" -> Set(2, 3))
    4: RuntimeException
    5: Compiler Error
    What is the result of “blending” those Maps?
    

    View Slide

31. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    1: Map("a" -> Set(1, 2))
    2: Map("a" -> Set(1, 2, 3))
    3: Map("a" -> Set(2, 3))
    4: RuntimeException
    5: Compiler Error
    What is the result of “blending” those Maps?
    

    View Slide

32. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    1: Map("a" -> Set(1, 2))
    2: Map("a" -> Set(1, 2, 3))
    3: Map("a" -> Set(2, 3))
    4: RuntimeException
    5: Compiler Error
    What is the result of “blending” those Maps?
    

    View Slide

33. Map("a" -> Set(1, 2)) ??? Map("a" -> Set(2, 3))
    Map("a" -> Set(1, 2, 3))
    

    View Slide

34. View Slide

35. View Slide

36. Let’s implement
    the “Blend” logic
    

    View Slide

37. “a” Set(1, 2) “a” Set(2, 3)
    “a” Set(1, 2) ++ Set(2, 3)
    

    View Slide

38. “a” Set(1, 2) “a” Set(2, 3)
    “a” Set(1, 2, 3)
    

    View Slide

39. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]]
    

    View Slide

40. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    mb foreach { case (k, v) =>
    ???
    }
    }
    

    View Slide

41. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    ???
    }
    result.toMap
    }
    

    View Slide

42. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    }
    

    View Slide

43. View Slide

44. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    

    View Slide

45. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    

    View Slide

46. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    mb.foldLeft(ma) { ... }
    

    View Slide

47. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    

    View Slide

48. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    

    View Slide

49. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    }
    

    View Slide

50. “When you get used to immutable data,
    ya kinda forget how to use mutable data
    in a sensible way.” – Jessica Kerr
    

    View Slide

51. (ma /: mb) { case (result,(k, v)) =>
    result + (k -> {
    result.get(k) match {
    case Some(vr) => vr ++ v
    case None => v
    }
    }
    

    View Slide

52. (ma /: mb) { case (result,(k, v)) =>
    result + (k -> {
    result.get(k) match {
    case Some(vr) => vr ++ v
    case None => v
    }
    // .map(_ ++ v).getOrElse(v)
    // .some(_ ++ v).none(v)
    // .fold(v)(_ ++ v)
    // .cata(_ ++ v, v)
    }}
    “FP with Bananas, Lenses, Envelopes and Barbed Wire” – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125
    http://en.wikipedia.org/wiki/Catamorphism
    

    View Slide

53. (ma /: mb) { case (result, (k, v)) =>
    result + (k -> result.get(k).cata(_ ++ v, v))
    }
    mb foreach { case (k, v) =>
    result += (k -> result.get(k).cata(_ ++ v, v))
    }
    

    View Slide

54. (ma /: mb) { case (result, (k, v)) =>
    result + (k -> result.get(k).cata(_ ++ v, v))
    }
    mb foreach { case (k, v) =>
    result += (k -> result.get(k).cata(_ ++ v, v))
    }
    

    View Slide

55. What if now we have to use
    a slightly different Map?
    

    View Slide

56. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    

    View Slide

57. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = ???
    def blend(ma: Map[String, Map[Int, Set[Int]]],
    mb: Map[String, Map[Int, Set[Int]]])
    : Map[String, Map[Int, Set[Int]]] = ???
    

    View Slide

58. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    (ma /: mb) { case (result, (k, v)) =>
    result + (k -> result.get(k).cata(_ ++ v, v))
    }
    (ma /: mb) { case (result, (k, v)) =>
    result + ??? // { ??? => { ??? } }
    }
    

    View Slide

59. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    trait Blendable[A] {
    def blend(ma: A, mb: A): A
    }
    

    View Slide

60. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    trait Blendable[A] {
    def blend(ma: A, mb: A): A
    }
    new Blendable[...] { // ad-hoc polymorphism
    def blend(ma: ..., mb: ...): ... = ???
    }
    

    View Slide

61. Let’s think about what
    “Blendable” means
    

    View Slide

62. What blended “as expected”?
    List using the ++ binary operator
    Set using the ++ binary operator
    List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6)
    Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)
    

    View Slide

63. (List, ++)
    (Set, ++)
    (1 blend 2) == ???
    What about Int?
    

    View Slide

64. (List, ++)
    (Set, ++)
    (Int, +)
    (1 blend 2) == 1 + 2 == 3
    What about Int?
    

    View Slide

65. (List, ++)
    (Set, ++)
    (Int, +)
    (String, +)
    ("ab" blend "cd") == ("ab" + "cd") == "abcd"
    String
    

    View Slide

66. (List, ++)
    (Set, ++)
    (Int, +)
    (String, +)
    (Map[...], Blendable[...].blend)
    Blendable[Map[String, Set[Int]]].blend(ma, mb)
    Map[String, Set[Int]]
    

    View Slide

67. What can
    Mathematicians
    tell us here?
    

    View Slide

68. View Slide

69. What is a Semigroup?
    Michael Barr, Charles Wells - Category Theory for Computing Science
    Semigroups
    “A semigroup is a set S together with an
    associative binary operation m: S × S → S”
    

    View Slide

70. View Slide

71. Closure Property
    http://en.wikipedia.org/wiki/Closure_(mathematics)
    trait Semigroup[T] {
    def op(a: T, b: T): T
    }
    For all a, b in T, the result of the operation a ⋅ b is also in T:
    ∀a, b ∈ T : a∙b ∈ T
    

    View Slide

72. http://en.wikipedia.org/wiki/Closure_(mathematics)
    trait Semigroup[T] {
    def op(a: T, b: T): T
    }
    def op(a: Boolean, b: Boolean): Boolean
    def op(a: Int, b: Int): Boolean
    ✓
    ✘
    Closure Property
    

    View Slide

73. Associative Property
    http://en.wikipedia.org/wiki/Associative_property
    trait Semigroup[T] {
    def op(a: T, b: T): T
    }
    For all a, b, and c in T, the equation (a ⋅ b) ⋅ c = a ⋅ (b ⋅ c) holds:
    ∀a, b, c ∈ T : (a∙b)∙c = a∙(b∙c)
    (a op b) op c
    ==
    a op (b op c)
    

    View Slide

74. What is a Semigroup?
    Michael Barr, Charles Wells - Category Theory for Computing Science
    Semigroups
    “A semigroup is a set S together with an
    associative binary operation m: S × S → S”
    

    View Slide

75. Semigroups in Scalaz
    

    View Slide

76. View Slide

77. import scalaz.std.set._
    implicit def setSemigroup[A]:Semigroup[Set[A]] =
    new Semigroup[Set[A]] {
    def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2
    }
    

    View Slide

78. implicit def setSemigroup[A]:Semigroup[Set[A]] =
    new Semigroup[Set[A]] {
    def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2
    }
    op
    

    View Slide

79. import scalaz.syntax.semigroup._
    import scalaz.std.list._
    List(1, 2) |+| List(3, 4)
    res: List[Int] = List(1, 2, 3, 4)
    import scalaz.syntax.semigroup._
    import scalaz.std.set._
    Set(1, 2) |+| Set(2, 3)
    res: Set[Int] = Set(1, 2, 3)
    

    View Slide

80. import scalaz.syntax.semigroup._
    import scalaz.std.anyVal._
    1 |+| 2 |+| 3
    res: Int = 6
    import scalaz.syntax.semigroup._
    import scalaz.std.string._
    "a" |+| "b" |+| "c"
    res: String = "abc"
    

    View Slide

81. /**
    * A semigroup in type F must satisfy two laws:
    *
    * - '''closure''': `∀ a, b in F, append(a, b)` is also in `F`.
    * - '''associativity''': `∀ a, b, c` in `F`, the equation
    * `append(append(a, b), c) = append(a, append(b , c))` holds.
    */
    trait SemigroupLaw {
    def associative(f1: F, f2: F, f3: F)(implicit F: Equal[F]): Boolean =
    F.equal(append(f1, append(f2, f3)), append(append(f1, f2), f3))
    }
    

    View Slide

82. import scalaz.scalacheck.ScalazProperties._
    import scalaz.std.anyVal._
    semigroup.laws[Int].check
    + semigroup.associative: OK, passed 100 tests.
    

    View Slide

83. semigroup.laws[String].check
    semigroup.laws[Set[Int]].check
    semigroup.laws[List[String]].check
    semigroup.laws[Map[Int, Int]].check
    + semigroup.associative: OK, passed 100 tests.
    + semigroup.associative: OK, passed 100 tests.
    + semigroup.associative: OK, passed 100 tests.
    + semigroup.associative: OK, passed 100 tests.
    

    View Slide

84. Does Map[_, Set[Int]] have
    a Semigroup instance?
    

    View Slide

85. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)
    res: Map[…] = Map(a -> 3, b -> 4)
    “Some data structures form interesting semigroups as long as
    the types of the elements they contain also form semigroups.”
    

    View Slide

86. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)
    res: Map[…] = Map(a -> 3, b -> 4)
    “Some data structures form interesting semigroups as long as
    the types of the elements they contain also form semigroups.”
    Adapted from: Functional Programming in Scala - Part 3 - Chapter 10 Monoids
    

    View Slide

87. View Slide

88. import scalaz.scalacheck.ScalazProperties._
    semigroup.laws[Map[String, Set[Int]]].check
    + semigroup.associative: OK, passed 100 tests.
    ✓
    

    View Slide

89. Map("a" -> Set(1, 2)) |+| Map("a" -> Set(2, 3))
    res: Map[…] = Map(a -> Set(1, 2, 3))
    ✓
    

    View Slide

90. Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(1, 3),
    "aaab" -> List(2, 4))))) |+| Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(5, 7),
    "aaab" -> List(6, 8)))))
    Map(a->Map(aa->Map(aaa->Map(aaaa->List(1, 3, 5, 7), aaab->List(2, 4, 6, 8)))))
    

    View Slide

91. October 2017
    @filippovitale
    $ tail -f questions
    

    View Slide

92. Benchmarking
    

    View Slide

93. “Experience indicates that nearly everybody
    has the wrong idea about the real bottlenecks
    in his programs” – Donald Knuth
    Computer programming as an art (1974) – http://dl.acm.org/citation.cfm?id=361612
    

    View Slide

94. Map[String, Set[Int]]
    

    View Slide

95. View Slide