$30 off During Our Annual Pro Sale. View Details »

Let’s Blend some Data Structures

Let’s Blend some Data Structures

How parametric polymorphism and Algebra can solve a real world problem

https://www.meetup.com/en-AU/AnnArborFP/events/243678261/

Filippo Vitale

October 24, 2017
Tweet

More Decks by Filippo Vitale

Other Decks in Programming

Transcript

  1. October 2017
    @filippovitale
    Let’s Blend some
    Data Structures

    View Slide

  2. Map(
    "k1" -> List(11, 12, 13),
    "k2" -> List(21, 22, 23),
    )

    View Slide

  3. Map(
    "k1" -> List(11, 12, 13),
    "k2" -> List(21, 22, 23),
    )
    key value
    key value
    key value

    View Slide

  4. Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(1, 3),
    "aaab" -> List(2, 4)))))

    View Slide

  5. View Slide

  6. Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(1, 3),
    "aaab" -> List(2, 4))))) ????? Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(5, 7),
    "aaab" -> List(6, 8)))))

    View Slide

  7. View Slide

  8. View Slide

  9. 3 Simple
    Data Structures

    View Slide

  10. Seq Set Map
    package scala.collection

    View Slide

  11. Seq Set Map
    package scala.collection
    v v v v

    View Slide

  12. Seq Set Map
    package scala.collection
    valueA
    valueB
    valueC

    View Slide

  13. Seq Set Map
    package scala.collection
    key value
    key value
    key value

    View Slide

  14. Seq Set Map
    package scala.collection
    base trait for
    mutable & immutable
    implementations

    View Slide

  15. “Mutability is an optimisation – perhaps premature”
    Seq Set Map
    base trait for
    mutable & immutable
    implementations

    View Slide

  16. Let’s “Blend” something!

    View Slide

  17. value
    value
    value
    Map Set
    key value
    key value
    key value
    List
    v v v v

    View Slide

  18. Immutable Collection Hierarchy
    Traversable Seq List
    http://docs.scala-lang.org/tutorials/FAQ/collections.html
    def ++[B]
    (that:Traversable[B])
    :Traversable[B]

    View Slide

  19. Blending Lists
    List(1, 2, 3) ++ List(4, 5, 6) == ???

    View Slide

  20. Blending Appending Lists
    List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6)
    http://www.scala-lang.org/api/current/#scala.collection.immutable.List
    http://docs.scala-lang.org/overviews/collections/seqs.html

    View Slide

  21. Immutable Collection Hierarchy
    Traversable Seq
    Set
    List
    HashSet
    http://docs.scala-lang.org/tutorials/FAQ/collections.html

    View Slide

  22. Blending Sets
    Set(1, 2, 3) ++ Set(4, 5, 6) == ???

    View Slide

  23. Blending Adding Sets
    http://www.scala-lang.org/api/current/#scala.collection.immutable.Set
    http://docs.scala-lang.org/overviews/collections/sets.html
    Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)

    View Slide

  24. Blending Adding Sets
    Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)
    Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)

    View Slide

  25. View Slide

  26. View Slide

  27. Immutable Collection Hierarchy
    Traversable Seq
    Set
    Map
    List
    HashSet
    HashMap
    http://docs.scala-lang.org/tutorials/FAQ/collections.html

    View Slide

  28. Blending Adding Maps
    Map() ++ Map("a" -> 1) == Map("a" -> 1)
    http://www.scala-lang.org/api/current/#scala.collection.immutable.Map
    http://docs.scala-lang.org/overviews/collections/maps.html

    View Slide

  29. Blending Adding Maps
    Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    http://www.scala-lang.org/api/current/#scala.collection.immutable.Map
    http://docs.scala-lang.org/overviews/collections/maps.html

    View Slide

  30. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    1: Map("a" -> Set(1, 2))
    2: Map("a" -> Set(1, 2, 3))
    3: Map("a" -> Set(2, 3))
    4: RuntimeException
    5: Compiler Error
    What is the result of “blending” those Maps?

    View Slide

  31. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    1: Map("a" -> Set(1, 2))
    2: Map("a" -> Set(1, 2, 3))
    3: Map("a" -> Set(2, 3))
    4: RuntimeException
    5: Compiler Error
    What is the result of “blending” those Maps?

    View Slide

  32. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
    1: Map("a" -> Set(1, 2))
    2: Map("a" -> Set(1, 2, 3))
    3: Map("a" -> Set(2, 3))
    4: RuntimeException
    5: Compiler Error
    What is the result of “blending” those Maps?

    View Slide

  33. Map("a" -> Set(1, 2)) ??? Map("a" -> Set(2, 3))
    Map("a" -> Set(1, 2, 3))

    View Slide

  34. View Slide

  35. View Slide

  36. Let’s implement
    the “Blend” logic

    View Slide

  37. “a” Set(1, 2) “a” Set(2, 3)
    “a” Set(1, 2) ++ Set(2, 3)

    View Slide

  38. “a” Set(1, 2) “a” Set(2, 3)
    “a” Set(1, 2, 3)

    View Slide

  39. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]]

    View Slide

  40. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    mb foreach { case (k, v) =>
    ???
    }
    }

    View Slide

  41. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    ???
    }
    result.toMap
    }

    View Slide

  42. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    }

    View Slide

  43. View Slide

  44. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap

    View Slide

  45. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap

    View Slide

  46. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap
    mb.foldLeft(ma) { ... }

    View Slide

  47. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap

    View Slide

  48. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    }
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    val result = mutable.Map() ++ ma
    mb foreach { case (k, v) =>
    if (result.contains(k))
    result += k -> (result(k) ++ v)
    else
    result += k -> v
    }
    result.toMap

    View Slide

  49. def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = {
    (ma /: mb) { case (result,(k, v)) =>
    if (result.contains(k))
    result + (k -> (result(k) ++ v))
    else
    result + (k -> v)
    }
    }

    View Slide

  50. “When you get used to immutable data,
    ya kinda forget how to use mutable data
    in a sensible way.” – Jessica Kerr

    View Slide

  51. (ma /: mb) { case (result,(k, v)) =>
    result + (k -> {
    result.get(k) match {
    case Some(vr) => vr ++ v
    case None => v
    }
    }

    View Slide

  52. (ma /: mb) { case (result,(k, v)) =>
    result + (k -> {
    result.get(k) match {
    case Some(vr) => vr ++ v
    case None => v
    }
    // .map(_ ++ v).getOrElse(v)
    // .some(_ ++ v).none(v)
    // .fold(v)(_ ++ v)
    // .cata(_ ++ v, v)
    }}
    “FP with Bananas, Lenses, Envelopes and Barbed Wire” – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125
    http://en.wikipedia.org/wiki/Catamorphism

    View Slide

  53. (ma /: mb) { case (result, (k, v)) =>
    result + (k -> result.get(k).cata(_ ++ v, v))
    }
    mb foreach { case (k, v) =>
    result += (k -> result.get(k).cata(_ ++ v, v))
    }

    View Slide

  54. (ma /: mb) { case (result, (k, v)) =>
    result + (k -> result.get(k).cata(_ ++ v, v))
    }
    mb foreach { case (k, v) =>
    result += (k -> result.get(k).cata(_ ++ v, v))
    }

    View Slide

  55. What if now we have to use
    a slightly different Map?

    View Slide

  56. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]

    View Slide

  57. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    def blend(ma: Map[String, Set[Int]],
    mb: Map[String, Set[Int]])
    : Map[String, Set[Int]] = ???
    def blend(ma: Map[String, Map[Int, Set[Int]]],
    mb: Map[String, Map[Int, Set[Int]]])
    : Map[String, Map[Int, Set[Int]]] = ???

    View Slide

  58. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    (ma /: mb) { case (result, (k, v)) =>
    result + (k -> result.get(k).cata(_ ++ v, v))
    }
    (ma /: mb) { case (result, (k, v)) =>
    result + ??? // { ??? => { ??? } }
    }

    View Slide

  59. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    trait Blendable[A] {
    def blend(ma: A, mb: A): A
    }

    View Slide

  60. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
    trait Blendable[A] {
    def blend(ma: A, mb: A): A
    }
    new Blendable[...] { // ad-hoc polymorphism
    def blend(ma: ..., mb: ...): ... = ???
    }

    View Slide

  61. Let’s think about what
    “Blendable” means

    View Slide

  62. What blended “as expected”?
    List using the ++ binary operator
    Set using the ++ binary operator
    List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6)
    Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)

    View Slide

  63. (List, ++)
    (Set, ++)
    (1 blend 2) == ???
    What about Int?

    View Slide

  64. (List, ++)
    (Set, ++)
    (Int, +)
    (1 blend 2) == 1 + 2 == 3
    What about Int?

    View Slide

  65. (List, ++)
    (Set, ++)
    (Int, +)
    (String, +)
    ("ab" blend "cd") == ("ab" + "cd") == "abcd"
    String

    View Slide

  66. (List, ++)
    (Set, ++)
    (Int, +)
    (String, +)
    (Map[...], Blendable[...].blend)
    Blendable[Map[String, Set[Int]]].blend(ma, mb)
    Map[String, Set[Int]]

    View Slide

  67. What can
    Mathematicians
    tell us here?

    View Slide

  68. View Slide

  69. What is a Semigroup?
    Michael Barr, Charles Wells - Category Theory for Computing Science
    Semigroups
    “A semigroup is a set S together with an
    associative binary operation m: S × S → S”

    View Slide

  70. View Slide

  71. Closure Property
    http://en.wikipedia.org/wiki/Closure_(mathematics)
    trait Semigroup[T] {
    def op(a: T, b: T): T
    }
    For all a, b in T, the result of the operation a ⋅ b is also in T:
    ∀a, b ∈ T : a∙b ∈ T

    View Slide

  72. http://en.wikipedia.org/wiki/Closure_(mathematics)
    trait Semigroup[T] {
    def op(a: T, b: T): T
    }
    def op(a: Boolean, b: Boolean): Boolean
    def op(a: Int, b: Int): Boolean


    Closure Property

    View Slide

  73. Associative Property
    http://en.wikipedia.org/wiki/Associative_property
    trait Semigroup[T] {
    def op(a: T, b: T): T
    }
    For all a, b, and c in T, the equation (a ⋅ b) ⋅ c = a ⋅ (b ⋅ c) holds:
    ∀a, b, c ∈ T : (a∙b)∙c = a∙(b∙c)
    (a op b) op c
    ==
    a op (b op c)

    View Slide

  74. What is a Semigroup?
    Michael Barr, Charles Wells - Category Theory for Computing Science
    Semigroups
    “A semigroup is a set S together with an
    associative binary operation m: S × S → S”

    View Slide

  75. Semigroups in Scalaz

    View Slide

  76. View Slide

  77. import scalaz.std.set._
    implicit def setSemigroup[A]:Semigroup[Set[A]] =
    new Semigroup[Set[A]] {
    def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2
    }

    View Slide

  78. implicit def setSemigroup[A]:Semigroup[Set[A]] =
    new Semigroup[Set[A]] {
    def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2
    }
    op

    View Slide

  79. import scalaz.syntax.semigroup._
    import scalaz.std.list._
    List(1, 2) |+| List(3, 4)
    res: List[Int] = List(1, 2, 3, 4)
    import scalaz.syntax.semigroup._
    import scalaz.std.set._
    Set(1, 2) |+| Set(2, 3)
    res: Set[Int] = Set(1, 2, 3)

    View Slide

  80. import scalaz.syntax.semigroup._
    import scalaz.std.anyVal._
    1 |+| 2 |+| 3
    res: Int = 6
    import scalaz.syntax.semigroup._
    import scalaz.std.string._
    "a" |+| "b" |+| "c"
    res: String = "abc"

    View Slide

  81. /**
    * A semigroup in type F must satisfy two laws:
    *
    * - '''closure''': `∀ a, b in F, append(a, b)` is also in `F`.
    * - '''associativity''': `∀ a, b, c` in `F`, the equation
    * `append(append(a, b), c) = append(a, append(b , c))` holds.
    */
    trait SemigroupLaw {
    def associative(f1: F, f2: F, f3: F)(implicit F: Equal[F]): Boolean =
    F.equal(append(f1, append(f2, f3)), append(append(f1, f2), f3))
    }

    View Slide

  82. import scalaz.scalacheck.ScalazProperties._
    import scalaz.std.anyVal._
    semigroup.laws[Int].check
    + semigroup.associative: OK, passed 100 tests.

    View Slide

  83. semigroup.laws[String].check
    semigroup.laws[Set[Int]].check
    semigroup.laws[List[String]].check
    semigroup.laws[Map[Int, Int]].check
    + semigroup.associative: OK, passed 100 tests.
    + semigroup.associative: OK, passed 100 tests.
    + semigroup.associative: OK, passed 100 tests.
    + semigroup.associative: OK, passed 100 tests.

    View Slide

  84. Does Map[_, Set[Int]] have
    a Semigroup instance?

    View Slide

  85. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)
    res: Map[…] = Map(a -> 3, b -> 4)
    “Some data structures form interesting semigroups as long as
    the types of the elements they contain also form semigroups.”

    View Slide

  86. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)
    res: Map[…] = Map(a -> 3, b -> 4)
    “Some data structures form interesting semigroups as long as
    the types of the elements they contain also form semigroups.”
    Adapted from: Functional Programming in Scala - Part 3 - Chapter 10 Monoids

    View Slide

  87. View Slide

  88. import scalaz.scalacheck.ScalazProperties._
    semigroup.laws[Map[String, Set[Int]]].check
    + semigroup.associative: OK, passed 100 tests.

    View Slide

  89. Map("a" -> Set(1, 2)) |+| Map("a" -> Set(2, 3))
    res: Map[…] = Map(a -> Set(1, 2, 3))

    View Slide

  90. Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(1, 3),
    "aaab" -> List(2, 4))))) |+| Map("a" ->
    Map("aa" ->
    Map("aaa" ->
    Map("aaaa" -> List(5, 7),
    "aaab" -> List(6, 8)))))
    Map(a->Map(aa->Map(aaa->Map(aaaa->List(1, 3, 5, 7), aaab->List(2, 4, 6, 8)))))

    View Slide

  91. October 2017
    @filippovitale
    $ tail -f questions

    View Slide

  92. Benchmarking

    View Slide

  93. “Experience indicates that nearly everybody
    has the wrong idea about the real bottlenecks
    in his programs” – Donald Knuth
    Computer programming as an art (1974) – http://dl.acm.org/citation.cfm?id=361612

    View Slide

  94. Map[String, Set[Int]]

    View Slide

  95. View Slide