Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Let’s Blend some Data Structures

Let’s Blend some Data Structures

How parametric polymorphism and Algebra can solve a real world problem

https://www.meetup.com/en-AU/AnnArborFP/events/243678261/

2fa9df92d8d7eba3e17207c86f953be3?s=128

Filippo Vitale

October 24, 2017
Tweet

Transcript

  1. October 2017 @filippovitale Let’s Blend some Data Structures

  2. Map( "k1" -> List(11, 12, 13), "k2" -> List(21, 22,

    23), )
  3. Map( "k1" -> List(11, 12, 13), "k2" -> List(21, 22,

    23), ) key value key value key value
  4. Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3),

    "aaab" -> List(2, 4)))))
  5. None
  6. Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3),

    "aaab" -> List(2, 4))))) ????? Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(5, 7), "aaab" -> List(6, 8)))))
  7. None
  8. None
  9. 3 Simple Data Structures

  10. Seq Set Map package scala.collection

  11. Seq Set Map package scala.collection v v v v

  12. Seq Set Map package scala.collection valueA valueB valueC

  13. Seq Set Map package scala.collection key value key value key

    value
  14. Seq Set Map package scala.collection base trait for mutable &

    immutable implementations
  15. “Mutability is an optimisation – perhaps premature” Seq Set Map

    base trait for mutable & immutable implementations
  16. Let’s “Blend” something!

  17. value value value Map Set key value key value key

    value List v v v v
  18. Immutable Collection Hierarchy Traversable Seq List http://docs.scala-lang.org/tutorials/FAQ/collections.html def ++[B] (that:Traversable[B])

    :Traversable[B]
  19. Blending Lists List(1, 2, 3) ++ List(4, 5, 6) ==

    ???
  20. Blending Appending Lists List(1, 2, 3) ++ List(4, 5, 6)

    == List(1, 2, 3, 4, 5, 6) http://www.scala-lang.org/api/current/#scala.collection.immutable.List http://docs.scala-lang.org/overviews/collections/seqs.html
  21. Immutable Collection Hierarchy Traversable Seq Set List HashSet http://docs.scala-lang.org/tutorials/FAQ/collections.html

  22. Blending Sets Set(1, 2, 3) ++ Set(4, 5, 6) ==

    ???
  23. Blending Adding Sets http://www.scala-lang.org/api/current/#scala.collection.immutable.Set http://docs.scala-lang.org/overviews/collections/sets.html Set(1, 2, 3) ++ Set(4,

    5, 6) == Set(5, 1, 6, 2, 3, 4)
  24. Blending Adding Sets Set(1, 2, 3) ++ Set(4, 5, 6)

    == Set(5, 1, 6, 2, 3, 4) Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)
  25. None
  26. None
  27. Immutable Collection Hierarchy Traversable Seq Set Map List HashSet HashMap

    http://docs.scala-lang.org/tutorials/FAQ/collections.html
  28. Blending Adding Maps Map() ++ Map("a" -> 1) == Map("a"

    -> 1) http://www.scala-lang.org/api/current/#scala.collection.immutable.Map http://docs.scala-lang.org/overviews/collections/maps.html
  29. Blending Adding Maps Map("a" -> Set(1, 2)) ++ Map("a" ->

    Set(2, 3)) == ??? http://www.scala-lang.org/api/current/#scala.collection.immutable.Map http://docs.scala-lang.org/overviews/collections/maps.html
  30. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) ==

    ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?
  31. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) ==

    ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?
  32. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) ==

    ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?
  33. Map("a" -> Set(1, 2)) ??? Map("a" -> Set(2, 3)) Map("a"

    -> Set(1, 2, 3))
  34. None
  35. None
  36. Let’s implement the “Blend” logic

  37. “a” Set(1, 2) “a” Set(2, 3) “a” Set(1, 2) ++

    Set(2, 3)
  38. “a” Set(1, 2) “a” Set(2, 3) “a” Set(1, 2, 3)

  39. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

  40. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { mb foreach { case (k, v) => ??? } }
  41. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { val result = mutable.Map() ++ ma mb foreach { case (k, v) => ??? } result.toMap }
  42. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap }
  43. None
  44. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
  45. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
  46. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap mb.foldLeft(ma) { ... }
  47. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
  48. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
  49. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

    = { (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } }
  50. “When you get used to immutable data, ya kinda forget

    how to use mutable data in a sensible way.” – Jessica Kerr
  51. (ma /: mb) { case (result,(k, v)) => result +

    (k -> { result.get(k) match { case Some(vr) => vr ++ v case None => v } }
  52. (ma /: mb) { case (result,(k, v)) => result +

    (k -> { result.get(k) match { case Some(vr) => vr ++ v case None => v } // .map(_ ++ v).getOrElse(v) // .some(_ ++ v).none(v) // .fold(v)(_ ++ v) // .cata(_ ++ v, v) }} “FP with Bananas, Lenses, Envelopes and Barbed Wire” – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125 http://en.wikipedia.org/wiki/Catamorphism
  53. (ma /: mb) { case (result, (k, v)) => result

    + (k -> result.get(k).cata(_ ++ v, v)) } mb foreach { case (k, v) => result += (k -> result.get(k).cata(_ ++ v, v)) }
  54. (ma /: mb) { case (result, (k, v)) => result

    + (k -> result.get(k).cata(_ ++ v, v)) } mb foreach { case (k, v) => result += (k -> result.get(k).cata(_ ++ v, v)) }
  55. What if now we have to use a slightly different

    Map?
  56. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]

  57. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] def blend(ma: Map[String, Set[Int]], mb:

    Map[String, Set[Int]]) : Map[String, Set[Int]] = ??? def blend(ma: Map[String, Map[Int, Set[Int]]], mb: Map[String, Map[Int, Set[Int]]]) : Map[String, Map[Int, Set[Int]]] = ???
  58. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] (ma /: mb) { case

    (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } (ma /: mb) { case (result, (k, v)) => result + ??? // { ??? => { ??? } } }
  59. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] trait Blendable[A] { def blend(ma:

    A, mb: A): A }
  60. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] trait Blendable[A] { def blend(ma:

    A, mb: A): A } new Blendable[...] { // ad-hoc polymorphism def blend(ma: ..., mb: ...): ... = ??? }
  61. Let’s think about what “Blendable” means

  62. What blended “as expected”? List using the ++ binary operator

    Set using the ++ binary operator List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6) Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)
  63. (List, ++) (Set, ++) (1 blend 2) == ??? What

    about Int?
  64. (List, ++) (Set, ++) (Int, +) (1 blend 2) ==

    1 + 2 == 3 What about Int?
  65. (List, ++) (Set, ++) (Int, +) (String, +) ("ab" blend

    "cd") == ("ab" + "cd") == "abcd" String
  66. (List, ++) (Set, ++) (Int, +) (String, +) (Map[...], Blendable[...].blend)

    Blendable[Map[String, Set[Int]]].blend(ma, mb) Map[String, Set[Int]]
  67. What can Mathematicians tell us here?

  68. None
  69. What is a Semigroup? Michael Barr, Charles Wells - Category

    Theory for Computing Science Semigroups “A semigroup is a set S together with an associative binary operation m: S × S → S”
  70. None
  71. Closure Property http://en.wikipedia.org/wiki/Closure_(mathematics) trait Semigroup[T] { def op(a: T, b:

    T): T } For all a, b in T, the result of the operation a ⋅ b is also in T: ∀a, b ∈ T : a∙b ∈ T
  72. http://en.wikipedia.org/wiki/Closure_(mathematics) trait Semigroup[T] { def op(a: T, b: T): T

    } def op(a: Boolean, b: Boolean): Boolean def op(a: Int, b: Int): Boolean ✓ ✘ Closure Property
  73. Associative Property http://en.wikipedia.org/wiki/Associative_property trait Semigroup[T] { def op(a: T, b:

    T): T } For all a, b, and c in T, the equation (a ⋅ b) ⋅ c = a ⋅ (b ⋅ c) holds: ∀a, b, c ∈ T : (a∙b)∙c = a∙(b∙c) (a op b) op c == a op (b op c)
  74. What is a Semigroup? Michael Barr, Charles Wells - Category

    Theory for Computing Science Semigroups “A semigroup is a set S together with an associative binary operation m: S × S → S”
  75. Semigroups in Scalaz

  76. None
  77. import scalaz.std.set._ implicit def setSemigroup[A]:Semigroup[Set[A]] = new Semigroup[Set[A]] { def

    append(f1: Set[A], f2: => Set[A]) = f1 ++ f2 }
  78. implicit def setSemigroup[A]:Semigroup[Set[A]] = new Semigroup[Set[A]] { def append(f1: Set[A],

    f2: => Set[A]) = f1 ++ f2 } op
  79. import scalaz.syntax.semigroup._ import scalaz.std.list._ List(1, 2) |+| List(3, 4) res:

    List[Int] = List(1, 2, 3, 4) import scalaz.syntax.semigroup._ import scalaz.std.set._ Set(1, 2) |+| Set(2, 3) res: Set[Int] = Set(1, 2, 3)
  80. import scalaz.syntax.semigroup._ import scalaz.std.anyVal._ 1 |+| 2 |+| 3 res:

    Int = 6 import scalaz.syntax.semigroup._ import scalaz.std.string._ "a" |+| "b" |+| "c" res: String = "abc"
  81. /** * A semigroup in type F must satisfy two

    laws: * * - '''closure''': `∀ a, b in F, append(a, b)` is also in `F`. * - '''associativity''': `∀ a, b, c` in `F`, the equation * `append(append(a, b), c) = append(a, append(b , c))` holds. */ trait SemigroupLaw { def associative(f1: F, f2: F, f3: F)(implicit F: Equal[F]): Boolean = F.equal(append(f1, append(f2, f3)), append(append(f1, f2), f3)) }
  82. import scalaz.scalacheck.ScalazProperties._ import scalaz.std.anyVal._ semigroup.laws[Int].check + semigroup.associative: OK, passed 100

    tests.
  83. semigroup.laws[String].check semigroup.laws[Set[Int]].check semigroup.laws[List[String]].check semigroup.laws[Map[Int, Int]].check + semigroup.associative: OK, passed 100

    tests. + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests.
  84. Does Map[_, Set[Int]] have a Semigroup instance?

  85. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)

    res: Map[…] = Map(a -> 3, b -> 4) “Some data structures form interesting semigroups as long as the types of the elements they contain also form semigroups.”
  86. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)

    res: Map[…] = Map(a -> 3, b -> 4) “Some data structures form interesting semigroups as long as the types of the elements they contain also form semigroups.” Adapted from: Functional Programming in Scala - Part 3 - Chapter 10 Monoids
  87. None
  88. import scalaz.scalacheck.ScalazProperties._ semigroup.laws[Map[String, Set[Int]]].check + semigroup.associative: OK, passed 100 tests.

  89. Map("a" -> Set(1, 2)) |+| Map("a" -> Set(2, 3)) res:

    Map[…] = Map(a -> Set(1, 2, 3)) ✓
  90. Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3),

    "aaab" -> List(2, 4))))) |+| Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(5, 7), "aaab" -> List(6, 8))))) Map(a->Map(aa->Map(aaa->Map(aaaa->List(1, 3, 5, 7), aaab->List(2, 4, 6, 8)))))
  91. October 2017 @filippovitale $ tail -f questions

  92. Benchmarking

  93. “Experience indicates that nearly everybody has the wrong idea about

    the real bottlenecks in his programs” – Donald Knuth Computer programming as an art (1974) – http://dl.acm.org/citation.cfm?id=361612
  94. Map[String, Set[Int]]

  95. None