Let’s Blend some Data Structures

How parametric polymorphism and Algebra can solve a real world problem

https://www.meetup.com/en-AU/AnnArborFP/events/243678261/

Transcript

1. October 2017 @filippovitale Let’s Blend some Data Structures
2. Map( "k1" -> List(11, 12, 13), "k2" -> List(21, 22, 23), )
3. Map( "k1" -> List(11, 12, 13), "k2" -> List(21, 22, 23), ) key value key value key value
4. Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3), "aaab" -> List(2, 4)))))
6. Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3), "aaab" -> List(2, 4))))) ????? Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(5, 7), "aaab" -> List(6, 8)))))
9. 3 Simple Data Structures
10. Seq Set Map package scala.collection
11. Seq Set Map package scala.collection v v v v
12. Seq Set Map package scala.collection valueA valueB valueC
13. Seq Set Map package scala.collection key value key value key value
14. Seq Set Map package scala.collection base trait for mutable & immutable implementations
15. “Mutability is an optimisation – perhaps premature” Seq Set Map base trait for mutable & immutable implementations
16. Let’s “Blend” something!
17. value value value Map Set key value key value key value List v v v v
18. Immutable Collection Hierarchy Traversable Seq List http://docs.scala-lang.org/tutorials/FAQ/collections.html def ++[B] (that:Traversable[B]) :Traversable[B]
19. Blending Lists List(1, 2, 3) ++ List(4, 5, 6) == ???
20. Blending Appending Lists List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6) http://www.scala-lang.org/api/current/#scala.collection.immutable.List http://docs.scala-lang.org/overviews/collections/seqs.html
21. Immutable Collection Hierarchy Traversable Seq Set List HashSet http://docs.scala-lang.org/tutorials/FAQ/collections.html
22. Blending Sets Set(1, 2, 3) ++ Set(4, 5, 6) == ???
23. Blending Adding Sets http://www.scala-lang.org/api/current/#scala.collection.immutable.Set http://docs.scala-lang.org/overviews/collections/sets.html Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)
24. Blending Adding Sets Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4) Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)
27. Immutable Collection Hierarchy Traversable Seq Set Map List HashSet HashMap http://docs.scala-lang.org/tutorials/FAQ/collections.html
28. Blending Adding Maps Map() ++ Map("a" -> 1) == Map("a" -> 1) http://www.scala-lang.org/api/current/#scala.collection.immutable.Map http://docs.scala-lang.org/overviews/collections/maps.html
29. Blending Adding Maps Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? http://www.scala-lang.org/api/current/#scala.collection.immutable.Map http://docs.scala-lang.org/overviews/collections/maps.html
30. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?
31. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?
32. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?
33. Map("a" -> Set(1, 2)) ??? Map("a" -> Set(2, 3)) Map("a" -> Set(1, 2, 3))
36. Let’s implement the “Blend” logic
37. “a” Set(1, 2) “a” Set(2, 3) “a” Set(1, 2) ++ Set(2, 3)
38. “a” Set(1, 2) “a” Set(2, 3) “a” Set(1, 2, 3)
39. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]
40. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { mb foreach { case (k, v) => ??? } }
41. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { val result = mutable.Map() ++ ma mb foreach { case (k, v) => ??? } result.toMap }
42. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap }
44. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
45. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
46. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap mb.foldLeft(ma) { ... }
47. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
48. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap
49. def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } }
50. “When you get used to immutable data, ya kinda forget how to use mutable data in a sensible way.” – Jessica Kerr
51. (ma /: mb) { case (result,(k, v)) => result + (k -> { result.get(k) match { case Some(vr) => vr ++ v case None => v } }
52. (ma /: mb) { case (result,(k, v)) => result + (k -> { result.get(k) match { case Some(vr) => vr ++ v case None => v } // .map(_ ++ v).getOrElse(v) // .some(_ ++ v).none(v) // .fold(v)(_ ++ v) // .cata(_ ++ v, v) }} “FP with Bananas, Lenses, Envelopes and Barbed Wire” – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125 http://en.wikipedia.org/wiki/Catamorphism
53. (ma /: mb) { case (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } mb foreach { case (k, v) => result += (k -> result.get(k).cata(_ ++ v, v)) }
54. (ma /: mb) { case (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } mb foreach { case (k, v) => result += (k -> result.get(k).cata(_ ++ v, v)) }
55. What if now we have to use a slightly different Map?
56. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
57. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = ??? def blend(ma: Map[String, Map[Int, Set[Int]]], mb: Map[String, Map[Int, Set[Int]]]) : Map[String, Map[Int, Set[Int]]] = ???
58. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] (ma /: mb) { case (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } (ma /: mb) { case (result, (k, v)) => result + ??? // { ??? => { ??? } } }
59. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] trait Blendable[A] { def blend(ma: A, mb: A): A }
60. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] trait Blendable[A] { def blend(ma: A, mb: A): A } new Blendable[...] { // ad-hoc polymorphism def blend(ma: ..., mb: ...): ... = ??? }
61. Let’s think about what “Blendable” means
62. What blended “as expected”? List using the ++ binary operator Set using the ++ binary operator List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6) Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)
63. (List, ++) (Set, ++) (1 blend 2) == ??? What about Int?
64. (List, ++) (Set, ++) (Int, +) (1 blend 2) == 1 + 2 == 3 What about Int?
65. (List, ++) (Set, ++) (Int, +) (String, +) ("ab" blend "cd") == ("ab" + "cd") == "abcd" String
66. (List, ++) (Set, ++) (Int, +) (String, +) (Map[...], Blendable[...].blend) Blendable[Map[String, Set[Int]]].blend(ma, mb) Map[String, Set[Int]]
67. What can Mathematicians tell us here?
69. What is a Semigroup? Michael Barr, Charles Wells - Category Theory for Computing Science Semigroups “A semigroup is a set S together with an associative binary operation m: S × S → S”
71. Closure Property http://en.wikipedia.org/wiki/Closure_(mathematics) trait Semigroup[T] { def op(a: T, b: T): T } For all a, b in T, the result of the operation a ⋅ b is also in T: ∀a, b ∈ T : a∙b ∈ T
72. http://en.wikipedia.org/wiki/Closure_(mathematics) trait Semigroup[T] { def op(a: T, b: T): T } def op(a: Boolean, b: Boolean): Boolean def op(a: Int, b: Int): Boolean ✓ ✘ Closure Property
73. Associative Property http://en.wikipedia.org/wiki/Associative_property trait Semigroup[T] { def op(a: T, b: T): T } For all a, b, and c in T, the equation (a ⋅ b) ⋅ c = a ⋅ (b ⋅ c) holds: ∀a, b, c ∈ T : (a∙b)∙c = a∙(b∙c) (a op b) op c == a op (b op c)
74. What is a Semigroup? Michael Barr, Charles Wells - Category Theory for Computing Science Semigroups “A semigroup is a set S together with an associative binary operation m: S × S → S”
75. Semigroups in Scalaz
77. import scalaz.std.set._ implicit def setSemigroup[A]:Semigroup[Set[A]] = new Semigroup[Set[A]] { def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2 }
78. implicit def setSemigroup[A]:Semigroup[Set[A]] = new Semigroup[Set[A]] { def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2 } op
79. import scalaz.syntax.semigroup._ import scalaz.std.list._ List(1, 2) |+| List(3, 4) res: List[Int] = List(1, 2, 3, 4) import scalaz.syntax.semigroup._ import scalaz.std.set._ Set(1, 2) |+| Set(2, 3) res: Set[Int] = Set(1, 2, 3)
80. import scalaz.syntax.semigroup._ import scalaz.std.anyVal._ 1 |+| 2 |+| 3 res: Int = 6 import scalaz.syntax.semigroup._ import scalaz.std.string._ "a" |+| "b" |+| "c" res: String = "abc"
81. /** * A semigroup in type F must satisfy two laws: * * - '''closure''': `∀ a, b in F, append(a, b)` is also in `F`. * - '''associativity''': `∀ a, b, c` in `F`, the equation * `append(append(a, b), c) = append(a, append(b , c))` holds. */ trait SemigroupLaw { def associative(f1: F, f2: F, f3: F)(implicit F: Equal[F]): Boolean = F.equal(append(f1, append(f2, f3)), append(append(f1, f2), f3)) }
82. import scalaz.scalacheck.ScalazProperties._ import scalaz.std.anyVal._ semigroup.laws[Int].check + semigroup.associative: OK, passed 100 tests.
83. semigroup.laws[String].check semigroup.laws[Set[Int]].check semigroup.laws[List[String]].check semigroup.laws[Map[Int, Int]].check + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests.
84. Does Map[_, Set[Int]] have a Semigroup instance?
85. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2) res: Map[…] = Map(a -> 3, b -> 4) “Some data structures form interesting semigroups as long as the types of the elements they contain also form semigroups.”
86. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2) res: Map[…] = Map(a -> 3, b -> 4) “Some data structures form interesting semigroups as long as the types of the elements they contain also form semigroups.” Adapted from: Functional Programming in Scala - Part 3 - Chapter 10 Monoids
88. import scalaz.scalacheck.ScalazProperties._ semigroup.laws[Map[String, Set[Int]]].check + semigroup.associative: OK, passed 100 tests. ✓
89. Map("a" -> Set(1, 2)) |+| Map("a" -> Set(2, 3)) res: Map[…] = Map(a -> Set(1, 2, 3)) ✓
90. Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3), "aaab" -> List(2, 4))))) |+| Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(5, 7), "aaab" -> List(6, 8))))) Map(a->Map(aa->Map(aaa->Map(aaaa->List(1, 3, 5, 7), aaab->List(2, 4, 6, 8)))))
91. October 2017 @filippovitale $ tail -f questions
92. Benchmarking
93. “Experience indicates that nearly everybody has the wrong idea about the real bottlenecks in his programs” – Donald Knuth Computer programming as an art (1974) – http://dl.acm.org/citation.cfm?id=361612
94. Map[String, Set[Int]]