Filippo Vitale
October 24, 2017
110

# Let’s Blend some Data Structures

How parametric polymorphism and Algebra can solve a real world problem

https://www.meetup.com/en-AU/AnnArborFP/events/243678261/

October 24, 2017

## Transcript

1. October 2017
@filippovitale
Let’s Blend some
Data Structures

2. Map(
"k1" -> List(11, 12, 13),
"k2" -> List(21, 22, 23),
)

3. Map(
"k1" -> List(11, 12, 13),
"k2" -> List(21, 22, 23),
)
key value
key value
key value

4. Map("a" ->
Map("aa" ->
Map("aaa" ->
Map("aaaa" -> List(1, 3),
"aaab" -> List(2, 4)))))

5. Map("a" ->
Map("aa" ->
Map("aaa" ->
Map("aaaa" -> List(1, 3),
"aaab" -> List(2, 4))))) ????? Map("a" ->
Map("aa" ->
Map("aaa" ->
Map("aaaa" -> List(5, 7),
"aaab" -> List(6, 8)))))

6. 3 Simple
Data Structures

7. Seq Set Map
package scala.collection

8. Seq Set Map
package scala.collection
v v v v

9. Seq Set Map
package scala.collection
valueA
valueB
valueC

10. Seq Set Map
package scala.collection
key value
key value
key value

11. Seq Set Map
package scala.collection
base trait for
mutable & immutable
implementations

12. “Mutability is an optimisation – perhaps premature”
Seq Set Map
base trait for
mutable & immutable
implementations

13. Let’s “Blend” something!

14. value
value
value
Map Set
key value
key value
key value
List
v v v v

15. Immutable Collection Hierarchy
Traversable Seq List
http://docs.scala-lang.org/tutorials/FAQ/collections.html
def ++[B]
(that:Traversable[B])
:Traversable[B]

16. Blending Lists
List(1, 2, 3) ++ List(4, 5, 6) == ???

17. Blending Appending Lists
List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6)
http://www.scala-lang.org/api/current/#scala.collection.immutable.List
http://docs.scala-lang.org/overviews/collections/seqs.html

18. Immutable Collection Hierarchy
Traversable Seq
Set
List
HashSet
http://docs.scala-lang.org/tutorials/FAQ/collections.html

19. Blending Sets
Set(1, 2, 3) ++ Set(4, 5, 6) == ???

20. Blending Adding Sets
http://www.scala-lang.org/api/current/#scala.collection.immutable.Set
http://docs.scala-lang.org/overviews/collections/sets.html
Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)

21. Blending Adding Sets
Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)
Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)

22. Immutable Collection Hierarchy
Traversable Seq
Set
Map
List
HashSet
HashMap
http://docs.scala-lang.org/tutorials/FAQ/collections.html

23. Blending Adding Maps
Map() ++ Map("a" -> 1) == Map("a" -> 1)
http://www.scala-lang.org/api/current/#scala.collection.immutable.Map
http://docs.scala-lang.org/overviews/collections/maps.html

24. Blending Adding Maps
Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
http://www.scala-lang.org/api/current/#scala.collection.immutable.Map
http://docs.scala-lang.org/overviews/collections/maps.html

25. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
1: Map("a" -> Set(1, 2))
2: Map("a" -> Set(1, 2, 3))
3: Map("a" -> Set(2, 3))
4: RuntimeException
5: Compiler Error
What is the result of “blending” those Maps?

26. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
1: Map("a" -> Set(1, 2))
2: Map("a" -> Set(1, 2, 3))
3: Map("a" -> Set(2, 3))
4: RuntimeException
5: Compiler Error
What is the result of “blending” those Maps?

27. Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ???
1: Map("a" -> Set(1, 2))
2: Map("a" -> Set(1, 2, 3))
3: Map("a" -> Set(2, 3))
4: RuntimeException
5: Compiler Error
What is the result of “blending” those Maps?

28. Map("a" -> Set(1, 2)) ??? Map("a" -> Set(2, 3))
Map("a" -> Set(1, 2, 3))

29. Let’s implement
the “Blend” logic

30. “a” Set(1, 2) “a” Set(2, 3)
“a” Set(1, 2) ++ Set(2, 3)

31. “a” Set(1, 2) “a” Set(2, 3)
“a” Set(1, 2, 3)

32. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]]

33. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
mb foreach { case (k, v) =>
???
}
}

34. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
???
}
result.toMap
}

35. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
if (result.contains(k))
result += k -> (result(k) ++ v)
else
result += k -> v
}
result.toMap
}

36. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
}
(ma /: mb) { case (result,(k, v)) =>
if (result.contains(k))
result + (k -> (result(k) ++ v))
else
result + (k -> v)
}
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
if (result.contains(k))
result += k -> (result(k) ++ v)
else
result += k -> v
}
result.toMap

37. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
}
(ma /: mb) { case (result,(k, v)) =>
if (result.contains(k))
result + (k -> (result(k) ++ v))
else
result + (k -> v)
}
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
if (result.contains(k))
result += k -> (result(k) ++ v)
else
result += k -> v
}
result.toMap

38. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
}
(ma /: mb) { case (result,(k, v)) =>
if (result.contains(k))
result + (k -> (result(k) ++ v))
else
result + (k -> v)
}
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
if (result.contains(k))
result += k -> (result(k) ++ v)
else
result += k -> v
}
result.toMap
mb.foldLeft(ma) { ... }

39. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
}
(ma /: mb) { case (result,(k, v)) =>
if (result.contains(k))
result + (k -> (result(k) ++ v))
else
result + (k -> v)
}
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
if (result.contains(k))
result += k -> (result(k) ++ v)
else
result += k -> v
}
result.toMap

40. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
}
(ma /: mb) { case (result,(k, v)) =>
if (result.contains(k))
result + (k -> (result(k) ++ v))
else
result + (k -> v)
}
val result = mutable.Map() ++ ma
mb foreach { case (k, v) =>
if (result.contains(k))
result += k -> (result(k) ++ v)
else
result += k -> v
}
result.toMap

41. def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = {
(ma /: mb) { case (result,(k, v)) =>
if (result.contains(k))
result + (k -> (result(k) ++ v))
else
result + (k -> v)
}
}

42. “When you get used to immutable data,
ya kinda forget how to use mutable data
in a sensible way.” – Jessica Kerr

43. (ma /: mb) { case (result,(k, v)) =>
result + (k -> {
result.get(k) match {
case Some(vr) => vr ++ v
case None => v
}
}

44. (ma /: mb) { case (result,(k, v)) =>
result + (k -> {
result.get(k) match {
case Some(vr) => vr ++ v
case None => v
}
// .map(_ ++ v).getOrElse(v)
// .some(_ ++ v).none(v)
// .fold(v)(_ ++ v)
// .cata(_ ++ v, v)
}}
“FP with Bananas, Lenses, Envelopes and Barbed Wire” – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125
http://en.wikipedia.org/wiki/Catamorphism

45. (ma /: mb) { case (result, (k, v)) =>
result + (k -> result.get(k).cata(_ ++ v, v))
}
mb foreach { case (k, v) =>
result += (k -> result.get(k).cata(_ ++ v, v))
}

46. (ma /: mb) { case (result, (k, v)) =>
result + (k -> result.get(k).cata(_ ++ v, v))
}
mb foreach { case (k, v) =>
result += (k -> result.get(k).cata(_ ++ v, v))
}

47. What if now we have to use
a slightly different Map?

48. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]

49. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
def blend(ma: Map[String, Set[Int]],
mb: Map[String, Set[Int]])
: Map[String, Set[Int]] = ???
def blend(ma: Map[String, Map[Int, Set[Int]]],
mb: Map[String, Map[Int, Set[Int]]])
: Map[String, Map[Int, Set[Int]]] = ???

50. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
(ma /: mb) { case (result, (k, v)) =>
result + (k -> result.get(k).cata(_ ++ v, v))
}
(ma /: mb) { case (result, (k, v)) =>
result + ??? // { ??? => { ??? } }
}

51. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
trait Blendable[A] {
def blend(ma: A, mb: A): A
}

52. Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]
trait Blendable[A] {
def blend(ma: A, mb: A): A
}
new Blendable[...] { // ad-hoc polymorphism
def blend(ma: ..., mb: ...): ... = ???
}

53. Let’s think about what
“Blendable” means

54. What blended “as expected”?
List using the ++ binary operator
Set using the ++ binary operator
List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6)
Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)

55. (List, ++)
(Set, ++)
(1 blend 2) == ???

56. (List, ++)
(Set, ++)
(Int, +)
(1 blend 2) == 1 + 2 == 3

57. (List, ++)
(Set, ++)
(Int, +)
(String, +)
("ab" blend "cd") == ("ab" + "cd") == "abcd"
String

58. (List, ++)
(Set, ++)
(Int, +)
(String, +)
(Map[...], Blendable[...].blend)
Blendable[Map[String, Set[Int]]].blend(ma, mb)
Map[String, Set[Int]]

59. What can
Mathematicians
tell us here?

60. What is a Semigroup?
Michael Barr, Charles Wells - Category Theory for Computing Science
Semigroups
“A semigroup is a set S together with an
associative binary operation m: S × S → S”

61. Closure Property
http://en.wikipedia.org/wiki/Closure_(mathematics)
trait Semigroup[T] {
def op(a: T, b: T): T
}
For all a, b in T, the result of the operation a ⋅ b is also in T:
∀a, b ∈ T : a∙b ∈ T

62. http://en.wikipedia.org/wiki/Closure_(mathematics)
trait Semigroup[T] {
def op(a: T, b: T): T
}
def op(a: Boolean, b: Boolean): Boolean
def op(a: Int, b: Int): Boolean

Closure Property

63. Associative Property
http://en.wikipedia.org/wiki/Associative_property
trait Semigroup[T] {
def op(a: T, b: T): T
}
For all a, b, and c in T, the equation (a ⋅ b) ⋅ c = a ⋅ (b ⋅ c) holds:
∀a, b, c ∈ T : (a∙b)∙c = a∙(b∙c)
(a op b) op c
==
a op (b op c)

64. What is a Semigroup?
Michael Barr, Charles Wells - Category Theory for Computing Science
Semigroups
“A semigroup is a set S together with an
associative binary operation m: S × S → S”

65. Semigroups in Scalaz

66. import scalaz.std.set._
implicit def setSemigroup[A]:Semigroup[Set[A]] =
new Semigroup[Set[A]] {
def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2
}

67. implicit def setSemigroup[A]:Semigroup[Set[A]] =
new Semigroup[Set[A]] {
def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2
}
op

68. import scalaz.syntax.semigroup._
import scalaz.std.list._
List(1, 2) |+| List(3, 4)
res: List[Int] = List(1, 2, 3, 4)
import scalaz.syntax.semigroup._
import scalaz.std.set._
Set(1, 2) |+| Set(2, 3)
res: Set[Int] = Set(1, 2, 3)

69. import scalaz.syntax.semigroup._
import scalaz.std.anyVal._
1 |+| 2 |+| 3
res: Int = 6
import scalaz.syntax.semigroup._
import scalaz.std.string._
"a" |+| "b" |+| "c"
res: String = "abc"

70. /**
* A semigroup in type F must satisfy two laws:
*
* - '''closure''': `∀ a, b in F, append(a, b)` is also in `F`.
* - '''associativity''': `∀ a, b, c` in `F`, the equation
* `append(append(a, b), c) = append(a, append(b , c))` holds.
*/
trait SemigroupLaw {
def associative(f1: F, f2: F, f3: F)(implicit F: Equal[F]): Boolean =
F.equal(append(f1, append(f2, f3)), append(append(f1, f2), f3))
}

71. import scalaz.scalacheck.ScalazProperties._
import scalaz.std.anyVal._
semigroup.laws[Int].check
+ semigroup.associative: OK, passed 100 tests.

72. semigroup.laws[String].check
semigroup.laws[Set[Int]].check
semigroup.laws[List[String]].check
semigroup.laws[Map[Int, Int]].check
+ semigroup.associative: OK, passed 100 tests.
+ semigroup.associative: OK, passed 100 tests.
+ semigroup.associative: OK, passed 100 tests.
+ semigroup.associative: OK, passed 100 tests.

73. Does Map[_, Set[Int]] have
a Semigroup instance?

74. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)
res: Map[…] = Map(a -> 3, b -> 4)
“Some data structures form interesting semigroups as long as
the types of the elements they contain also form semigroups.”

75. Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2)
res: Map[…] = Map(a -> 3, b -> 4)
“Some data structures form interesting semigroups as long as
the types of the elements they contain also form semigroups.”
Adapted from: Functional Programming in Scala - Part 3 - Chapter 10 Monoids

76. import scalaz.scalacheck.ScalazProperties._
semigroup.laws[Map[String, Set[Int]]].check
+ semigroup.associative: OK, passed 100 tests.

77. Map("a" -> Set(1, 2)) |+| Map("a" -> Set(2, 3))
res: Map[…] = Map(a -> Set(1, 2, 3))

78. Map("a" ->
Map("aa" ->
Map("aaa" ->
Map("aaaa" -> List(1, 3),
"aaab" -> List(2, 4))))) |+| Map("a" ->
Map("aa" ->
Map("aaa" ->
Map("aaaa" -> List(5, 7),
"aaab" -> List(6, 8)))))
Map(a->Map(aa->Map(aaa->Map(aaaa->List(1, 3, 5, 7), aaab->List(2, 4, 6, 8)))))

79. October 2017
@filippovitale
\$ tail -f questions

80. Benchmarking

81. “Experience indicates that nearly everybody
has the wrong idea about the real bottlenecks
in his programs” – Donald Knuth
Computer programming as an art (1974) – http://dl.acm.org/citation.cfm?id=361612

82. Map[String, Set[Int]]