Slide 1

Slide 1 text

October 2017 @filippovitale Let’s Blend some Data Structures

Slide 2

Slide 2 text

Map( "k1" -> List(11, 12, 13), "k2" -> List(21, 22, 23), )

Slide 3

Slide 3 text

Map( "k1" -> List(11, 12, 13), "k2" -> List(21, 22, 23), ) key value key value key value

Slide 4

Slide 4 text

Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3), "aaab" -> List(2, 4)))))

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3), "aaab" -> List(2, 4))))) ????? Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(5, 7), "aaab" -> List(6, 8)))))

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

3 Simple Data Structures

Slide 10

Slide 10 text

Seq Set Map package scala.collection

Slide 11

Slide 11 text

Seq Set Map package scala.collection v v v v

Slide 12

Slide 12 text

Seq Set Map package scala.collection valueA valueB valueC

Slide 13

Slide 13 text

Seq Set Map package scala.collection key value key value key value

Slide 14

Slide 14 text

Seq Set Map package scala.collection base trait for mutable & immutable implementations

Slide 15

Slide 15 text

“Mutability is an optimisation – perhaps premature” Seq Set Map base trait for mutable & immutable implementations

Slide 16

Slide 16 text

Let’s “Blend” something!

Slide 17

Slide 17 text

value value value Map Set key value key value key value List v v v v

Slide 18

Slide 18 text

Immutable Collection Hierarchy Traversable Seq List http://docs.scala-lang.org/tutorials/FAQ/collections.html def ++[B] (that:Traversable[B]) :Traversable[B]

Slide 19

Slide 19 text

Blending Lists List(1, 2, 3) ++ List(4, 5, 6) == ???

Slide 20

Slide 20 text

Blending Appending Lists List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6) http://www.scala-lang.org/api/current/#scala.collection.immutable.List http://docs.scala-lang.org/overviews/collections/seqs.html

Slide 21

Slide 21 text

Immutable Collection Hierarchy Traversable Seq Set List HashSet http://docs.scala-lang.org/tutorials/FAQ/collections.html

Slide 22

Slide 22 text

Blending Sets Set(1, 2, 3) ++ Set(4, 5, 6) == ???

Slide 23

Slide 23 text

Blending Adding Sets http://www.scala-lang.org/api/current/#scala.collection.immutable.Set http://docs.scala-lang.org/overviews/collections/sets.html Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4)

Slide 24

Slide 24 text

Blending Adding Sets Set(1, 2, 3) ++ Set(4, 5, 6) == Set(5, 1, 6, 2, 3, 4) Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Immutable Collection Hierarchy Traversable Seq Set Map List HashSet HashMap http://docs.scala-lang.org/tutorials/FAQ/collections.html

Slide 28

Slide 28 text

Blending Adding Maps Map() ++ Map("a" -> 1) == Map("a" -> 1) http://www.scala-lang.org/api/current/#scala.collection.immutable.Map http://docs.scala-lang.org/overviews/collections/maps.html

Slide 29

Slide 29 text

Blending Adding Maps Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? http://www.scala-lang.org/api/current/#scala.collection.immutable.Map http://docs.scala-lang.org/overviews/collections/maps.html

Slide 30

Slide 30 text

Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?

Slide 31

Slide 31 text

Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?

Slide 32

Slide 32 text

Map("a" -> Set(1, 2)) ++ Map("a" -> Set(2, 3)) == ??? 1: Map("a" -> Set(1, 2)) 2: Map("a" -> Set(1, 2, 3)) 3: Map("a" -> Set(2, 3)) 4: RuntimeException 5: Compiler Error What is the result of “blending” those Maps?

Slide 33

Slide 33 text

Map("a" -> Set(1, 2)) ??? Map("a" -> Set(2, 3)) Map("a" -> Set(1, 2, 3))

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

Let’s implement the “Blend” logic

Slide 37

Slide 37 text

“a” Set(1, 2) “a” Set(2, 3) “a” Set(1, 2) ++ Set(2, 3)

Slide 38

Slide 38 text

“a” Set(1, 2) “a” Set(2, 3) “a” Set(1, 2, 3)

Slide 39

Slide 39 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]]

Slide 40

Slide 40 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { mb foreach { case (k, v) => ??? } }

Slide 41

Slide 41 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { val result = mutable.Map() ++ ma mb foreach { case (k, v) => ??? } result.toMap }

Slide 42

Slide 42 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap }

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap

Slide 45

Slide 45 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap

Slide 46

Slide 46 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap mb.foldLeft(ma) { ... }

Slide 47

Slide 47 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap

Slide 48

Slide 48 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { } (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } val result = mutable.Map() ++ ma mb foreach { case (k, v) => if (result.contains(k)) result += k -> (result(k) ++ v) else result += k -> v } result.toMap

Slide 49

Slide 49 text

def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = { (ma /: mb) { case (result,(k, v)) => if (result.contains(k)) result + (k -> (result(k) ++ v)) else result + (k -> v) } }

Slide 50

Slide 50 text

“When you get used to immutable data, ya kinda forget how to use mutable data in a sensible way.” – Jessica Kerr

Slide 51

Slide 51 text

(ma /: mb) { case (result,(k, v)) => result + (k -> { result.get(k) match { case Some(vr) => vr ++ v case None => v } }

Slide 52

Slide 52 text

(ma /: mb) { case (result,(k, v)) => result + (k -> { result.get(k) match { case Some(vr) => vr ++ v case None => v } // .map(_ ++ v).getOrElse(v) // .some(_ ++ v).none(v) // .fold(v)(_ ++ v) // .cata(_ ++ v, v) }} “FP with Bananas, Lenses, Envelopes and Barbed Wire” – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.125 http://en.wikipedia.org/wiki/Catamorphism

Slide 53

Slide 53 text

(ma /: mb) { case (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } mb foreach { case (k, v) => result += (k -> result.get(k).cata(_ ++ v, v)) }

Slide 54

Slide 54 text

(ma /: mb) { case (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } mb foreach { case (k, v) => result += (k -> result.get(k).cata(_ ++ v, v)) }

Slide 55

Slide 55 text

What if now we have to use a slightly different Map?

Slide 56

Slide 56 text

Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]]

Slide 57

Slide 57 text

Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] def blend(ma: Map[String, Set[Int]], mb: Map[String, Set[Int]]) : Map[String, Set[Int]] = ??? def blend(ma: Map[String, Map[Int, Set[Int]]], mb: Map[String, Map[Int, Set[Int]]]) : Map[String, Map[Int, Set[Int]]] = ???

Slide 58

Slide 58 text

Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] (ma /: mb) { case (result, (k, v)) => result + (k -> result.get(k).cata(_ ++ v, v)) } (ma /: mb) { case (result, (k, v)) => result + ??? // { ??? => { ??? } } }

Slide 59

Slide 59 text

Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] trait Blendable[A] { def blend(ma: A, mb: A): A }

Slide 60

Slide 60 text

Map[String, Set[Int]] Map[String, Map[Int, Set[Int]]] trait Blendable[A] { def blend(ma: A, mb: A): A } new Blendable[...] { // ad-hoc polymorphism def blend(ma: ..., mb: ...): ... = ??? }

Slide 61

Slide 61 text

Let’s think about what “Blendable” means

Slide 62

Slide 62 text

What blended “as expected”? List using the ++ binary operator Set using the ++ binary operator List(1, 2, 3) ++ List(4, 5, 6) == List(1, 2, 3, 4, 5, 6) Set(1, 2) ++ Set(2, 3) == Set(1, 2, 3)

Slide 63

Slide 63 text

(List, ++) (Set, ++) (1 blend 2) == ??? What about Int?

Slide 64

Slide 64 text

(List, ++) (Set, ++) (Int, +) (1 blend 2) == 1 + 2 == 3 What about Int?

Slide 65

Slide 65 text

(List, ++) (Set, ++) (Int, +) (String, +) ("ab" blend "cd") == ("ab" + "cd") == "abcd" String

Slide 66

Slide 66 text

(List, ++) (Set, ++) (Int, +) (String, +) (Map[...], Blendable[...].blend) Blendable[Map[String, Set[Int]]].blend(ma, mb) Map[String, Set[Int]]

Slide 67

Slide 67 text

What can Mathematicians tell us here?

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

What is a Semigroup? Michael Barr, Charles Wells - Category Theory for Computing Science Semigroups “A semigroup is a set S together with an associative binary operation m: S × S → S”

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

Closure Property http://en.wikipedia.org/wiki/Closure_(mathematics) trait Semigroup[T] { def op(a: T, b: T): T } For all a, b in T, the result of the operation a ⋅ b is also in T: ∀a, b ∈ T : a∙b ∈ T

Slide 72

Slide 72 text

http://en.wikipedia.org/wiki/Closure_(mathematics) trait Semigroup[T] { def op(a: T, b: T): T } def op(a: Boolean, b: Boolean): Boolean def op(a: Int, b: Int): Boolean ✓ ✘ Closure Property

Slide 73

Slide 73 text

Associative Property http://en.wikipedia.org/wiki/Associative_property trait Semigroup[T] { def op(a: T, b: T): T } For all a, b, and c in T, the equation (a ⋅ b) ⋅ c = a ⋅ (b ⋅ c) holds: ∀a, b, c ∈ T : (a∙b)∙c = a∙(b∙c) (a op b) op c == a op (b op c)

Slide 74

Slide 74 text

What is a Semigroup? Michael Barr, Charles Wells - Category Theory for Computing Science Semigroups “A semigroup is a set S together with an associative binary operation m: S × S → S”

Slide 75

Slide 75 text

Semigroups in Scalaz

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

import scalaz.std.set._ implicit def setSemigroup[A]:Semigroup[Set[A]] = new Semigroup[Set[A]] { def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2 }

Slide 78

Slide 78 text

implicit def setSemigroup[A]:Semigroup[Set[A]] = new Semigroup[Set[A]] { def append(f1: Set[A], f2: => Set[A]) = f1 ++ f2 } op

Slide 79

Slide 79 text

import scalaz.syntax.semigroup._ import scalaz.std.list._ List(1, 2) |+| List(3, 4) res: List[Int] = List(1, 2, 3, 4) import scalaz.syntax.semigroup._ import scalaz.std.set._ Set(1, 2) |+| Set(2, 3) res: Set[Int] = Set(1, 2, 3)

Slide 80

Slide 80 text

import scalaz.syntax.semigroup._ import scalaz.std.anyVal._ 1 |+| 2 |+| 3 res: Int = 6 import scalaz.syntax.semigroup._ import scalaz.std.string._ "a" |+| "b" |+| "c" res: String = "abc"

Slide 81

Slide 81 text

/** * A semigroup in type F must satisfy two laws: * * - '''closure''': `∀ a, b in F, append(a, b)` is also in `F`. * - '''associativity''': `∀ a, b, c` in `F`, the equation * `append(append(a, b), c) = append(a, append(b , c))` holds. */ trait SemigroupLaw { def associative(f1: F, f2: F, f3: F)(implicit F: Equal[F]): Boolean = F.equal(append(f1, append(f2, f3)), append(append(f1, f2), f3)) }

Slide 82

Slide 82 text

import scalaz.scalacheck.ScalazProperties._ import scalaz.std.anyVal._ semigroup.laws[Int].check + semigroup.associative: OK, passed 100 tests.

Slide 83

Slide 83 text

semigroup.laws[String].check semigroup.laws[Set[Int]].check semigroup.laws[List[String]].check semigroup.laws[Map[Int, Int]].check + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests. + semigroup.associative: OK, passed 100 tests.

Slide 84

Slide 84 text

Does Map[_, Set[Int]] have a Semigroup instance?

Slide 85

Slide 85 text

Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2) res: Map[…] = Map(a -> 3, b -> 4) “Some data structures form interesting semigroups as long as the types of the elements they contain also form semigroups.”

Slide 86

Slide 86 text

Map("a" -> 1, "b" -> 4) |+| Map("a" -> 2) res: Map[…] = Map(a -> 3, b -> 4) “Some data structures form interesting semigroups as long as the types of the elements they contain also form semigroups.” Adapted from: Functional Programming in Scala - Part 3 - Chapter 10 Monoids

Slide 87

Slide 87 text

No content

Slide 88

Slide 88 text

import scalaz.scalacheck.ScalazProperties._ semigroup.laws[Map[String, Set[Int]]].check + semigroup.associative: OK, passed 100 tests. ✓

Slide 89

Slide 89 text

Map("a" -> Set(1, 2)) |+| Map("a" -> Set(2, 3)) res: Map[…] = Map(a -> Set(1, 2, 3)) ✓

Slide 90

Slide 90 text

Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(1, 3), "aaab" -> List(2, 4))))) |+| Map("a" -> Map("aa" -> Map("aaa" -> Map("aaaa" -> List(5, 7), "aaab" -> List(6, 8))))) Map(a->Map(aa->Map(aaa->Map(aaaa->List(1, 3, 5, 7), aaab->List(2, 4, 6, 8)))))

Slide 91

Slide 91 text

October 2017 @filippovitale $ tail -f questions

Slide 92

Slide 92 text

Benchmarking

Slide 93

Slide 93 text

“Experience indicates that nearly everybody has the wrong idea about the real bottlenecks in his programs” – Donald Knuth Computer programming as an art (1974) – http://dl.acm.org/citation.cfm?id=361612

Slide 94

Slide 94 text

Map[String, Set[Int]]

Slide 95

Slide 95 text

No content