When Everything Fits: The Beauty of Composition

https://2019.flatmap.no/talks/hauck

Transcript

1. Beautiful Composition Markus Hauck (@markus1189)
2. Introduction Monoids Applicatives Streaming Conclusion Composition (how to do more with less) Markus Hauck (@markus1189) Beautiful Composition 1
3. Introduction Monoids Applicatives Streaming Conclusion Composition • most “patterns” in OO don’t compose well • definitely not: Design Patterns • SimpleBeanFactoryAwareAspectInstanceFactory • luckily, many concepts in FP compose naturally • powerful building blocks (feels like LEGO) https://github.com/markus1189/flatmap-beautiful-composition Markus Hauck (@markus1189) Beautiful Composition 2
4. Introduction Monoids Applicatives Streaming Conclusion The Paper Gibbons, Jeremy, and Bruno C. D. S. Oliveira. ”The essence of the iterator pattern.” Journal of functional programming 19.3-4 (2009): 377-402. Markus Hauck (@markus1189) Beautiful Composition 3
5. Introduction Monoids Applicatives Streaming Conclusion Content Content 2 Monoids 1 Introduction 5 Conclusion 4 Streaming 3 Applicatives Markus Hauck (@markus1189) Beautiful Composition 4
6. Introduction Monoids Applicatives Streaming Conclusion Beautiful Composition — Themes Principle of Least Power Composable Abstractions • design: the principle of least power • important property of an abstraction: composition Markus Hauck (@markus1189) Beautiful Composition 5
7. Introduction Monoids Applicatives Streaming Conclusion The Principle Of Least Power • something I notice often A Programmer “I just use flatMap and be done with it” • but: there are benefits of using sth with less power • parallel execution with Applicatives • less power equals more possibilities (Either vs. Validated) Markus Hauck (@markus1189) Beautiful Composition 6
8. Introduction Monoids Applicatives Streaming Conclusion The Principle Of Least Power The Principle “… picking not the most powerful solution but the least powerful” • https://www.w3.org/DesignIssues/Principles.html • same is valid for abstractions from functional programming • if you don’t need Monad, don’t require it • Applicative vs Functor, Monoid vs Semigroup, etc. The reason for this is that the less powerful the language, the more you can do with the data stored… Markus Hauck (@markus1189) Beautiful Composition 7
9. Introduction Monoids Applicatives Streaming Conclusion Composition • abstractions help to hide details and reason at a higher level • real power comes when those abstractions compose (otherwise throwaway islands) • most “classic” GoF Design Patterns in OO fail to compose • many “mathematical” abstractions in FP compose naturally • example: inductive monoid, product of monoids (same for Applicatives, …) Markus Hauck (@markus1189) Beautiful Composition 8
10. Introduction Monoids Applicatives Streaming Conclusion The Case Study • we want to analyze text • collect metrics • single traversal • in essence a simple version of the GNU wc commandline tool 1 bash> wc moby-dick.txt 2 21206 208425 1193382 moby-dick.txt • lines, words and chars Markus Hauck (@markus1189) Beautiful Composition 9
11. Introduction Monoids Applicatives Streaming Conclusion Not For The Faint Of Heart Gibbons, Jeremy, and Bruno C. D. S. Oliveira. “The essence of the iterator pattern.” Markus Hauck (@markus1189) Beautiful Composition 10
12. Introduction Monoids Applicatives Streaming Conclusion A First Solution 1 def run(input: Iterator[Char]): (Int, Int, Int) = { 2 var (nl, nw, nc) = (0, 0, 0) 3 var state = false 4 5 input.foreach { c => 6 nc += 1 7 if (c == '\n') nl += 1 8 if (c == ' ' || c == '\n' || c == '\t') { 9 state = false 10 } else if (!state) { 11 state = true 12 nw += 1 13 } 14 } 15 16 (nl, nw, nc) 17 } Markus Hauck (@markus1189) Beautiful Composition 11
13. Introduction Monoids Applicatives Streaming Conclusion Problems • hard to understand • hard to change • mixed logic of the three metrics • run only certain metrics? (e.g., only words) • hardcoded control flow of char-by-char iteration • very little abstraction • can we do better? Markus Hauck (@markus1189) Beautiful Composition 12
14. Introduction Monoids Applicatives Streaming Conclusion Metrics • number of chars (count each char) • number of lines (count newline characters) • number of words (harder, because it is context sensitive) • open for extension (closed for modification) Markus Hauck (@markus1189) Beautiful Composition 13
15. Introduction Monoids Applicatives Streaming Conclusion Monoids Monoids Markus Hauck (@markus1189) Beautiful Composition 14
16. Introduction Monoids Applicatives Streaming Conclusion Get Our FP Tools! Markus Hauck (@markus1189) Beautiful Composition 15
17. Introduction Monoids Applicatives Streaming Conclusion Monoids • Quick Recap: Monoids • binary method combine and nullary method empty 1 trait Monoid[A] { 2 def combine(x: A, y: A): A 3 def empty: A 4 } 5 //infix combine operator: x |+| y Markus Hauck (@markus1189) Beautiful Composition 16
18. Introduction Monoids Applicatives Streaming Conclusion Monoids: Basic Idea • basic approach: iterate over all Char and accumulate using a monoid Markus Hauck (@markus1189) Beautiful Composition 17
19. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h 0 + 1 = 1 e l l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
20. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e 1 + 1 = 2 l l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
21. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l 2 + 1 = 3 l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
22. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l 3 + 1 = 4 o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
23. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o 4 + 1 = 5 \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
24. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n 5 + 1 = 6 w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
25. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w 6 + 1 = 7 o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
26. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w o 7 + 1 = 8 r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
27. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w o r 8 + 1 = 9 l d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
28. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w o r l 9 + 1 = 10 d ! \n Markus Hauck (@markus1189) Beautiful Composition 18
29. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w o r l d 10 + 1 = 11 ! \n Markus Hauck (@markus1189) Beautiful Composition 18
30. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w o r l d ! 11 + 1 = 12 \n Markus Hauck (@markus1189) Beautiful Composition 18
31. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Chars • counting chars is easy, use (Int, +) as a Monoid • count 1 (combine 1) for every character h e l l o \n w o r l d ! \n 12 + 1 = 13 • so the result is 13 chars in total Markus Hauck (@markus1189) Beautiful Composition 18
32. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h 0 + 0 = 0 e l l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
33. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e 0 + 0 = 0 l l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
34. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l 0 + 0 = 0 l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
35. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l 0 + 0 = 0 o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
36. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o 0 + 0 = 0 \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
37. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n 0 + 1 = 1 w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
38. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w 1 + 0 = 1 o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
39. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w o 1 + 0 = 1 r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
40. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w o r 1 + 0 = 1 l d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
41. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w o r l 1 + 0 = 1 d ! \n Markus Hauck (@markus1189) Beautiful Composition 19
42. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w o r l d 1 + 0 = 1 ! \n Markus Hauck (@markus1189) Beautiful Composition 19
43. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w o r l d ! 1 + 0 = 1 \n Markus Hauck (@markus1189) Beautiful Composition 19
44. Introduction Monoids Applicatives Streaming Conclusion Monoids: Counting Lines • to count lines, use again (Int, +) as a Monoid • but only count 1 if the character is a \n h e l l o \n w o r l d ! \n 1 + 1 = 2 • we counted 2 lines in total Markus Hauck (@markus1189) Beautiful Composition 19
45. Introduction Monoids Applicatives Streaming Conclusion Composing Monoids • now: count chars and lines • for multiple metrics, do multiple passes?! • no — because monoids compose • inductive: monoid + base monoid • product: tuple of monoids Markus Hauck (@markus1189) Beautiful Composition 20
46. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Induction • some Monoids are based inductively on others 1 def optionMonoid[A: Monoid] = new Monoid[Option[A]] { /*...*/ } • Option, Future, IO, Task, … • the Option-Monoid works like this: 1 None |+| y === y 2 x |+| None === x 3 Some(x) |+| Some(y) === Some(x |+| y) Markus Hauck (@markus1189) Beautiful Composition 21
47. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • as an example: filter out (don’t count) stopwords • stopwords = most common words that are not interesting (“the”, “a”, …) • idea: if it is a stopword, use None, otherwise regular count with Some Markus Hauck (@markus1189) Beautiful Composition 22
48. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • assuming both “is” and “a” are classified as stopwords: this None |+| Some(1) = Some(1) is a test text Markus Hauck (@markus1189) Beautiful Composition 23
49. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • assuming both “is” and “a” are classified as stopwords: this is Some(1) |+| None = Some(1) a test text Markus Hauck (@markus1189) Beautiful Composition 23
50. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • assuming both “is” and “a” are classified as stopwords: this is a Some(1) |+| None = Some(1) test text Markus Hauck (@markus1189) Beautiful Composition 23
51. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • assuming both “is” and “a” are classified as stopwords: this is a test Some(1) |+| Some(1) = Some(2) text Markus Hauck (@markus1189) Beautiful Composition 23
52. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • assuming both “is” and “a” are classified as stopwords: this is a test text Some(2) |+| Some(1) = Some(3) • count without stopwords is 3 Markus Hauck (@markus1189) Beautiful Composition 23
53. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Option And Stopwords • use Option plus Max,Min to get longest/shortest non-stopword • more options: • don’t count chars like !?,. etc. using Option again • use Future/Task/IO to get parallelism • and sooo much more Markus Hauck (@markus1189) Beautiful Composition 24
54. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Induction • base instance does not have to be a Monoid • using Option we can lift any Semigroup • empty becomes None • useful for e.g. Max and Min to represent lower/upper bound Markus Hauck (@markus1189) Beautiful Composition 25
55. Introduction Monoids Applicatives Streaming Conclusion Monoid Composition — Tuple • if A and B have a Monoid instance, so does (A,B) 1 def tupleMonoid[A: Monoid, B: Monoid]: Monoid[(A, B)] = 2 new Monoid[(A, B)] { 3 def empty = (Monoid[A].empty, Monoid[B].empty) 4 5 def combine(x: (A, B), y: (A, B)) = (x._1 |+| y._1, x._2 |+| y._2) 6 } • combine the two A’s and the two B’s • we can fuse our two metrics! Markus Hauck (@markus1189) Beautiful Composition 26
56. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h 0 + 0 = 0 0 + 1 = 1 … e l l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
57. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e 0 + 0 = 0 1 + 1 = 2 … l l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
58. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l 0 + 0 = 0 2 + 1 = 3 … l o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
59. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l 0 + 0 = 0 3 + 1 = 4 … o \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
60. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o 0 + 0 = 0 4 + 1 = 5 … \n w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
61. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n 0 + 1 = 1 5 + 1 = 6 … w o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
62. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w 1 + 0 = 1 6 + 1 = 7 … o r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
63. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w o 1 + 0 = 1 7 + 1 = 8 … r l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
64. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w o r 1 + 0 = 1 8 + 1 = 9 … l d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
65. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w o r l 1 + 0 = 1 9 + 1 = 10 … d ! \n Markus Hauck (@markus1189) Beautiful Composition 27
66. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w o r l d 1 + 0 = 1 10 + 1 = 11 … ! \n Markus Hauck (@markus1189) Beautiful Composition 27
67. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w o r l d ! 1 + 0 = 1 11 + 1 = 12 … \n Markus Hauck (@markus1189) Beautiful Composition 27
68. Introduction Monoids Applicatives Streaming Conclusion Multiple Monoids — One Traversal h e l l o \n w o r l d ! \n 1 + 1 = 2 12 + 1 = 13 … • result: 2 lines and 13 chars Markus Hauck (@markus1189) Beautiful Composition 27
69. Introduction Monoids Applicatives Streaming Conclusion More Monoids • for wc we have two of our three target metrics • but, we can do a lot more than that! • find longest word • count occurrences by word • average word length (as own monoid or derive) • map of key to value (for any monoid as value) Markus Hauck (@markus1189) Beautiful Composition 28
70. Introduction Monoids Applicatives Streaming Conclusion Wordcount (Monoids, Chars) 1 def run(input: Iterator[Char]): (Int, Int, Int) = runMonoid(step)(input) 2 3 def runMonoid[M: Monoid](f: Char => M)(input: Iterator[Char]): M = 4 input.map(f).foldLeft(Monoid.empty[M])(_ |+| _) 5 6 def step(c: Char): (Int, Int, Int) = 7 (countLines(c), countWords(c), countChars(c)) 8 9 def countChars(c: Char): Int = 1 10 11 def countLines(c: Char): Int = if (c == '\n') 1 else 0 12 13 def countWords(c: Char): Int = 0 // how? Markus Hauck (@markus1189) Beautiful Composition 29
71. Introduction Monoids Applicatives Streaming Conclusion Wordcount (Monoids, Chars) 1 def run(input: Iterator[Char]): (Int, Int, Int) = runMonoid(step)(input) 2 3 def runMonoid[M: Monoid](f: Char => M)(input: Iterator[Char]): M = 4 input.map(f).foldLeft(Monoid.empty[M])(_ |+| _) 5 6 def step(c: Char): (Int, Int, Int) = 7 (countLines(c), countWords(c), countChars(c)) 8 9 def countChars(c: Char): Int = 1 10 11 def countLines(c: Char): Int = if (c == '\n') 1 else 0 12 13 def countWords(c: Char): Int = 0 // how? Markus Hauck (@markus1189) Beautiful Composition 29
72. Introduction Monoids Applicatives Streaming Conclusion Wordcount (Monoids, Chars) • the problem: we can’t detect words • we require some form of memory or state • alternative idea: pre-split the text into words Markus Hauck (@markus1189) Beautiful Composition 30
73. Introduction Monoids Applicatives Streaming Conclusion Wordcount (Monoids, Strings) 1 def run(input: Iterator[Char]): (Int, Int, Int) = 2 // wordsAndSkipped: Iterator[Char] => Iterator[(Int, String)] 3 runMonoid(step)(wordsAndSkipped(input)) 4 5 def runMonoid[M: Monoid]( 6 f: (Int, String) => M 7 )(input: Iterator[(Int, String)]): M = 8 Monoid[M].combineAll(input.map(f.tupled)) 9 10 def step(skip: Int, w: String): (Int, Int, Int) = 11 (countLines(w), countWords(w), countChars(skip, w)) 12 13 def countChars(skip: Int, w: String): Int = skip + w.length 14 15 def countWords(w: String): Int = 1 16 17 def countLines(w: String): Int = 0 // damn... Markus Hauck (@markus1189) Beautiful Composition 31
74. Introduction Monoids Applicatives Streaming Conclusion Wordcount (Monoids, Strings) 1 def run(input: Iterator[Char]): (Int, Int, Int) = 2 // wordsAndSkipped: Iterator[Char] => Iterator[(Int, String)] 3 runMonoid(step)(wordsAndSkipped(input)) 4 5 def runMonoid[M: Monoid]( 6 f: (Int, String) => M 7 )(input: Iterator[(Int, String)]): M = 8 Monoid[M].combineAll(input.map(f.tupled)) 9 10 def step(skip: Int, w: String): (Int, Int, Int) = 11 (countLines(w), countWords(w), countChars(skip, w)) 12 13 def countChars(skip: Int, w: String): Int = skip + w.length 14 15 def countWords(w: String): Int = 1 16 17 def countLines(w: String): Int = 0 // damn... Markus Hauck (@markus1189) Beautiful Composition 31
75. Introduction Monoids Applicatives Streaming Conclusion Monoids — Review • stuck for now, but: • composes very well from small blocks • easy to extend using custom metrics • Option can lift any Semigroup • works beautifully with everything that can be folded 1 trait Foldable[F[_]] { 2 // rest omitted 3 def foldMap[A, B](fa: F[A])(f: A => B)(implicit B: Monoid[B]): B 4 } Markus Hauck (@markus1189) Beautiful Composition 32
76. Introduction Monoids Applicatives Streaming Conclusion Applicative Applicatives (a.k.a. “the bigger hammer”) Markus Hauck (@markus1189) Beautiful Composition 33
77. Introduction Monoids Applicatives Streaming Conclusion From Monoids To Applicatives • also called Monoidal Functors • “monoidal” in the effects (pure = “empty” effect, <*> = “combine effects”) 1 trait Applicative[F[_]] extends Functor[F] { 2 def pure[A](a: A): F[A] 3 4 def ap[A, B](ff: F[A => B])(fa: F[A]): F[B] 5 } Markus Hauck (@markus1189) Beautiful Composition 34
78. Introduction Monoids Applicatives Streaming Conclusion Applicative as Effect Monoids • Applicatives adds richer structure, monoidal in effects • pure is like empty for effects • product1 is like combine for effects 1 def empty : F 2 def pure(x: A) : F[A] 3 4 def combine( x: F, y: F) : F 5 def product(fa: F[A], fb: F[B]): F[(A, B)] 1from Semigroupal Markus Hauck (@markus1189) Beautiful Composition 35
79. Introduction Monoids Applicatives Streaming Conclusion No Monoids? • but wait a moment, was all we learned about Monoids for nothing?! • rejoice, we can reuse everything we have until now • in two ways • lift Monoid inside Applicative • promote any Monoid to an Applicative Markus Hauck (@markus1189) Beautiful Composition 36
80. Introduction Monoids Applicatives Streaming Conclusion Monoids Inside Applicatives • for every F[A], given Applicative[F] and Monoid[A] • we can define 1 def empty: F[A] = pure(Monoid[A].empty) 2 3 def combine(x: F[A], y: F[A]): F[A] = 4 Applicative[F].map2(x, y)(Monoid[A].combine) • that’s Applicative → Monoid • what about Monoid → Applicative Markus Hauck (@markus1189) Beautiful Composition 37
81. Introduction Monoids Applicatives Streaming Conclusion The Const Datatype 1 final case class Const[A, B](getConst: A) • simple case class with phantom type parameter • … and one actual value • define Functor and Applicative Markus Hauck (@markus1189) Beautiful Composition 38
82. Introduction Monoids Applicatives Streaming Conclusion The Const Datatype 1 final case class Const[A, B](getConst: A) 1 implicit def constTryApplicative[X]: Applicative[Const[X, ?]] = 2 new Applicative[Const[X, ?]] { 3 override def map[A, B](fa: Const[X, A])(f: A => B): Const[X, B] = 4 Const(fa.getConst) 5 6 override def pure[A](a: A): Const[X, A] = Const(???) 7 8 override def ap[A, B](ff: Const[X, A => B])(fa: Const[X, A]): Const[X, B] = 9 Const(???) 10 } Markus Hauck (@markus1189) Beautiful Composition 39
83. Introduction Monoids Applicatives Streaming Conclusion The Const Datatype 1 final case class Const[A, B](getConst: A) 1 implicit def constApplicative[X: Monoid]: Applicative[Const[X, ?]] = 2 new Applicative[Const[X, ?]] { 3 override def map[A, B](fa: Const[X, A])(f: A => B): Const[X, B] = 4 Const(fa.getConst) 5 6 override def pure[A](a: A): Const[X, A] = Const(Monoid[X].empty) 7 8 override def ap[A, B](ff: Const[X, A => B])(fa: Const[X, A]): Const[X, B] = 9 Const(ff.getConst |+| fa.getConst) 10 } Markus Hauck (@markus1189) Beautiful Composition 40
84. Introduction Monoids Applicatives Streaming Conclusion The Const Datatype • the functor instance is a little strange • but the applicative instance is super awesome • remember Option lifting any Semigroup? • allows you to lift any Monoid into an Applicative! • that means we can still use everything we already have • (nb: Const is a monoid isomorphism) Markus Hauck (@markus1189) Beautiful Composition 41
85. Introduction Monoids Applicatives Streaming Conclusion Composition And Applicatives • remember Monoid composition? Nesting and tupling • we get the same for Applicative (from e.g. cats) • Nested • Tuple2K • why is that a big deal again? Markus Hauck (@markus1189) Beautiful Composition 42
86. Introduction Monoids Applicatives Streaming Conclusion Composition And Applicatives 1 def example1Monoid: (Option[Int], Int) = (Option(1), 5) 1 def example1Applicative: Const[(Option[Int], Int), JFrame] = 2 Const.of[JFrame]((Option(1), 5)) Markus Hauck (@markus1189) Beautiful Composition 43
87. Introduction Monoids Applicatives Streaming Conclusion Composition And Applicatives 1 def example2 2 : Nested[IO, Tuple2K[Const[Int, ?], Validated[List[Throwable], ?], ?], String] 3 Nested { 4 IO { 5 Tuple2K(Const.of[String](1), Validated.valid("valid string")) 6 } 7 } 8 } Markus Hauck (@markus1189) Beautiful Composition 44
88. Introduction Monoids Applicatives Streaming Conclusion Composition And Applicatives IO Const(1) Validated.Valid("...") Markus Hauck (@markus1189) Beautiful Composition 45
89. Introduction Monoids Applicatives Streaming Conclusion Composition And Applicatives Tuple2K IO(<operation 1>) IO(<operation 2>) Markus Hauck (@markus1189) Beautiful Composition 46
90. Introduction Monoids Applicatives Streaming Conclusion Composition And Applicatives Tuple2K IO(<operation 1>) Tuple2K IO(<operation 2>) IO(<operation 3>) Markus Hauck (@markus1189) Beautiful Composition 47
91. Introduction Monoids Applicatives Streaming Conclusion Traversing With Applicatives • for monoids, foldMap is the essential operation • for applicatives, change to traverse • we can derive foldMap from traverse 1 trait Traverse[F[_]] extends Functor[F] with Foldable[F] { 2 def traverse[G[_]: Applicative, A, B](fa: F[A])(f: A => G[B]): G[F[B]] 3 } Markus Hauck (@markus1189) Beautiful Composition 48
92. Introduction Monoids Applicatives Streaming Conclusion Counting Words — With State • we can re-use the counting of chars and lines (and any monoidal metric) • as seen, have to be more clever for words • how to do this with applicatives? • State, IO, ST, … Markus Hauck (@markus1189) Beautiful Composition 49
93. Introduction Monoids Applicatives Streaming Conclusion Counting Words — With State 1 def countWords[A](c: Char): Nested[State[Boolean, ?], Const[Int, ?], A] = 2 Nested { 3 for { 4 before <- State.get[Boolean] 5 after = !c.isWhitespace 6 _ <- State.set[Boolean](after) 7 } yield { 8 (!before && after) ?? Const.of[A](1) 9 } 10 } • ?? is from typelevel/mouse Markus Hauck (@markus1189) Beautiful Composition 50
94. Introduction Monoids Applicatives Streaming Conclusion Counting Words — With State State[Boolean, ?] Const[Int, ?] Markus Hauck (@markus1189) Beautiful Composition 51
95. Introduction Monoids Applicatives Streaming Conclusion Counting Words — With State • putting it together 1 input.traverse_ { c => 2 Tuple2K(Tuple2K(countChars[Unit](c), countLines[Unit](c)), countWords[Unit](c)) 3 } Markus Hauck (@markus1189) Beautiful Composition 52
96. Introduction Monoids Applicatives Streaming Conclusion Monads Must Be Great! • if Applicative brings us so much power, how good must it be with Monad • unfortunately, Monads are not as good as they seem • Functor and Applicative are closed under composition • arbitrary nesting of Functor inside Functor / or Applicative inside Applicative • Monad is not closed, not guaranteed to be a Monad at all! • and Product (Tuple) does not work either Markus Hauck (@markus1189) Beautiful Composition 53
97. Introduction Monoids Applicatives Streaming Conclusion Streaming Streaming Markus Hauck (@markus1189) Beautiful Composition 54
98. Introduction Monoids Applicatives Streaming Conclusion From Iteration To Streaming • there is no traverse for real streams?! • it does not make sense for a stream (why?) • but: do we need the actual traverse • realization: traverse_ (Foldable) is enough! 1 trait Foldable[F[_]] { 2 def traverse_[G[_], A, B](fa: F[A])(f: A => G[B])( 3 implicit G: Applicative[G] 4 ): G[Unit] 5 } Markus Hauck (@markus1189) Beautiful Composition 55
99. Introduction Monoids Applicatives Streaming Conclusion Traverse and Foldable • so what’s the difference between traverse and traverse_ • the regular traverse keeps the whole structure! • the traverse_ variant only sequences effects! • can be implemented using a foldlike method • important: Applicative effect sequencing is associative as per the laws • that means we can work on the stream in parallel Markus Hauck (@markus1189) Beautiful Composition 56
100. Introduction Monoids Applicatives Streaming Conclusion Implement traverse_ for fs2 • our framework needs the traverse_ • can be implemented for anything that has a proper fold • example with fs2 Stream 1 implicit class Fs2StreamOps[A, G[_], F[_]](s: fs2.Stream[F, A]) { 2 def traverse_[B]( 3 f: A => G[B] 4 )(implicit A: Applicative[G]): fs2.Stream[F, G[Unit]] = 5 s.fold(Applicative[G].pure(()))(_ <* f(_)) 6 } Markus Hauck (@markus1189) Beautiful Composition 57
101. Introduction Monoids Applicatives Streaming Conclusion Counting Words With fs2 1 fs2.Stream 2 .fromIterator[IO, Char](input) 3 .traverse_( 4 c => 5 Tuple2K( 6 Tuple2K(countChars[Unit](c), countLines[Unit](c)), 7 countWords[Unit](ref)(c) 8 ) 9 ) 10 .compile 11 .lastOrError 12 .unsafeRunSync() Markus Hauck (@markus1189) Beautiful Composition 58
102. Introduction Monoids Applicatives Streaming Conclusion Counting Words With Streams • suddenly we can use all the stream processing goodness • we could chunk the input and process in parallel (but keep order) • by carefully choosing the Applicative, operate with constant memory • use existing connectors to read from DB, Kafka, etc. • very flexible and almost for free! Markus Hauck (@markus1189) Beautiful Composition 59
103. Introduction Monoids Applicatives Streaming Conclusion Conclusion • flexible and composable way to cacluate metrics over text • using Monoid and Applicative • works with iteration and streaming • Principle Of Least Power: sometimes Monads are overrated Markus Hauck (@markus1189) Beautiful Composition 60
104. Introduction Monoids Applicatives Streaming Conclusion The End • Introduction • Monoids • Applicatives • Streaming • Conclusion https://github.com/markus1189/flatmap-beautiful-composition Markus Hauck (@markus1189) Beautiful Composition 61
105. Overview Semigroup Monoid Applicative Markus Hauck (@markus1189) Beautiful Composition 62
106. Abstraction Semigroup Monoid Applicative Markus Hauck (@markus1189) Beautiful Composition 63
107. Brand New: Selective Functors • like applicatives, we can use a product of selective functors • example use case: find file containing a word • applicative: cannot stop and iterates over everything, static analysis using free structure • monad: can abort iteration after first match, no static analysis • selective: abort early and also analysis via Free structure Markus Hauck (@markus1189) Beautiful Composition 64