Rhein-Main Scala Enthusiasts: Functional Programming Awesomeness

Slides for the Rhein-Main Scala Enthusiasts Meetup on 7. September 2017

1. Functional Programming Awesomeness Markus Hauck (@markus1189) Scala Enthusiasts Rhein Main, 2017
2. Intro Composition Monoids Errors Conclusion Introduction • FP becomes more popular (finally!) • Time to have a look • There are many useful patterns • Monoids • Better Error Handling Markus Hauck (@markus1189) Functional Programming Awesomeness 2 / 41
3. Intro Composition Monoids Errors Conclusion Lego vs Duplo pictures from shop.lego.com Markus Hauck (@markus1189) Functional Programming Awesomeness 3 / 41
4. Intro Composition Monoids Errors Conclusion Lego vs Duplo pictures from shop.lego.com Markus Hauck (@markus1189) Functional Programming Awesomeness 4 / 41
5. Intro Composition Monoids Errors Conclusion Lego vs Duplo • Duplo favours large specialized building blocks • blocks tend to be too big • limited reuse Markus Hauck (@markus1189) Functional Programming Awesomeness 5 / 41
6. Intro Composition Monoids Errors Conclusion Lego vs Duplo • Duplo favours large specialized building blocks • blocks tend to be too big • limited reuse • Lego focuses on small composable building blocks • blocks can conveniently be reused for other purposes • limited use of specialized building blocks Markus Hauck (@markus1189) Functional Programming Awesomeness 5 / 41
7. Intro Composition Monoids Errors Conclusion Lego vs Duplo • Duplo favours large specialized building blocks • blocks tend to be too big • limited reuse • Lego focuses on small composable building blocks • blocks can conveniently be reused for other purposes • limited use of specialized building blocks OO tends to be like Duplo, FP tends to be like Lego Markus Hauck (@markus1189) Functional Programming Awesomeness 5 / 41
8. Intro Composition Monoids Errors Conclusion Monoids • intuition: “combine stuff” • you can create values from thin air via Monoid.empty • combine two values via Monoid.combine / |+| • additionally: laws (don’t write buggy implementations) Markus Hauck (@markus1189) Functional Programming Awesomeness 6 / 41
9. Intro Composition Monoids Errors Conclusion Monoid Laws // 1) left identity empty |+| x === x // 2) right identity x |+| empty === x // 3) associative x |+| (y |+| z) === (x |+| y) |+| z Markus Hauck (@markus1189) Functional Programming Awesomeness 7 / 41
10. Intro Composition Monoids Errors Conclusion Monoid Typeclass trait Monoid[A] { def empty: A def combine(lhs: A, rhs: A): A // infix operator: |+| } implicit val plus: Monoid[Int] = new Monoid[Int] { def empty: Int = 0 def combine(lhs: Int, rhs: Int): Int = lhs + rhs } object Monoid { def apply[A:Monoid]: Monoid[A] = implicitly } Markus Hauck (@markus1189) Functional Programming Awesomeness 8 / 41
11. Intro Composition Monoids Errors Conclusion Using Our Monoid > Monoid[Int].empty 0 > Monoid[Int].combine(1,2) 3 > 1 |+| 2 3 > 42 |+| Monoid[Int].empty 42 > List(1,2,3).foldLeft(Monoid[Int].empty)(_ |+| _) 6 Markus Hauck (@markus1189) Functional Programming Awesomeness 9 / 41
12. Intro Composition Monoids Errors Conclusion More Monoids • Monoid instance not unique: addition / multiplication / min / max • most collections are Monoids: List / Vector / Set • let’s see some more examples Markus Hauck (@markus1189) Functional Programming Awesomeness 10 / 41
13. Intro Composition Monoids Errors Conclusion Monoid Zoo List[A] is a Monoid Markus Hauck (@markus1189) Functional Programming Awesomeness 11 / 41
14. Intro Composition Monoids Errors Conclusion Monoid Zoo List[A] is a Monoid A => B if B is a Monoid Markus Hauck (@markus1189) Functional Programming Awesomeness 11 / 41
15. Intro Composition Monoids Errors Conclusion Monoid Zoo List[A] is a Monoid A => B if B is a Monoid (A,B) if A and B are Monoids Markus Hauck (@markus1189) Functional Programming Awesomeness 11 / 41
16. Intro Composition Monoids Errors Conclusion Monoid Zoo List[A] is a Monoid A => B if B is a Monoid (A,B) if A and B are Monoids Future[A] if A is a Monoid Markus Hauck (@markus1189) Functional Programming Awesomeness 11 / 41
17. Intro Composition Monoids Errors Conclusion Monoid Zoo List[A] is a Monoid A => B if B is a Monoid (A,B) if A and B are Monoids Future[A] if A is a Monoid Map[A,B] if B is a Monoid Markus Hauck (@markus1189) Functional Programming Awesomeness 11 / 41
18. Intro Composition Monoids Errors Conclusion Monoid Zoo List[A] is a Monoid A => B if B is a Monoid (A,B) if A and B are Monoids Future[A] if A is a Monoid Map[A,B] if B is a Monoid val m1 = Map("as" -> 21, "bs" -> 4) val m2 = Map("as" -> 21, "cs" -> 2) m1 |+| m2 // Map("as" -> 42, "bs" -> 4, "cs" -> 2) Markus Hauck (@markus1189) Functional Programming Awesomeness 11 / 41
19. Intro Composition Monoids Errors Conclusion Monoids Compose (Lego Principle) Config => A Markus Hauck (@markus1189) Functional Programming Awesomeness 12 / 41
20. Intro Composition Monoids Errors Conclusion Monoids Compose (Lego Principle) Config => A Config => Future[A] Markus Hauck (@markus1189) Functional Programming Awesomeness 12 / 41
21. Intro Composition Monoids Errors Conclusion Monoids Compose (Lego Principle) Config => A Config => Future[A] Config => Future[Map[String,A]] Markus Hauck (@markus1189) Functional Programming Awesomeness 12 / 41
22. Intro Composition Monoids Errors Conclusion Monoids Compose (Lego Principle) Config => A Config => Future[A] Config => Future[Map[String,A]] Config => Future[Map[String,(A,B)]] Markus Hauck (@markus1189) Functional Programming Awesomeness 12 / 41
23. Intro Composition Monoids Errors Conclusion Monoids Compose (Lego Principle) Config => A Config => Future[A] Config => Future[Map[String,A]] Config => Future[Map[String,(A,B)]] Config => Future[Map[String,(A,Option[B])]] Markus Hauck (@markus1189) Functional Programming Awesomeness 12 / 41
24. Intro Composition Monoids Errors Conclusion Monoids Compose (Lego Principle) Config => A Config => Future[A] Config => Future[Map[String,A]] Config => Future[Map[String,(A,B)]] Config => Future[Map[String,(A,Option[B])]] Config => Future[Map[String,(Set[A],Option[B])]] Markus Hauck (@markus1189) Functional Programming Awesomeness 12 / 41
25. Intro Composition Monoids Errors Conclusion Lot’s of theory . . . Markus Hauck (@markus1189) Functional Programming Awesomeness 13 / 41
26. Intro Composition Monoids Errors Conclusion • analysis of a potentially huge text • calculate metrics over text • word count • char count • min/max word length • avg word length • . . . (be flexible) • goal: single traversal ↔ easy composition Markus Hauck (@markus1189) Functional Programming Awesomeness 14 / 41
27. Intro Composition Monoids Errors Conclusion RDDs and Folds abstract class RDD[T] { /** * Aggregate the elements of each partition, * and then the results for all the partitions, * using a given associative function and a * neutral "zero value". */ def fold(zeroValue: T)(op: (T, T) => T): T } Markus Hauck (@markus1189) Functional Programming Awesomeness 15 / 41
28. Intro Composition Monoids Errors Conclusion Monoidal RDDs implicit class MonoidRDD[T](val rdd: RDD[T]) { // avoid conflicts with fold/reduce etc def combine(implicit M: Monoid[T]): T = rdd.fold(M.empty)(M.combine(_,_)) } Markus Hauck (@markus1189) Functional Programming Awesomeness 16 / 41
29. Intro Composition Monoids Errors Conclusion The Program val sc: SparkContext = ??? val file: String = ??? val data = sc.textFile(file). // read the file flatMap(_.split("""\W+""")). // split into words map(expand) // action! def expand(w: String) = (1, w.length, Map(w -> 1)) val (words,chars,wordMap) = data.combine Markus Hauck (@markus1189) Functional Programming Awesomeness 17 / 41
30. Intro Composition Monoids Errors Conclusion Running this program Scala Meetup: Rhein-Main Scala Enthusiasts Markus Hauck (@markus1189) Functional Programming Awesomeness 18 / 41
31. Intro Composition Monoids Errors Conclusion Running this program Scala Meetup: Rhein-Main Scala Enthusiasts Seq("Scala","Meetup","Rhein","Main","Scala", ...) Markus Hauck (@markus1189) Functional Programming Awesomeness 18 / 41
32. Intro Composition Monoids Errors Conclusion Running this program Scala Meetup: Rhein-Main Scala Enthusiasts Seq("Scala","Meetup","Rhein","Main","Scala", ...) Seq( // expand(w: String) = (1,w.length,Map(w->1)) (1, 5, Map("Scala" -> 1)), (1, 6, Map("Meetup" -> 1)), (1, 5, Map("Rhein" -> 1)), (1, 4, Map("Main" -> 1)), // ... ) Markus Hauck (@markus1189) Functional Programming Awesomeness 18 / 41
33. Intro Composition Monoids Errors Conclusion Running this program Scala Meetup: Rhein-Main Scala Enthusiasts Seq("Scala","Meetup","Rhein","Main","Scala", ...) Seq( // expand(w: String) = (1,w.length,Map(w->1)) (1, 5, Map("Scala" -> 1)), (1, 6, Map("Meetup" -> 1)), (1, 5, Map("Rhein" -> 1)), (1, 4, Map("Main" -> 1)), // ... ) (6,36,Map("Scala" -> 2,"Meetup" -> 1,...)) Markus Hauck (@markus1189) Functional Programming Awesomeness 18 / 41
34. Intro Composition Monoids Errors Conclusion Easy Extension val data: RDD[String] = ??? // as before def expand(w: String) = ( 1, Max(w.length), // max word length Min(w.length), // min word length Map(w.length -> Set(w)) // words by count ) val (count,max,min,byCount) = data.combine Markus Hauck (@markus1189) Functional Programming Awesomeness 19 / 41
35. Intro Composition Monoids Errors Conclusion The new program Scala Meetup: Rhein-Main Scala Enthusiasts Markus Hauck (@markus1189) Functional Programming Awesomeness 20 / 41
36. Intro Composition Monoids Errors Conclusion The new program Scala Meetup: Rhein-Main Scala Enthusiasts Seq("Scala","Meetup","Rhein","Main","Scala", ...) Markus Hauck (@markus1189) Functional Programming Awesomeness 20 / 41
37. Intro Composition Monoids Errors Conclusion The new program Scala Meetup: Rhein-Main Scala Enthusiasts Seq("Scala","Meetup","Rhein","Main","Scala", ...) Seq( (1, Max(5), Min(5), Map(5 -> Set("Scala"))), (1, Max(6), Min(6), Map(6 -> Set("Meetup"))), (1, Max(5), Min(5), Map(5 -> Set("Rhein"))), (1, Max(4), Min(4), Map(4 -> Set("Main"))), (1, Max(5), Min(5), Map(5 -> Set("Scala"))), // ... ) Markus Hauck (@markus1189) Functional Programming Awesomeness 20 / 41
38. Intro Composition Monoids Errors Conclusion The new program Scala Meetup: Rhein-Main Scala Enthusiasts Seq("Scala","Meetup","Rhein","Main","Scala", ...) Seq( (1, Max(5), Min(5), Map(5 -> Set("Scala"))), (1, Max(6), Min(6), Map(6 -> Set("Meetup"))), (1, Max(5), Min(5), Map(5 -> Set("Rhein"))), (1, Max(4), Min(4), Map(4 -> Set("Main"))), (1, Max(5), Min(5), Map(5 -> Set("Scala"))), // ... ) (6,Max(11),Min(4),Map(5->Set("Scala","Rhein"),...)) Markus Hauck (@markus1189) Functional Programming Awesomeness 20 / 41
39. Intro Composition Monoids Errors Conclusion More Monoid Tricks • “filter” values via mempty value • map + reduce == two phase computation via monoids • finger trees, choose Monoid, get random access list / queue / . . . • Monoids Theme and Variations (Functional Pearl) Markus Hauck (@markus1189) Functional Programming Awesomeness 21 / 41
40. Intro Composition Monoids Errors Conclusion Part Two: Errors Markus Hauck (@markus1189) Functional Programming Awesomeness 22 / 41
41. Intro Composition Monoids Errors Conclusion The Traditional Way • Java style: try/catch/finally • checked exceptions • compiler help • reduce return value checking • unchecked exceptions • only visible via docs, if documented at all • the “way to go” in Java? • errors? return null / custom classes Markus Hauck (@markus1189) Functional Programming Awesomeness 23 / 41
42. Intro Composition Monoids Errors Conclusion The Functional Way • Scala: only unchecked exceptions • but: throw and catch discouraged in FP anyway • FP: type system + first class values = referential transparency! Markus Hauck (@markus1189) Functional Programming Awesomeness 24 / 41
43. Intro Composition Monoids Errors Conclusion Referential Transparency • referential transparency (sounds scary?) • enables you to replace 40 + 2 with 42 • local reasoning & equational reasoning ♥ • parts can be reused independently like lego bricks Markus Hauck (@markus1189) Functional Programming Awesomeness 25 / 41
44. Intro Composition Monoids Errors Conclusion Out Of The Box • Either / Try • Either unbiased (before 2.12) • Try not a lawful monad. . . • get the most by using a FP library Markus Hauck (@markus1189) Functional Programming Awesomeness 26 / 41
45. Intro Composition Monoids Errors Conclusion Try and Catch def convert(is: String*): List[Int] = is.map(_.toInt).toList convert("1","2","3","Hello World!","5") java.lang.NumberFormatException: For input string: "Hello World" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) ... Markus Hauck (@markus1189) Functional Programming Awesomeness 27 / 41
46. Intro Composition Monoids Errors Conclusion Try and Catch: Not Compositonal try { convert("1","2","3","Hello World!","5") } catch { case e: NumberFormatException => println("Oops") } • type says nothing about errors • only one option: fail fast • what about getting all errors or non-fatal warnings • problem is that try/catch does not compose (Duplo) Markus Hauck (@markus1189) Functional Programming Awesomeness 28 / 41
47. Intro Composition Monoids Errors Conclusion Example Time • password validation • constraints: • length ≥ 8 • contains at least one number • contains no spaces • contains at least one upper char Markus Hauck (@markus1189) Functional Programming Awesomeness 29 / 41
48. Intro Composition Monoids Errors Conclusion Validating Passwords sealed trait LoginError case object PwTooShort extends LoginError case object PwContainsSpace extends LoginError def checkLength(s: String): Option[LoginError] def checkSpace(s: String): Option[LoginError] def convert[E](o: Option[E]): Either[E, Unit] def login(s: String): Either[LoginError,Token] = for { _ <- convert(checkLength(input)) _ <- convert(checkSpace(input)) // ... token = createToken() } yield token Markus Hauck (@markus1189) Functional Programming Awesomeness 30 / 41
49. Intro Composition Monoids Errors Conclusion Using our Function > cabbage Sorry the password must be more than 8 chars > boiled cabbage Sorry, the password must contain at least one digit > 1 boiled cabbage Sorry, the password cannot have spaces > 50damnedboiledcabbages Sorry, the password must contain at least one upper char Markus Hauck (@markus1189) Functional Programming Awesomeness 31 / 41
50. Intro Composition Monoids Errors Conclusion Using our Function Markus Hauck (@markus1189) Functional Programming Awesomeness 32 / 41
51. Intro Composition Monoids Errors Conclusion Validated • scenario: multiple unrelated error conditions, result is fail/success • Either,Xor,etc. are made for fail-fast, or short circuiting • solution: Validated / Validation Markus Hauck (@markus1189) Functional Programming Awesomeness 33 / 41
52. Intro Composition Monoids Errors Conclusion Passwords with Validated sealed trait LoginError case object PwTooShort extends LoginError case object PwContainsSpace extends LoginError case object PwContainsNoDigit extends LoginError def checkLength(s: String): Option[LoginError] def checkSpace(s: String): Option[LoginError] Markus Hauck (@markus1189) Functional Programming Awesomeness 34 / 41
53. Intro Composition Monoids Errors Conclusion Passwords with Validated def convert[E](o:Option[E]):ValidatedNel[E, Unit] def login(s: String): ValidatedNel[LoginError,Token] = { (convert(checkLength(input)) |+| convert(checkSpace(input)) |+| ... ).as(createToken()) } Markus Hauck (@markus1189) Functional Programming Awesomeness 35 / 41
54. Intro Composition Monoids Errors Conclusion Using Validated login("cabbage") Invalid(NonEmptyList(PwTooShort, PwContainsNoDigit, PwContainsNoSpecialChar)) > cabbage Sorry the password is too short (min. 8 chars), contains no digit and no special character Markus Hauck (@markus1189) Functional Programming Awesomeness 36 / 41
55. Intro Composition Monoids Errors Conclusion Using Validated Markus Hauck (@markus1189) Functional Programming Awesomeness 37 / 41
56. Intro Composition Monoids Errors Conclusion Ior (cats) and \&/ (scalaz) • what about “non-fatal” exceptions • instead of short circuiting, continue with warning • there might still be fatal situations with short circuiting • solution: Ior and \&/ Markus Hauck (@markus1189) Functional Programming Awesomeness 38 / 41
57. Intro Composition Monoids Errors Conclusion First Class Errors • replace try / catch with first class values • Either / Xor and friends = fail-fast • Validated /Validation = multiple independent errors • Ior = fail/succeed/succeed with warnings Markus Hauck (@markus1189) Functional Programming Awesomeness 39 / 41
58. Intro Composition Monoids Errors Conclusion • FP has some nice patterns for you • Monoids: combine stuff • Functional Error Handling Markus Hauck (@markus1189) Functional Programming Awesomeness 40 / 41
59. Intro Composition Monoids Errors Conclusion Enough Duplo, time for Lego! . THANKS! Markus Hauck (@markus1189) Functional Programming Awesomeness 41 / 41