Algebraic Data Types - Functional Conf 2018

Functional programming is built around a foundation of well-defined Types, and when you throw in Typeclasses into the mix, you get the love-child that is Algebraic Data Types. In this talk I aim to explore the mathematical foundations of Type theory and how it can be used practically in Scala for wide variety of applications like Machine Learning (Apache Spark MLlib), API design (using Vertx), DSLs and the like.

I will also be introducing the scalacheck library for Property-based testing and how you can quickly validate your ADT domains.

The talk will further deep-dive into how you can utilize the amazing Cats library and Shapeless to build generic libraries around your ADTs, having Circe as a case-study.

Presented at: https://functionalconf.com/schedule.html

Transcript

1. 1 Ajay Viswanathan Sr. Software Engineer @ MiQ Algebraic Data Types - My kinda’ type Algebraic Data Types | Functional Conf 2018 | 15th December 2018
2. About me • Apache Spark was my entry into FP • 4+ years of experience with Scala and Functional Programming • I have presented previously at Scala by the Bay, 2015 and Big Data by the Bay, 2015 • Data Processing Team @ MiQ • I prefer cats over scalaz • I blog at www.scala.ninja Algebraic Data Types | Functional Conf 2018 | 15th December 2018 2
3. LEARN PREPARE TEST EXECUTE Data Processing Ecosystem @ MiQ Algebraic Data Types | Functional Conf 2018 | 15th December 2018 3 Data Transformation Service • Data Preparation • Data Cleaning • Data Sampling Data Exploration Service • Data Mining Feature Engineering Service • Feature Generation • Feature Selection Cleaning and Enhancement Machine Learning and optimizations Modeling Service Model Hyperparameter Tuning Parallel Model Execution Model Persistence Model Selection Model Combination Model Deployment Validation and Evaluation Validation Service Model Performance Evaluation Prediction Performance Evaluation Prediction Service Prediction Tuning Clustering Service User Segmentation Experimentation Service A/B Testing Multi-Arm Bandit Testing Prediction and Analysis Notebook Execution Engine Cluster Management Service
4. What is a type? Algebraic Data Types | Functional Conf 2018 | 15th December 2018 4
5. Boolean = {true, false} Int = ! ! ∈ ℤ String = {%& %' … |%& %' … ∈ *!+%,-.} Simple Types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 5
6. Types are not the same as a class Types can be equivalent, classes usually aren’t val distance: Double <=> val weight: Double def convertToLightYears(distance: Double) // not type safe This is why we use classes for our domains Scala also allows you to create value classes to encode 1- attribute classes case class Distance(v: Double) extends AnyVal case class Weight(v: Double) extends AnyVal def convertToLightYears(distance: Distance) // type safe Understanding Types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 6
7. Invariant trait Zoo[T] List[Dog] can only contain Dogs Covariant trait Zoo[+T] List[Dog] can contain Puppies Contrvariant trait Zoo[-T] List[Dog] can contain Animals Type Variance Code shown only for demonstrative purposes Algebraic Data Types | Functional Conf 2018 | 15th December 2018 7 class Animal class Dog extends Animal class Puppy extends Dog
8. Ok, but where’s the Algebra? Algebraic Data Types | Functional Conf 2018 | 15th December 2018 8
9. def f[A, B](a: A): B Domain -> cardinality(A) Codomain -> cardinality(B) Range -> actual cardinality cardinality(f) ≡ "# $(# ∗') ≡ $ ' ∗# ≡ ($')# f: (A, B) -> C <=> f: (B, A) -> C <=> f: A -> B -> C Currying of functions Functions and Relations Class 12th Math Refresher Algebraic Data Types | Functional Conf 2018 | 15th December 2018 9
10. Nothing -> 0 Unit -> 1 Boolean -> 2 Byte -> 256 String, Integer -> too many, more than 14,000,605 Counting Types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 10
11. Pair -> 2 (Boolean, Boolean) -> 2 * 2 (Byte, Boolean) -> 256 * 2 (Int, Boolean) -> cardinality(Int) * 2 Enum -> exhaustive Char OR Boolean -> 256 + 2 Enum OR Nothing -> cardinality(Enum) + 0 Let extend counting to Derived Types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 11
12. Tuples Case classes Products Product = {"#$ #% … |#$ ∈ )$ ∧ #% ∈ )% … } Pair[A,B] = {(-, /) |- ∈ 1 ∧ / ∈ 2}{(-, /)┤|-∈1∧/∈2} Employee = {34567899(:-49, -;9) |:-49 ∈ <=>?:; ∧ -;9 ∈ @:=} Is Pair[String, Int] same as Employee? case class Employee(name: String, age: Int) case class City(name: String, population: Int) Product Type A and B Algebraic Data Types | Functional Conf 2018 | 15th December 2018 12
13. Disjointed Union Enum Either Sum = !"# "# ∈ %# ∪ !"# |"( ∈ %( … Either[A, B] = *+,-(/) / ∈ 1 ∪ 234ℎ- 6 6 ∈ 7 sealed trait Boolean trait True extends Boolean trait False extends Boolean Sum Type A OR B Algebraic Data Types | Functional Conf 2018 | 15th December 2018 13
14. sealed trait Address case class Residential(street: String, pincode: Int) extends Address case class Ip(v4: String) extends Address def printAddress(address: Address) { case Residential(street, pincode) => println(s”$street - $pincode”) case Ip(v4) => println(v4) } Mixing it up Algebraic Data Types | Functional Conf 2018 | 15th December 2018 14
15. sealed trait Tree[T] case object Empty extends Tree[Nothing] case class Leaf[T](v: T) extends Tree[T] case class Node[T](v: T, l: Tree[T], r: Tree[T]) extends Tree[T] def exists[T](t: T, tree: Tree[T]): Boolean = tree match { case Empty => false case Leaf(v) => if (t == v) true else false case Node(v, l, r) => if (t == v) true else (exists(t, l) || exists(t, r)) } Mixing it up Algebraic Data Types | Functional Conf 2018 | 15th December 2018 15
16. Commutativity (a * b) <=> (b * a) (a + b) <=> (b + a) def combine[T](a: T, b: T): T Semigroup Associativity (a * b) * c <=> a * (b * c) (a + b) + c <=> a + (b + c) combine(combine(a, b), c) Mapreduce in a nutshell Algebraic operations on types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 16
17. Distributivity (a * b) + (a * c) <=> a * (b + c) case class Person(name: String, age: Int) case class City(name: String, population: Int) sealed trait Identity { def name: String} case class Person(name: String, age: Int) extends Identity case class City(name: String, population: Int) extends Identity Algebraic operations on types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 17
18. Identity a * e = a a + e = a combine(Some(1), None) = Some(1) Monoid Inverse a * inv(a) = e a + inv(a) = e Group Algebraic operations on types Algebraic Data Types | Functional Conf 2018 | 15th December 2018 18
19. Is not the same as an interface Lets you define some behaviour on a Type trait Noisy[T] { def makeSomeNoise(t: T): String } class Dog class Cat implicit val noisyDog = new Noisy[Dog] { def makeSomeNoise(“bark”) } implicit val noisyCat = new Noisy[Cat] { def makeSomeNoise(“meow”) } def printNoise[T](t: T)(implicit ev: Noisy[T]) = println(ev.makeSomeNoise(t)) Typeclasses Algebraic Data Types | Functional Conf 2018 | 15th December 2018 19
20. 1-hole Types – Option, List, Set etc trait Type[F[_]] Functor def map[A, B](fa: F[A])(f: A => B): F[B] Apply – extends Functor def ap[A, B](fa: F[A => B])(f: F[A]): F[B] Applicative – extends Apply def pure[A](a: A): F[A] Monad – extends Applicative def flatten[A, B](fa: Option[A])(f: A => Option[B]): Option[B] Not composable Category Theory 101 – Algebraic Data Structures Based on cats library Algebraic Data Types | Functional Conf 2018 | 15th December 2018 20
21. Enough talk show me some code Algebraic Data Types | Functional Conf 2018 | 15th December 2018 21
22. sealed trait Param case class StringParam(value: String) extends Param case class IntParam(value: String) extends Param case class ParamValue(values: List[(String, Param)]) extends Param ADT for common params Used by microservices across our ecosystems Algebraic Data Types | Functional Conf 2018 | 15th December 2018 22
23. trait ParamEncoder[T] { def encode(value: T): Param } trait ParamObjectEncoder[T] { def encode(value: T): ParamValue } implicit val stringEncoder = new ParamEncoder[String] { def encode(value: String) => StringParam(value) } implicit val intEncoder = new ParamEncoder[Int] { def encode(value: Int) => IntParam(value) } Adding Typeclasses For encoding class objects to Param Algebraic Data Types | Functional Conf 2018 | 15th December 2018 23
24. case class InputConfig(path: String, format: String) implicit val encoder = new ParamEncoder[IntputConfig] { def encode(value: InputConfig) = ParamValue(List( (“path”, stringEncoder.encode(value.path)), (“format”, stringEncoder.encode(value.format)) )) Creating a domain class Algebraic Data Types | Functional Conf 2018 | 15th December 2018 24
25. val genInputConfig: Gen[InputConfig] = for { path <- Gen.alphaLowerStr format <- Gen.alphaLowerStr } yield InputConfig(path, format) Prop.forAll { config => val encoded = ParamEncoder[InputConfig].encode(config) assert(encoded.values.toMap.contains(“path”)) assert(encoded.values.toMap.contain(“format”)) } Property-Based testing “A good programmer is someone who looks both ways before crossing a one-way street.”—Doug Linder Algebraic Data Types | Functional Conf 2018 | 15th December 2018 25
26. implicit val hnilEncoder = new ParamObjectEncoder[Hnil] { def encode(value: Hnil) => ParamValue(Nil) } implicit def hlistEncoder[K <: Symbol, H, T <: Hlist] (implicit witness: Witness.Aux[K], hEncoder: Lazy[ParamEncoder[H], tEncoder: ParamValueEncoder[T]): ParamValueEncoder[FieldType[K, H] :: T] = { val fieldName: String = witness.value.name def encode(value: FieldType[K, H] :: T) = { val head = hEncoder.value.encode(hlist.head) val tail = tEncoder.encoder(hlist.tail) ParamValue((fieldName, head) :: tail.value) } Generics with Shapeless For autamic encoder inference Algebraic Data Types | Functional Conf 2018 | 15th December 2018 26
27. implicit def genericEncoder[T, H](implicit generic: LabelledGeneric.Aux[T, H], hEncoder: Lazy[ParamObjectEncoder[H]]): ParamEncoder[T] = { def encode(value: T) = hEncoder.value.encode(generic.to(value)) } Generics with Shapeless For automatic encoder inference Algebraic Data Types | Functional Conf 2018 | 15th December 2018 27
28. • Defining reusable validations • Defining codegen encoders • Optimization of configs • Less code + More reliability == Less maintainabiity + Less testing == Happy developer One ADT, Multiple Uses Algebraic Data Types | Functional Conf 2018 | 15th December 2018 28
29. “Give someone a program, you frustrate them for a day; teach them how to program, you frustrate them for a lifetime.” Algebraic Data Types | Functional Conf 2018 | 15th December 2018 29 David Leinweber, author of Nerds on Wall Street
30. For indepth understand of Functional Programming concepts - http://learnyouahaskell.com/ Useful for practise and learning - https://www.scala-exercises.org/ The Type Astronaut’s Guide to Shapeless - https://underscore.io/books/shapeless-guide/ Property based testing in Scalatest - http://www.scalatest.org/user_guide/property_based_testing http://tpolecat.github.io/presentations/algebraic_types.html https://swsnr.de/blog/2018/02/24/algebraic-data-types-in-scala/ Algebraic data types for fun and profit by Clément Delafargue, https://www.youtube.com/watch?v=EPxi546vVHI Lectures on Category theory by https://bartoszmilewski.com/about/ https://medium.com/statuscode/the-12-most-retweeted- programming-quotes-2b039c45ca39 References Algebraic Data Types | Functional Conf 2018 | 15th December 2018 30
31. Questions You can reach out to me @MonadicNomad on twitter Algebraic Data Types | Functional Conf 2018 | 15th December 2018 31