Tweet
Tweet

Slide 1

Slide 1 text

BE LIKE WATER A Shapeless Primer

Slide 2

Slide 2 text

Edoardo Vacchi — UniCredit R&D ▸ @evacchi — twitter, github ▸ read more on rnduja.github.io

Slide 3

Slide 3 text

RUNNING EXAMPLE: DATAFLOW PROGRAMMING

Slide 4

Slide 4 text

RUNNING EXAMPLE: A.K.A. Reactive PROGRAMMING

Slide 5

Slide 5 text

A PIPELINE $ find ~ | xargs grep hello | wc -l case class Node[T, R](f: T => R) // then, for each input t: T val output = f(t)

Slide 6

Slide 6 text

What about graphs ?

Slide 7

Slide 7 text

WE KNOW HOW TO DO THIS WITH fixed ARITY /* * ### E.g: ARITY 2 * * f is a **binary** function */ case class Node[T1,T2,R](f:(T1,T2) => R) val in1 = readInput1() val in2 = readInput2() // f.tupled is the **unary** function that takes the **pair** (T1,T2) val ft: Tuple2[T1, T2] => R = f.tupled ft( (in1, in2) ) // apply f.tupled to the pair

Slide 8

Slide 8 text

ARBITRARY ARITY case class Node[T1, T2, ..., TK, R](f: (T1, T2, ..., TK) => R) // 1) how do you construct a K-tuple, for any K ? // 2) what would the type signature for Node be ? Function1[T,R] Function2[T1,T2,R] Tuple2[T1,T2] ... ... Function22[T1,T2,...,T22,R] Tuple22[T1,T2,...,T22]

Slide 9

Slide 9 text

- Code Generation: (macros ?) - Error prone. How to test ? - Reflection - Runtime errors Maybe Shapeless could help

Slide 10

Slide 10 text

SHAPELESS

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Documentation

Slide 13

Slide 13 text

SHAPELESS WIKI «FACILITIES FOR ABSTRACTING OVER ARITY» import syntax.std.function._ import ops.function._ def applyProduct[P <: Product, F, L <: HList, R](p: P)(f: F) (implicit gen: Generic.Aux[P, L], fp: FnToProduct.Aux[F, L => R]) = f.toProduct(gen.to(p)) scala> applyProduct(1, 2)((_: Int)+(_: Int)) res0: Int = 3 scala> applyProduct(1, 2, 3)((_: Int)*(_: Int)*(_: Int)) res1: Int = 6

Slide 14

Slide 14 text

«FACILITIES FOR ABSTRACTING OVER ARITY» «Conversions between tuples and HList's, and between ordinary Scala functions of arbitrary arity and functions which take a single corresponding HList argument allow higher order functions to abstract over the arity of the functions and values they are passed»

Slide 15

Slide 15 text

WTF?

Slide 16

Slide 16 text

LISTS AND HETEROGENEOUS LISTS (HLISTS) val l = 1 :: "String" :: 2.0 :: Nil scala> val listFirst = l(0) listFirst: Any = 1 val hl = 1 :: "String" :: 2.0 :: HNil hl match { case x :: xs => ... // matches head and tail } scala> val first = hl(0) first: Int = 1 scala> val second = hl(1) second: String = "String" HLists also support map, folds, etc.

Slide 17

Slide 17 text

HLISTS AS TUPLES val t = (1, "String", 2.0) val hl = 1 :: "String" :: 2.0 :: HNil val (a, b, c) = t val x :: y :: z :: HNil = hl t match { case (1, s, _) => ... } hl match { case 1 :: s :: _ :: HNil => ... }

Slide 18

Slide 18 text

WITH arbitrary arity ! hl match { case 1 :: rest => ... // matches when head == 1 (any size) case x :: y :: rest => ... // matches size >= 2 case x :: y :: HNil => ... // matches when it is exactly a pair }

Slide 19

Slide 19 text

CONVERSIONS BETWEEN TUPLES, CASE CLASSES AND HLISTS val hl: String :: Int :: HNil = 1 :: "String" :: 2.0 :: HNil val t: (String, Int) = hl.tupled // to tuple val hl2: String :: Int :: HNil = t.productElements // to HList val p: Person = Person("John", 30) val hl3: String :: Int :: HNil = p.productElements // to HList

Slide 20

Slide 20 text

THE Generic[T] OBJECT // hl.tupled and t.productElements methods // are short-hands for: val gp = Generic[Person] val john = Person("John", 40) val hl: String :: Int :: HNil = gp.to(john) val p: Person = gp.from(hl) assert( john == p ) // works for tuples as well val gp = Generic[(String,Int)] val johnTuple = ("John", 40) val hl: String :: Int :: HNil = gp.to(johnTuple) val tp: (String,Int) = gp.from(hl) assert( johnTuple == tp )

Slide 21

Slide 21 text

▸ Tuples and case classes implement the Product trait ▸ You can convert between HLists, Tuples and case classes This is no surprise: they are all product types Similar to Cartesian product: A x B = { (a,b) | a ∊ A, b ∊ B }

Slide 22

Slide 22 text

/* * `Generic[P]` (here `Generic.Aux[P,L]`) * - converts between product types * - enables the `t.toProduct` syntax * What about `FnToProduct` ? */ def applyProduct[P <: Product, F, L <: HList, R](p: P)(f: F) (implicit gen: Generic.Aux[P, L], fp: FnToProduct.Aux[F, L => R] ) = f.toProduct(gen.to(p)) scala> applyProduct( (1, 2) )((x: Int, y: Int) => x + y) res0: Int = 3

Slide 23

Slide 23 text

K-ARY FUNCTIONS AND HLISTS f.tupled for a K-ary function = a unary function of one K-tuple tuples are fixed-size! we cannot partially match; but with HLists... import syntax.std.function._ import ops.function._ val fhl: String::Int::HNil => String = f.toProduct f.toProduct is shorthand syntax for val fp = FnToProduct[(String, Int) => String] val fhl = fp.apply(f)

Slide 24

Slide 24 text

BACK TO THE EXAMPLE def applyProduct[P <: Product, F, L <: HList, R](p: P)(f: F) (implicit gen: Generic.Aux[P, L], fp: FnToProduct.Aux[F, L => R] ) = f.toProduct(gen.to(p)) // we can understand this! val gen = Generic[Person] val fhl: String :: Int :: HNil => String = f.toProduct val hl: String :: Int :: HNil = gen.to(p) fhl(hl)

Slide 25

Slide 25 text

Wait, what about those .Aux thingamajigs ? Ok, let's rule this out very quick.

Slide 26

Slide 26 text

AN IMPLEMENTATION DETAIL: THE AUX ENCODING /* * When you create a Generic[P] instance * you want to let Scala infer the result HList type `L`. */ val gen = Generic[Person] // otherwise you would have to spell it out val gen = Generic[Person, String :: Int :: HNil] /* * when `Generic` is used to **enforce a constraint**, * then you might want to use `Generic.Aux[P,L]`. * * `Generic.Aux[P,L]` reads as "that `Generic[P]` instance that converts P into L" * FnToProduct.Aux[F, L => R] reads as "that FnToProduct[F] that converts F into L => R" */

Slide 27

Slide 27 text

wait, what ? are you saying that implicits can be used to enforce compile-time constraints ? how ?

Slide 28

Slide 28 text

      MAGIC

Slide 29

Slide 29 text

IMPLICITS ▸ Controversial feature ▸ Hard to grasp for beginners ▸ Can cause headaches even to experienced users. ▸ The term implicit may be arguable

Slide 30

Slide 30 text

      IMPLICITS

Slide 31

Slide 31 text

IMPLICIT VALUES & IMPLICIT DEFS // e.g. config objects implicit val implicitJohn = Person("John", 40) def somePersonString(implicit p: Person) = p.toString scala> somePersonString res0: String = "Person(John,40)" // *generate* values as they are needed implicit def personProvider = Person("Random Guy", (math.random * 100).toInt) def somePersonString(implicit p: Person) = p.toString scala> somePersonString res1: String = "Person(Random Guy,26)" scala> somePersonString res2: String = "Person(Random Guy,31)"

Slide 32

Slide 32 text

But implicits can be used as logic predicates

Slide 33

Slide 33 text

PROLOG

Slide 34

Slide 34 text

A QUICK PROLOG INTRODUCTION ▸ facts ▸ rules to derive new facts ▸ contained in the knowledge base (KB) ▸ Executing a Prolog program = solving the constraints in a query

Slide 35

Slide 35 text

Every man is mortal Sokrates is a man => Sokrates is mortal ∀ x: person(x) ! mortal(x) => person(sokrates) ! mortal(sokrates)

Slide 36

Slide 36 text

% fact person(sokrates). % rule mortal(X) :- person(X). ?- mortal(sokrates). true ?- mortal(X) X = sokrates

Slide 37

Slide 37 text

child, grandchild child(john, carl). child(carl, tom). grandchild(A, B) :- child(A, X), child(X, B). ∀ A, B, X: child(A,X) ∧ child(X,B) " grandchild(A,B)

Slide 38

Slide 38 text

% find the pair(s) X,Y % for which the relation holds ?- grandchild(X, Y). X = john Y = tom % find the john's grandchildren ?- grandchild(john, Y). Y = tom % find tom's grandparents ?- grandchild(X, tom). X = john

Slide 39

Slide 39 text

% false predicates: ?- grandchild(X, X). false ?- grandchild(tom, X). false ?- grandchild(X, john). false

Slide 40

Slide 40 text

INTRODUCING LOGIC SCALA ▸ A superset of Scala with 2 new keywords ▸ fact ▸ rule

Slide 41

Slide 41 text

DECLARING RELATIONS AND ATOMS trait Child[A,B] trait John trait Carl trait Tom

Slide 42

Slide 42 text

DECLARING FACTS // in Logic Scala, facts are named fact johncarl = new Child[John,Carl]{} // an instance of type Child fact carltom = new Child[Carl,Tom ]{} you are introducing into the knowledge base concrete evidences, witnesses (obj instances) that these facts hold true

Slide 43

Slide 43 text

DECLARING RULES trait GrandChild[A,B] // in Logic Scala, even rules are named rule grandChild[A,B,X]( facts xy: Child[A,X], yz: Child[X,B] ) = new GrandChild[A,B] {} rule[A,B,X](Child[A,X], Child[X,B]): GrandChild[A,B]

Slide 44

Slide 44 text

QUERYING LOGIC SCALA > query[ GrandChild[John, Tom] ] // (compiles; returns the fact instance) > query[ GrandChild[John, Carl] ] Compilation Failed

Slide 45

Slide 45 text

PLOT TWIST: LOGIC SCALA is ACTUALLY SCALA

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

trait John trait Carl trait Tom trait Child[T,U] implicit val john_carl = new Child[John,Carl]{} implicit val carl_tom = new Child[Carl,Tom ]{} trait GrandChild[T,U] implicit def grandChild[X,Y,Z]( implicit xy: Child[X,Y], yz: Child[Y,Z] ) = new GrandChild[X,Z] {} > implicitly[ GrandChild[John, Tom] ] // (compiles; returns the fact instance) > implicitly[ GrandChild[John, Carl] ] Compilation Failed

Slide 48

Slide 48 text

Scala | Prolog ---------------------------+----------------------------- implicit vals | facts implicit defs | rules type parameters in def | variables in a rule implicit parameter lists | bodies of a rule

Slide 49

Slide 49 text

applyProduct

Slide 50

Slide 50 text

can_apply_product(P, F, L, R) :- generic(P,L), fn_to_product(F, fn(L, R)). def applyProduct[P <: Product, F, L <: HList, R](p: P)(f: F) (implicit gen: Generic.Aux[P, L], fp: FnToProduct.Aux[F, L => R] ) = f.toProduct(gen.to(p)) scala> applyProduct(1, 2)((x: Int, y: Int) => x + y) res0: Int = 3

Slide 51

Slide 51 text

SECOND PLOT TWIST: Generic[P] AND FnToProduct[F] ARE TYPECLASSES

Slide 52

Slide 52 text

TYPECLASSES AS PREDICATES ▸ value of an implicit parameter = evidence that a predicate holds ▸ when one such an object instance provides methods, then it is called a typeclass. ▸ Generic[P] provides the methods to(P) and from(HList).

Slide 53

Slide 53 text

Q & A

Slide 54

Slide 54 text

REFERENCES ▸ Andrea Ferretti's dependent types encoded in Scala's type system ▸ Mark Harrah's series on type-level programming ▸ Stefan Zeiger's slides on Type-level Computations ▸ Sam Halliday's Shapeless For Mortals ▸ Miles Sabin's introductory talks ▸ Why is the Aux technique required for type-level computations? on StackOverflow ▸ Shapeless Wiki ▸ Shapeless Examples ▸ Eugene Yokota's “Herding Cats”

Slide 55

Slide 55 text

BONUS TRACK

Slide 56

Slide 56 text

Wait, aren't you forgetting something ? What about those .Aux thingamajigs ?

Slide 57

Slide 57 text

AN IMPLEMENTATION DETAIL: THE AUX ENCODING /* * When you create a Generic[P] instance, * you generally **do not** want to specify * the result HList type `L`. */ val gen = Generic[Person] // otherwise you would have to write val gen = Generic[Person, String :: Int :: HNil] /* * when `Generic` is used as a **predicate** (as an **implicit**), * then you can use `Generic.Aux[P,L]`. * You can think of it as if it were defined as follows (pseudo-code) */ trait Generic[P] { type Out <: HList } type Generic.Aux[P,L] = Generic[P] { type Out = L }

Slide 58

Slide 58 text

Generic.Aux[P,L] is sometimes called a type extractor because it extracts the type member value Generic.Out by raising it into the signature of type Generic.Aux.

Slide 59

Slide 59 text

PROLOG ENCODING generic(P, L) :- product(P), (X, Y) = P, hlist(L), [X, Y] = L. % in short: generic((X,Y), [X,Y]).

Slide 60

Slide 60 text

% let us represent function signatures with fn fn((arg_type1, arg_type2, ..., arg_typeN), ret_type) fn_to_product(F, fn(L, R)) :- fn(Args,R) = F, generic(Args, L). % in short: fn_to_product(fn(Args,R), L, R) :- generic(Args, L).

Slide 61

Slide 61 text

generic((X,Y), [X,Y]). fn_to_product(fn(Args,R), L, R) :- generic(Args, L). can_apply_product(P, F, L, R) :- generic(P,L), fn_to_product(F, fn(L, R)).