Recursion schemes and fixed point data types

Its always challenge to build a good abstraction during work on library. Library codebase must be flexible and modular when user code must be clean and easy understandable. One of variants to build API between library and customer its Free evaluation approach. In that case you as developer is free to interpret customer actions as you want and customer is free to define actions as he want. In my talk I will show how to build API based on recursive abstract data types, describe why its cool and demonstrate some tricks on type system to make customer experience with library better.

Transcript

1. Recursion schemes in Scala and also fixed point data types
2. My name is Arthur and I'm increasing technical debt in Scala projects about 3 years already leading platform for selling cars in EU AWS based Scala microservices autonomous teams and you build you run https://github.com/AutoScout24/hiring Yo
3. Today we will speak about freaky stuff, that useless in day-to-day job, enjoy :) *use this slide in case of WTF Btw
4. everywhere! Recursive schemas Lists and trees Filesystems Databases list match { // Cons(1, Cons(2, Cons(3, Nil))) case 1 :: 2 :: 3 :: Nil => case Nil => }
5. Lets find problem on our head 01
6. Equals And Or Logic expression evaluation
7. Lets define our ADT sealed trait Expr case class StringValue(value: String) extends Expr case class BooleanValue(value: Boolean) extends Expr case class Equals(field: String, expr: Expr) extends Expr case class And(expr: Seq[Expr]) extends Expr case class Or(expr: Seq[Expr]) extends Expr
8. Example equation Or ( And ( Equals ( "firstName", StringValue("A") ), Equals ( "lastName", StringValue("B") ) ), BooleanValue(true) ) (firstName == "A" && lastName == "B") || true
9. val evaluateSql: Expr => String = { case StringValue(value) => "\"" + value + "\"" case BooleanValue(value) => value.toString case Equals(field, expr) => s"$field = ${evaluateSql(expr)}" case And(expr) => "(" + expr.map(evaluateSql).mkString(" AND ") + ")" case Or(expr) => "(" + expr.map(evaluateSql).mkString(" OR ") + ")" }
10. val evaluateQuery: Expr => String = { case StringValue(value) => "\"" + value + "\"" case BooleanValue(value) => value.toString case Equals(field, expr) => s"$field == ${evaluateQuery(expr)}" case And(expr) => "(" + expr.map(evaluateQuery).mkString(" && ") + ")" case Or(expr) => "(" + expr.map(evaluateQuery).mkString(" || ") + ")" }
11. It will work! but... evaluateSql(expression) // ((firstName = "A" AND lastName = "B") OR true) evaluateQuery(expression) // ((firstName = "A" && lastName = "B") || true)
12. There are a lot of mess And not only... case Equals(field, expr) => s"$field = ${evaluateSql(expr)}"
13. Problem of partial interpretation val optimize: Expr => Expr = { case Or(expr) if expr.contains(BooleanValue(true)) => BooleanValue(true) case other => ??? } So how we can improve?
14. throwing traversing out 02
15. Generalize it sealed trait Expr[T] case class StringValue[T](value: String) extends Expr[T] case class BooleanValue[T](v: Boolean) extends Expr[T] case class Equals[T](f: String, v: T) extends Expr[T] case class And[T](expr: Seq[T]) extends Expr[T] case class Or[T](expr: Seq[T]) extends Expr[T]
16. val sqlAlgebra: Expr[String] => String = { case StringValue(v) => s""""$v"""" case BooleanValue(v) => v.toString case Equals(field, expected) => s"$field = $expected" case Or(expr) => "(" + expr.mkString(" OR ") + ")" case And(expr) => "(" + expr.mkString(" AND ") + ")" } Clean interpreter
17. val expression = And( Equals( "fieldName", StringValue("A") ) ) Looks nice
18. val expression: Expr[Expr[Expr[Unit] = And( Equals( "fieldName", StringValue("A") ) ) def myPerfectFunction(expr: Expr[???]) Oh...types
19. its like a hiding of part that doesn't matter fixed point types
20. F[_] is a generic type with a hole, after wrapping of Expr by this we will have Fix[Exp] type. let's add typehack case class Fix[F[_]](unFix: F[Fix[F]])
21. Fix...waaat? case class Fix[F[_]](unFix: F[Fix[F]]) // Fix[Expr] And[Expr[???]](expr1, expr2, exprN) Fix(And[Expr[???]])(expr1, expr2, exprN)) Fix(And[Fix[Expr]])(Fix(expr1), Fix(expr2))) StringValue("A") Fix(StringValue[???]("A")) Fix(StringValue[Fix[Expr]]("A"))
22. Hacked code type LogicalExpr = Fix[Expr] val expression: LogicalExpr = Fix(And( Fix(Equals( "fieldName", Fix(StringValue("A")) )) )) def myPerfectFunction(expr: LogicalExpr)
23. object Expr { def string(v: String) = Fix(StringValue[Fix[Expr]](v)) def eq(f: String, v: Fix[Expr]) = Fix(Equals(f, v)) } val expression: LogicalExpr = Expr.eq("fieldName", Expr.string("testField") ) Let's do it cleaner
24. searching for thrown out traversing logic 03
25. def map[F[_], A, B](fa: F[A])(f: A=>B): F[B] Functor 911
26. Scalaz example case class Container[T](data: T) implicit val functor = new Functor[Container] { override def map[A, B](fa: Container[A]) (f: A => B): Container[B] = Container(f(fa.data)) } functor.map(Container(1))(_.toString) // "1" For cats you will have same code
27. implicit val functor = new Functor[Expr] { override def map[A, B](fa: Expr[A])(f: (A) => B): Expr[B] = fa match { case StringValue(v) => StringValue(v) case BooleanValue(v) => BooleanValue(v) case Equals(field, e) => Equals(field, f(e)) case Or(expressions) => Or(expressions.map(f)) case And(expressions) => And(expressions.map(f)) } }
28. morphisms 04
29. “Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire”, by Erik Meijer generalized folding operation catamorphism
30. Cata...waat? def cata[F[_], T]( expr: Fix[F], functor: Functor[F], interpretor: F[T] => T): T = interpretor( functor.map(expr.unfix)( cata(_, functor, interpretor) // Fix[Expr] => T ) )
31. So as a result val expression = Expr.or( Expr.and( Expr.eq("firstName", Expr.string("A")), Expr.eq("lastName", Expr.string("B")) ), Expr.boolean(true) ) cata(expression, functor, sqlAlgebra) // ((firstName = "A" AND lastName = "B") OR true)
32. do you remember about partial interpretation? No more pain with copy-paste code! Extra profit val optimize: Expr[Fix[Expr]] => Fix[Expr] = { case Or(expr) if expr.contains(Expr.boolean(true)) => Expr.boolean(true) case other => Fix(other) }
33. morphism from Matryoshka library https://github.com/slamdata/matryoshka Out of the box implicit val functor: Functor[Expr] val sqlAlgebra: Algebra[Expr, String] val expression: Fix[Expr] import matryoshka.implicits._ expression.cata(sqlAlgebra)
34. Easily extendable API No copy-paste logic Partially evaluation is easy as possible Everything hidden under abstraction Pros
35. Builder based on types and macroses Expr.equals[User](_.firstName, "A") Better syntax, smth like a Slick API Folding base on non-recursive algorithm Things to improve
36. we are finally here any questions? Arthur Kushka // @arhelmus