Leveraging Scala Macros for Better Validation

A talk given at ScalaDays 2015 in San Francisco, California.

Data validation is a common enough problem that numerous attempts have been made to solve it elegantly. The de-facto solution in Java (JSR 303) has a number of shortcomings and fails to leverage the powerful Scala type system. The release of Scala 2.10.x introduced a couple of experimental metaprogramming features, namely reflection and macros. In this talk I'll introduce macros by way of a practical example: implementing a full-blown data validation engine, utilizing def macros and a Scala DSL to enable elegant validator definition syntax and call-site.

Transcript

1. Leveraging Scala Macros for Better Validation Tomer Gabel, Wix March 2015
2. I Have a Dream •  Definition: case  class  Person(      firstName:  String,      lastName:  String     )     implicit  val  personValidator  =        validator[Person]  {  p  ⇒          p.firstName  is  notEmpty          p.lastName  is  notEmpty      }  
3. I Have a Dream •  Usage: validate(Person("Wernher",  "von  Braun"))        ==  Success     validate(Person("",  "No  First  Name"))        ==  Failure(Set(RuleViolation(                value              =  "",                constraint    =  "must  not  be  empty",                description  =  "firstName"            )))
4. ENTER: ACCORD.
5. Basic Architecture API Combinator Library DSL Macro Transformation
6. The Accord API •  Validation can succeed or fail •  A failure comprises one or more violations sealed  trait  Result   case  object  Success  extends  Result   case  class  Failure(violations:  Set[Violation])      extends  Result •  The validator typeclass: trait  Validator[-­‐T]  extends  (T  ⇒  Result)
7. Why Macros? •  Quick refresher: implicit  val  personValidator  =      validator[Person]  {  p  ⇒          p.firstName  is  notEmpty          p.lastName  is  notEmpty      }   Implicit “and” Automatic description generation
8. Full Disclosure Macros are (sort of) experimental Macros are hard I will gloss over a lot of details … and simplify many things
9. Abstract Syntax Trees •  An intermediate representation of code – Structure (semantics) – Metadata (e.g. types) – optional! •  Provided by the reflection API •  Alas, mutable – Until scala.meta comes along…?
10. Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase    
11. Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase       Apply(      Select(          Ident(newTermName("param")),          newTermName("toUpperCase")      ),      List()   )  
12. Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase       ValDef(      Modifiers(PARAM),        newTermName("param"),        Select(          Ident(scala.Predef),          newTypeName("String")      ),      EmptyTree            //  Value   )
13. Abstract Syntax Trees def  method(param:  String)  =  param.toUpperCase       DefDef(      Modifiers(),        newTermName("method"),        List(),                  //  Type  parameters      List(                      //  Parameter  lists          List(parameter)      ),        TypeTree(),          //  Return  type      implementation   )
14. Def Macro 101 •  Looks and acts like a normal function def  radix(s:  String,  base:  Int):  Long   val  result  =  radix("2710",  16)   //  result  ==  10000L   •  Two fundamental differences: – Invoked at compile time instead of runtime – Operates on ASTs instead of values
15. Def Macro 101 •  Needs a signature & implementation def  radix(s:  String,  base:  Int):  Long  =      macro  radixImpl     def  radixImpl      (c:  Context)      (s:  c.Expr[String],  base:  c.Expr[Int]):      c.Expr[Long] Values ASTs
16. Def Macro 101 •  What’s in a context? –  Enclosures (position) –  Error handling –  Logging –  Infrastructure
17. Basic Architecture API Combinator Library DSL Macro Transformation
18. Overview implicit  val  personValidator  =      validator[Person]  {  p  ⇒          p.firstName  is  notEmpty          p.lastName  is  notEmpty      }     •  The validator macro: – Rewrites each rule by addition a description – Aggregates rules with an and combinator Macro Application Validation Rules
19. Signature def  validator[T](v:  T  ⇒  Unit):  Validator[T]  =      macro  ValidationTransform.apply[T]               def  apply[T  :  c.WeakTypeTag]      (c:  Context)      (v:  c.Expr[T  ⇒  Unit]):      c.Expr[Validator[T]]
20. Brace yourselves Here be dragons
21. Walkthrough Search for rule Process rule Generate description Rewrite rule
22. Walkthrough Search for rule Process rule Generate description Rewrite rule
23. Search for Rule •  A rule is an expression of type Validator[_] •  We search by: – Recursively pattern matching over an AST – On match, apply a function on the subtree – Encoded as a partial function from Tree to R
24. Search for Rule def  collectFromPattern[R]              (tree:  Tree)              (pattern:  PartialFunction[Tree,  R]):  List[R]  =  {      var  found:  Vector[R]  =  Vector.empty      new  Traverser  {          override  def  traverse(subtree:  Tree)  {              if  (pattern  isDefinedAt  subtree)                  found  =  found  :+  pattern(subtree)              else                  super.traverse(subtree)          }      }.traverse(tree)      found.toList   }
25. Search for Rule •  Putting it together: case  class  Rule(ouv:  Tree,  validation:  Tree)     def  processRule(subtree:  Tree):  Rule  =  ???     def  findRules(body:  Tree):  Seq[Rule]  =  {      val  validatorType  =  typeOf[Validator[_]]        collectFromPattern(body)  {          case  subtree  if  subtree.tpe  <:<  validatorType  ⇒              processRule(subtree)      }   }  
26. Walkthrough Search for rule Process rule Generate description Rewrite rule
27. Process Rule •  The user writes: p.firstName  is  notEmpty •  The compiler emits: Contextualizer(p.firstName).is(notEmpty) Object Under Validation (OUV) Validation Type: Validator[_]
28. Process Rule Contextualizer(p.firstName).is(notEmpty) •  This is effectively an Apply AST node •  The left-hand side is the OUV •  The right-hand side is the validation –  But we can use the entire expression! •  Contextualizer is our entry point
29. Process Rule Contextualizer(p.firstName).is(notEmpty) Apply Select Apply TypeApply Contextualizer String Select Ident(“p”) firstName is notEmpty
30. Process Rule Contextualizer(p.firstName).is(notEmpty) Apply Select Apply TypeApply Contextualizer String Select Ident(“p”) firstName is notEmpty
31. Process Rule Apply TypeApply Contextualizer String Select Ident(“p”) firstName
32. Process Rule Apply TypeApply Contextualizer Φ Select Ident(“p”) firstName
33. Process Rule Apply TypeApply Contextualizer Φ Select Ident(“p”) firstName
34. Process Rule case  Apply(TypeApply(Select(_,  `term`),  _),  ouv  ::  Nil)  ⇒   Apply TypeApply Contextualizer Φ OUV Φ Φ
35. Process Rule •  Putting it together: val  term  =  newTermName("Contextualizer")     def  processRule(subtree:  Tree):  Rule  =      extractFromPattern(subtree)  {          case  Apply(TypeApply(Select(_,  `term`),  _),  ouv  ::  Nil)  ⇒              Rule(ouv,  subtree)      }  getOrElse  abort(subtree.pos,  "Not  a  valid  rule")  
36. Walkthrough Search for rule Process rule Generate description Rewrite rule
37. Generate Description Contextualizer(p.firstName).is(notEmpty) •  Consider the object under validation •  In this example, it is a field accessor •  The function prototype is the entry point Select Ident(“p”) firstName validator[Person]  {  p  ⇒      ...   }
38. Generate Description •  How to get at the prototype? •  The macro signature includes the rule block:   def  apply[T  :  c.WeakTypeTag]      (c:  Context)      (v:  c.Expr[T  ⇒  Unit]):      c.Expr[Validator[T]]   •  To extract the prototype: val  Function(prototype  ::  Nil,  body)  =        v.tree          //  prototype:  ValDef
39. Generate Description   •  Putting it all together: def  describeRule(rule:  ValidationRule)  =  {      val  para  =  prototype.name      val  Select(Ident(`para`),  description)  =          rule.ouv      description.toString   }  
40. Walkthrough Search for rule Process rule Generate description Rewrite rule
41. Rewrite Rule •  We’re constructing a Validator[Person] •  A rule is itself a Validator[T]. For example: Contextualizer(p.firstName).is(notEmpty)   •  We need to: – Lift the rule to validate the enclosing type – Apply the description to the result
42. Quasiquotes •  Provide an easy way to construct ASTs: Apply(      Select(          Ident(newTermName"x"),          newTermName("$plus")      ),        List(          Ident(newTermName("y"))      )   )     q"x  +  y"    
43. Quasiquotes •  Quasiquotes also let you splice trees: def  greeting(whom:  c.Expr[String])  =      q"Hello  \"$whom\"!" •  And can be used in pattern matching: val  q"$x  +  $y"  =  tree
44. Rewrite Rule Contextualizer(p.firstName).is(notEmpty)   new  Validator[Person]  {      def  apply(p:  Person)  =  {          val  validation  =              Contextualizer(p.firstName).is(notEmpty)          validation(p.firstName)  withDescription  "firstName"      }   }
45. Rewrite Rule •  Putting it all together: def  rewriteRule(rule:  ValidationRule)  =  {      val  desc  =  describeRule(rule)      val  tree  =  Literal(Constant(desc))      q"""      new  com.wix.accord.Validator[${weakTypeOf[T]}]  {          def  apply($prototype)  =  {              val  validation  =  ${rule.validation}              validation(${rule.ouv})  withDescription  $tree          }      }      """   }  
46. The Last Mile
47. Epilogue •  The finishing touch: and combinator def  apply[T  :  c.WeakTypeTag]      (c:  Context)      (v:  c.Expr[T  ⇒  Unit]):  c.Expr[Validator[T]]  =  {        val  Function(prototype  ::  Nil,  body)  =  v.tree      //  ...  all  the  stuff  we  just  discussed        val  rules  =  findRules(body)  map  rewriteRule      val  result  =          q"new  com.wix.accord.combinators.And(..$rules)"      c.Expr[Validator[T]](result)   }
48. WE’RE DONE HERE! Thank you for listening tomer@tomergabel.com @tomerg http://il.linkedin.com/in/tomergabel Check out Accord at: http://github.com/wix/accord