Real World Scalaz

Scalaz is a highly controversial library in the Scala community often used to argue that Scala has a high learning curve and is impractical. Even those who do not feel this way find it hard to use Scalaz practically in their day jobs. Over the past 6 to 8 months Scalaz has become so prominent in our architecture that every incoming request to our service leverages Scalaz in some way. Our new engineers get their hands dirty with Scalaz on the first day, even the engineers who have not written a line of Scala before.

The talk will explore how and why we began using Scalaz, the benefits gained from adopting it, and why your codebase could benefit from it too. This talk will focus on the road we travelled to adopt, integrate and teach the library within our organization. It will also examine the practical parts of the library that we find most useful, while intentionally omitting discussion around portions that we have not yet put to use in production. In addition, it will briefly explore libraries we have used and are writing ourselves that are built on Scalaz. We will also discuss the transformation that is happening with Scalaz 7, why we still use Scalaz 6, and what portions of Scalaz 7 we look forward to.

Transcript

1. REAL W⊛RLD SCALAZ http://speakerdeck.com/u/jrwest/p/real-world-scalaz changed talk title (“practical scalaz” by Chris Marshall http://skillsmatter.com/podcast/scala/practical-scalaz-2518)
2. “If you are thinking of using Scalaz, stop now while you still have your sanity" * context: java devs learning scala * source: http://zeroturnaround.com/blog/scala-sink-or-swim-part-1/#scalaz
3. “How does this help you ship features or improve your product?” * context: commenting on the use of lenses * source: https://twitter.com/#!/coda/status/167794305107361796)
4. “It will not immediately help you write better programs that are easy to understand” * hear/read this often * source: http://zeroturnaround.com/blog/scala-sink-or-swim-part-1/#scalaz
5. “There are two types of [libraries]: the ones people hate and the ones nobody uses” - unknown * applies to scala too * know correct attribution? contact me
6. HOW & WHY? * those quotes pretty much boil down to these two questions: * how: apply tools given by scalaz to real world problems? to handle human scale? * why would we choose this solution over other ones?
7. @_jrwest github.com/jrwest blog.loopedstrange.com * a little context before we talk about the “how & why?”
8. I WORK @ * small team, < 15 engineers * many have java bg, but not all, most w/ little to no scala experience * most have oop bg, varying knowledge of FP topics
9. WE USE SCALA { a lot } * 15+ public facing services supported by many other internal modules/libs * most built on netty, also I/O bound * originally java, now > 70% scala
10. libraryDependencies += “org.scalaz” %% “scalaz-core” % “6.0.3” * almost all our services depend on scalaz * usage levels and parts of scalaz used vary among services * blocked on upgrading to 6.0.4 b/c of binary compatibility issues
11. SOME EXAMPLES
12. JSON Mapping def FileManifestJSONR = new JSONR[AppID=>FileManifest] { def read(json: JValue) = manifestNoId.applyJSON( field(jsonManifestKey), field(changeLogEntryCachedJsonKey) )(json) } } * lift-json-scalaz (https://github.com/lift/framework/tree/master/core/json-scalaz) * validationnel applicative, error accumulation = better user-facing errors & debugging * composition
13. for { mbExists <- manifestFiles member path _ <- manifestFiles -= path _ <- changeLogEntryFiles -=? (path, mbExists >| false | true) } yield mbExists S @-@ A / State[S,A] * easy manipulation of highly nested case classes (ex: file manifests) * better implementation hiding * rich libraries (e.g. MapLens, SetLens, & provided state actions)
14. sideEffect.pure[IO] type V[X]=ValidationNEL[Throwable,X] def fetchKeys(keys: List[String]) = { keys.map(bucket.fetch[CustomDomain](_)) .sequence .map(_.sequence[V,Option[CustomDomain]]) } * can’t guarantee purity, can delineate impurity * examples: storing data in riak, uploading files to s3 and phonegap build svc * composition of IO makes side-effecting code much more reusable
15. NonEmptyList[A] Pure[P[_]] OptionW[A] Kleisli[M[_],A,B] Functor[F[_]] Apply[A[_]] Traverse[T[_]] Bind[Z[_]] Semigroup[A] Zero[A] Tree[A] Foldable[F[_]] Identity MA[M[_],A] MAB[M[_],A,B] * we make use of a lot more as well * data structures like Validation and NonEmptyList * most type classes and “enrichments”
16. ITS NOT ALL OR NOTHING * shades of grey, not black and white * scalaz is really many libraries, use the parts you want * we started using “encrichments” and some data structures, grew from there -- original image: http://www.chicagonow.com/chicago-tough/files/2012/03/chipsallin.jpg
17. MAKE BORDERS * don’t abuse scala’s flexibility & power * toss pure/impure code into a salad bowl & mixing is a bad idea * define borders where pure code meets impure code -- original image: http://media.nowpublic.net/images//59/4/59455eabd625a15e2afb583dcfbb27ce.jpg
18. Public Interface Implementation Dependency Calls * your code can be pure * public interface is the border, it runs IO, throws exceptions, etc * existing libraries probably aren’t pure -- many are java after all
19. Outside World Business Logic Data Access * communicate with clients via HTTP * the most important part -- your code -- make pure* * perform side-effects like talk to database, read/write files, etc
20. Impure Pure Impure * “the sandwich” - common pattern in our code -- original image: http://fastfood.ocregister.com/files/2010/05/ice-cream-sandwich1.jpg
21. :) * code has become much more DRY/modular and as a result more unit testable * concurrent code becomes “simpler” to reason about when pure * FP makes reasoning about bugs simpler, narrow down cause faster
22. :( * scala hardsips: bin. compatability problems, tooling deficiencies, type inference limits * import scalaz._; import Scalaz._ adds a lot of clutter in the current scope * lack of docs/related content (in Scala!)
23. scalaz7 * not currently used in our production systems but one or two of my side projects * much better for teaching (explicit type class instance usage -- w/o syntax ext.) * addresses some of the issues we have like importing “the scalaz world”
24. “SCALE”?
25. “Human Scale” * the complexity comments may be FUD, but that does not mean there is not stuff to learn here * how do we help new employees learn scala & scalaz?
26. HIRE PEOPLE WHO WANT TO LEARN * hiring from all backgrounds instead of hiring a “scala or java developer” is much easier * those people need to be open to exploring and learning new tech/methods
27. Shared Resources * easy win, shares effort in finding good resources * dropbox full of downloaded papers and books purchased by the company * internal talks available on github
28. LEARN TOGETHER * code review/pull requests * face to face discussion on a whiteboard * team talks on fridays, any one can attend, anyone can talk, neither mandatory
29. ON YOUR OWN * RTFC -- in the end it all just code * apply what you know, this is what really makes you learn it * internal talks mostly make you aware of it, give you starting point, application is key
30. ITS NOT ALL OR NOTHING * focus on understanding a part of the library * break that part down, use it in the repl in a few cases (w/ a few types, examples, etc) * as you learn more about each part and then more parts the bigger picture emerges -- original image: http://www.chicagonow.com/chicago-tough/files/2012/03/chipsallin.jpg
31. Learn You a ?? * not necessary * understanding syntax, basics can be useful while learning concepts until scala docs fill out -- haskell logo: http://www.haskell.org/wikistatic/haskellwiki_logo.png
32. ASK #scalaz FOR HELP
33. BREAK RULES * you may hear things you cannot do, “validation is not a monad” * ignore and explore yourself -- these are the real learning opportunites * understand how to implement validation monad and why no error accum. happens
34. PAYING UPFRONT * of course this have some upfront cost * so does writing tests and similarly a good balance pays off
35. A1 A2 A3 L1 L2 L3 Level Up * scalaz covers most advanced features in scala language * great guide for library and code design (to an extent). Scalaz guys have done serious amount of research in this regard
36. Thanks! talk recap slide
37. SOME RESOURCES • Atlassian’s Typeclassopedia - http://typeclassopedia.bitbucket.org • Apocalisp Blog - http://apocalisp.wordpress.com/ • “Functional Programming in Scala” MEAP - http://manning.com/bjarnason/ • Eric Torreborre’s Blog - http://etorreborre.blogspot.com/ • StackMob Engineering Blog - http://www.stackmob.com/category/engineering/ • Learn You a Scalaz (my WIP, about to undergo major rework) - https://github.com/jrwest/learn-you-a-scalaz • Chris Marshall’s “Practical Scalaz” Talk - http://skillsmatter.com/podcast/scala/ practical-scalaz-2518