Confessions of a Ruby Developer Whose Heart was Stolen by Scala

Talk @ Scala Days 2013

Transcript

1. Confessions of a Ruby Developer Whose Heart was Stolen by Scala Ryan LeCompte @ryanlecompte Scala Days 2013 Tuesday, June 11, 13
2. Background • Scala developer at Quantifind • Author of redis_failover • http://github.com/ryanlecompte • Twitter: @ryanlecompte • ryan@quantifind.com Tuesday, June 11, 13
3. “Ruby and JavaScript are dynamically typed, and types and objects can change shape at runtime. This is a confluence of all the hardest-to-optimize language characteristics. In both cases, the best we can do is to attempt to predict common type and object shapes and insert guards for when we're wrong, but it's not possible to achieve the performance of a system with fully-predictable type and object shapes. Prove me wrong.” - Charles Nutter (JRuby Lead Developer) Tuesday, June 11, 13
4. #Scala IRC channel “I think there is a threshold beyond which it is impossible to seriously change any program written in a dynamic language" Tuesday, June 11, 13
5. #Scala IRC channel “You end up patching around the edges to avoid unintended consequences, and eventually it's a big ball of sticky tape surrounding the original program" Tuesday, June 11, 13
6. #Scala IRC channel “Tests, grep, prayer." Tuesday, June 11, 13
7. Ruby • Dynamic, object-oriented • Expressive, concise, flexible • Multiple implementations (MRI, JRuby, Rubinius) • Objects, classes, modules (mixins), blocks (closures) • Some functional constructs in collections library (each_cons, map, flat_map, filter) Tuesday, June 11, 13
8. Scala • Object-oriented / functional hybrid • Immutability encouraged • Concurrency first-class citizen • Statically typed with powerful type inference • Objects, classes, traits (mixins), functions Tuesday, June 11, 13
9. Ruby’s Philosophy • Programmer should always be “happy” when programming in Ruby • Duck typing (i.e., who cares what your actual “type” is) • Malleable language (class definitions modifiable at runtime, methods can be injected virtually anywhere) Tuesday, June 11, 13
10. Ruby Modules vs. Scala Traits • Ruby’s modules are akin to Scala’s traits • Modules can be mixed into classes and other modules • Modules can be mixed into instances after they are created • Encapsulate cross-cutting concerns Tuesday, June 11, 13
11. Ruby Module Example module RedisFailover # Base class for strategies that determine which # node is used during failover. module FailoverStrategy include Util # Returns a candidate node as determined by this strategy. # # @param [Hash<Node, NodeSnapshot>] the node snapshots # @return [Node] the candidate node or nil if one couldn't be found def find_candidate(snapshots) raise NotImplementedError end end end No way to define abstract methods Types encoded in comments (tomdoc) Tuesday, June 11, 13
12. Module Mixin Abuse module ActiveRecord class Base extend ActiveModel::Naming extend ActiveSupport::Benchmarkable extend ActiveSupport::DescendantsTracker extend ConnectionHandling extend QueryCache::ClassMethods extend Querying extend Translation extend DynamicMatchers extend Explain include Persistence include ReadonlyAttributes include ModelSchema include Inheritance include Scoping include Sanitization include AttributeAssignment # 19 more modules included ... end end Separate concerns are jammed together in a single namespace Tuesday, June 11, 13
13. Where is the method defined? module M1 def foo; puts "M1 foo"; end end module M2 def foo; puts "M2 foo"; end end module M3 def foo; puts "M3 foo"; end end class A def foo; puts "A foo"; end end class B < A include M1 include M2 def foo; puts "B foo"; end end b = B.new b.extend(M3) b.foo Which foo implementation gets used? No IDE to help you. Must rely on runtime introspection (i.e., method_locator gem) Tuesday, June 11, 13
14. Scala Traits • Never guess where a method is defined • Avoid mixin abuse with better abstractions (e.g. type classes, stackable traits, monads) • Encode dependencies and constraints using the type system (e.g., cake pattern) Tuesday, June 11, 13
15. Monkey patching in Ruby • Very common practice in the Ruby/Rails community for providing new behavior for existing classes • Often takes the form of modules that are injected / mixed into existing classes from other libraries • Often invasive and munges together disparate concerns Tuesday, June 11, 13
16. Example: Storage units 250.bytes 1.megabyte 1.petabyte Integers don’t have storage unit methods by default. Goal is to somehow extend the numeric type to support them. Tuesday, June 11, 13
17. Ruby on Rails Approach class Numeric KILOBYTE = 1024 MEGABYTE = KILOBYTE * 1024 GIGABYTE = MEGABYTE * 1024 def kilobytes self * KILOBYTE end alias :kilobyte :kilobytes def megabytes self * MEGABYTE end alias :megabyte :megabytes def gigabytes self * GIGABYTE end alias :gigabyte :gigabytes end Ruby’s Numeric class is re-opened and directly modified. What if other libraries do the same thing at runtime? Collisions. Tuesday, June 11, 13
18. Monkey patching in Scala • Implicit conversions are a safe compile-time enrichment for a particular type • Only one implicit conversion can be used for a given call site • IDE can inform you which implicit conversion is in use for a particular method Tuesday, June 11, 13
19. Scala Approach package com.twitter.conversions object storage { implicit class RichWholeNumber(wrapped: Long) { def byte = bytes def bytes = new StorageUnit(wrapped) def kilobyte = kilobytes def kilobytes = new StorageUnit(wrapped * 1024) def megabyte = megabytes def megabytes = new StorageUnit(wrapped * 1024 * 1024) def gigabyte = gigabytes def gigabytes = new StorageUnit(wrapped * 1024 * 1024 * 1024) } // user code: bring in the implicit conversion at compile-time, not run-time import com.twitter.conversions.storage._ println(s"1 gigabyte is ${1.gigabyte.inBytes} bytes") Long numeric type isn’t directly modified. Namespace stays clean! Tuesday, June 11, 13
20. will_paginate library Pagination behavior mixed directly into ActiveRecord base class # mix everything into Active Record ::ActiveRecord::Base.extend PerPage ::ActiveRecord::Base.extend Pagination ::ActiveRecord::Base.extend BaseMethods # a new Post model now has pagination support class Post < ActiveRecord::Base end Post.page(params[:page]).order('created_at DESC') Tuesday, June 11, 13
21. Type classes • Compiler discovers and injects the new behavior for you • Safer alternative to Ruby monkey patching when providing new functionality for existing types • Namespace isn’t further bloated with new methods / behavior since they live in type class instances Tuesday, June 11, 13
22. Pagination with Type Classes trait Pager[A] { def page(model: A, page: Int): Seq[A] } object Pager { implicit object ModelPager extends Pager[Model] { def page(model: Model, page: Int): Seq[Model] = { // default DB pagination } } } object Example { def process[A](model: A, page: Int)(implicit pager: Pager[A]) { val records = pager.page(model, page) // do something with records } } Pager typeclass Default implementation Compiler supplies our Pager instance Tuesday, June 11, 13
23. Extending core language libraries • Ruby’s approach usually involves injecting a custom module into core collection modules / classes (e.g., Enumerable) or direct modification • Scala’s approach: implicit conversions (safer, less invasive than direct modification) Tuesday, June 11, 13
24. Ruby: Adding Hash#except and String#underscore {'a' => 1, 'b' => 2, 'c' => 3}.except('a', 'c') {'b' => 2} 'HelloThereEveryone'.underscore "hello_there_everyone" Tuesday, June 11, 13
25. Extending Ruby collections # example from will_paginate gem unless Hash.method_defined? :except Hash.class_eval do def except(*keys) has_convert = respond_to?(:convert_key) rejected = Set.new(has_convert ? keys.map { |k| convert_key(k) } : keys) reject { |key,| rejected.include?(key) } end end end unless String.method_defined? :underscore String.class_eval do def underscore self.to_s.gsub(/::/, '/'). gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2'). gsub(/([a-z\d])([A-Z])/,'\1_\2'). tr("-", "_"). downcase end end end Brittle approach: Use of class_eval and respond_to? to directly modify classes at runtime Tuesday, June 11, 13
26. Scala: cycle & sample Vector(1,2,3).cycle.take(10).mkString(",") => 1,2,3,1,2,3,1,2,3,1 val nums = 0 to 1000 nums.sample(5) => IndexedSeq[Int] = WrappedArray(675, 510, 795, 169, 886) Tuesday, June 11, 13
27. cycle implicit class RichSeq[A, C[A] <: Seq[A]](underlying: C[A]) { def cycle: Iterator[A] = { lazy val circular: Stream[A] = underlying.toStream #::: circular circular.iterator } } Tuesday, June 11, 13
28. sample implicit class RichSeq[A: ClassManifest, C[A] <: Seq[A]](underlying: C[A]) { // see https://en.wikipedia.org/wiki/Reservoir_sampling def sample(k: Int): IndexedSeq[A] = { val rnd = new Random val elems = new Array[A](math.min(k, underlying.size)) // avoiding zipWithIndex below for performance var idx = 0 underlying.foreach { elt => if (idx < k) elems(idx) = elt else { val nextIdx = rnd.nextInt(idx + 1) if (nextIdx < k) elems(nextIdx) = elt } idx += 1 } elems } } Tuesday, June 11, 13
29. Better Abstractions • Tail-recursive methods • For expressions • Pattern matching & custom extractors • Stackable traits Tuesday, June 11, 13
30. @tailrec methods groupRuns(Vector(1, 1, 3, 3, 4, 2, 2, 5, 6)) List(Vector(1, 1), Vector(3, 3), Vector(4), Vector(2, 2), Vector(5),Vector(6)) @tailrec def groupRuns[A](c: Seq[A], acc: Seq[Seq[A]] = Seq.empty): Seq[Seq[A]] = { c match { case Seq() => acc case xs => val (same, rest) = xs.span { _ == xs.head } groupRuns(rest, acc :+ same) } } Tuesday, June 11, 13
31. For expressions import scala.io.Source import scala.util.control.Exception.allCatch case class Record(blogUrl: String, postId: String, userId: String) // build lazy stream of parsed & valid records def parseRecords(path: String): Stream[Record] = { for { line <- Source.fromFile(path).getLines.toStream data <- allCatch.opt { parse[Map[String, String]](line) } text <- data.get("text") if text.trim.nonEmpty postId <- data.get("postId") blogUrl <- data.get("blogUrl") userId <- data.get("userId") } yield Record(blogUrl, postId, userId) } // retrieve first 20 records for a specific user parseRecords(“records.txt”).filter { case Record(_, _, “ryan”) }.take(20) Desugars to filter, map, and flatMap calls. Your own classes can work with for expressions. Tuesday, June 11, 13
32. Pattern matching & extractors // extractor object Reachable { private val SeenByThreshold = 5 def unapply(node: Node): Option[Node] = { if (node.seenBy >= SeenByThreshold) Some(node) else None } } // extract host for the first reachable node val reachable = nodes.collectFirst { case Reachable(node) => node.address } Reusable matching logic Tuesday, June 11, 13
33. Stackable traits (Scalatra) trait Handler { // Handles a request and writes to the response. def handle(request: HttpServletRequest, res: HttpServletResponse) } Tuesday, June 11, 13
34. // Scalatra handler for gzipped responses. trait GZipSupport extends Handler { abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) { if (isGzip) { // ... perform gzip handling, then delegate ... super.handle(req, response) } else super.handle(req, res) } } } // Redirects unsecured requests to the corresponding secure URL. trait SslRequirement extends Handler { abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) { if (!req.isSecure) { // build new uri and redirect res.redirect(uri) } else super.handle(req, res) } } Tuesday, June 11, 13
35. Composing stackable traits // pick which traits we want to use/stack for our service class MyService extends ScalatraServlet with SslRequirement with GZipSupport { // custom service logic goes here (unaware of surrounding functionality!) } Tuesday, June 11, 13
36. Refactoring • Developers often spend their time refactoring existing components • Ability to easily refactor is extremely important for large code bases • Without constant refactoring, code bases become cluttered with increased technical debt over time as new features are added Tuesday, June 11, 13
37. Refactoring in Ruby • Refactoring only effective when integration tests are plentiful • Difficult to ensure correctness in a large code base • Tests may pass, but a lurking NoMethodError can await you in production Tuesday, June 11, 13
38. Refactoring in Scala • Type system & compiler always have your back • Immediate feedback when performing invasive refactoring • IDE not necessary, but extremely helpful for revealing types in old or new code Tuesday, June 11, 13
39. Types Revealed Tuesday, June 11, 13
40. Conclusion • In my opinion, Ruby (and other dynamic languages) are great for small-scale development • Building complex systems with a highly dynamic language like Ruby is challenging • Scala empowers you to be efficient both at small-scale and large-scale development Tuesday, June 11, 13
41. We’re hiring! Tuesday, June 11, 13
42. Thanks! • http://github.com/ryanlecompte • Twitter: @ryanlecompte • ryan@quantifind.com Tuesday, June 11, 13