Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Confessions of a Ruby Developer Whose Heart was Stolen by Scala

Confessions of a Ruby Developer Whose Heart was Stolen by Scala

Talk @ Scala Days 2013

Ryan LeCompte

June 12, 2013
Tweet

More Decks by Ryan LeCompte

Other Decks in Programming

Transcript

  1. Confessions of a Ruby
    Developer Whose Heart was
    Stolen by Scala
    Ryan LeCompte
    @ryanlecompte
    Scala Days 2013
    Tuesday, June 11, 13

    View Slide

  2. Background
    • Scala developer at Quantifind
    • Author of redis_failover
    • http://github.com/ryanlecompte
    • Twitter: @ryanlecompte
    [email protected]find.com
    Tuesday, June 11, 13

    View Slide

  3. “Ruby and JavaScript are dynamically typed, and types
    and objects can change shape at runtime. This is a
    confluence of all the hardest-to-optimize language
    characteristics. In both cases, the best we can do is to
    attempt to predict common type and object shapes and
    insert guards for when we're wrong, but it's not possible to
    achieve the performance of a system with fully-predictable
    type and object shapes. Prove me wrong.”
    - Charles Nutter (JRuby Lead Developer)
    Tuesday, June 11, 13

    View Slide

  4. #Scala IRC channel
    “I think there is a threshold beyond which
    it is impossible to seriously change any
    program written in a dynamic language"
    Tuesday, June 11, 13

    View Slide

  5. #Scala IRC channel
    “You end up patching around the edges to avoid
    unintended consequences, and eventually
    it's a big ball of sticky tape surrounding the
    original program"
    Tuesday, June 11, 13

    View Slide

  6. #Scala IRC channel
    “Tests, grep, prayer."
    Tuesday, June 11, 13

    View Slide

  7. Ruby
    • Dynamic, object-oriented
    • Expressive, concise, flexible
    • Multiple implementations (MRI, JRuby, Rubinius)
    • Objects, classes, modules (mixins), blocks
    (closures)
    • Some functional constructs in collections library
    (each_cons, map, flat_map, filter)
    Tuesday, June 11, 13

    View Slide

  8. Scala
    • Object-oriented / functional hybrid
    • Immutability encouraged
    • Concurrency first-class citizen
    • Statically typed with powerful type
    inference
    • Objects, classes, traits (mixins), functions
    Tuesday, June 11, 13

    View Slide

  9. Ruby’s Philosophy
    • Programmer should always be “happy”
    when programming in Ruby
    • Duck typing (i.e., who cares what your
    actual “type” is)
    • Malleable language (class definitions
    modifiable at runtime, methods can be
    injected virtually anywhere)
    Tuesday, June 11, 13

    View Slide

  10. Ruby Modules vs. Scala
    Traits
    • Ruby’s modules are akin to Scala’s traits
    • Modules can be mixed into classes and
    other modules
    • Modules can be mixed into instances after
    they are created
    • Encapsulate cross-cutting concerns
    Tuesday, June 11, 13

    View Slide

  11. Ruby Module Example
    module RedisFailover
    # Base class for strategies that determine which
    # node is used during failover.
    module FailoverStrategy
    include Util
    # Returns a candidate node as determined by this strategy.
    #
    # @param [Hash] the node snapshots
    # @return [Node] the candidate node or nil if one couldn't be found
    def find_candidate(snapshots)
    raise NotImplementedError
    end
    end
    end No way to define
    abstract methods
    Types encoded in
    comments
    (tomdoc)
    Tuesday, June 11, 13

    View Slide

  12. Module Mixin Abuse
    module ActiveRecord
    class Base
    extend ActiveModel::Naming
    extend ActiveSupport::Benchmarkable
    extend ActiveSupport::DescendantsTracker
    extend ConnectionHandling
    extend QueryCache::ClassMethods
    extend Querying
    extend Translation
    extend DynamicMatchers
    extend Explain
    include Persistence
    include ReadonlyAttributes
    include ModelSchema
    include Inheritance
    include Scoping
    include Sanitization
    include AttributeAssignment
    # 19 more modules included ...
    end
    end
    Separate concerns
    are jammed together
    in a single
    namespace
    Tuesday, June 11, 13

    View Slide

  13. Where is the method
    defined?
    module M1
    def foo; puts "M1 foo"; end
    end
    module M2
    def foo; puts "M2 foo"; end
    end
    module M3
    def foo; puts "M3 foo"; end
    end
    class A
    def foo; puts "A foo"; end
    end
    class B < A
    include M1
    include M2
    def foo; puts "B foo"; end
    end
    b = B.new
    b.extend(M3)
    b.foo
    Which foo implementation
    gets used? No IDE to help you.
    Must rely on runtime
    introspection (i.e.,
    method_locator gem)
    Tuesday, June 11, 13

    View Slide

  14. Scala Traits
    • Never guess where a method is defined
    • Avoid mixin abuse with better abstractions
    (e.g. type classes, stackable traits, monads)
    • Encode dependencies and constraints using
    the type system (e.g., cake pattern)
    Tuesday, June 11, 13

    View Slide

  15. Monkey patching in
    Ruby
    • Very common practice in the Ruby/Rails
    community for providing new behavior for
    existing classes
    • Often takes the form of modules that are
    injected / mixed into existing classes from
    other libraries
    • Often invasive and munges together
    disparate concerns
    Tuesday, June 11, 13

    View Slide

  16. Example: Storage units
    250.bytes
    1.megabyte
    1.petabyte
    Integers don’t have storage unit
    methods by default. Goal is to
    somehow extend the numeric
    type to support them.
    Tuesday, June 11, 13

    View Slide

  17. Ruby on Rails Approach
    class Numeric
    KILOBYTE = 1024
    MEGABYTE = KILOBYTE * 1024
    GIGABYTE = MEGABYTE * 1024
    def kilobytes
    self * KILOBYTE
    end
    alias :kilobyte :kilobytes
    def megabytes
    self * MEGABYTE
    end
    alias :megabyte :megabytes
    def gigabytes
    self * GIGABYTE
    end
    alias :gigabyte :gigabytes
    end
    Ruby’s Numeric class is
    re-opened and directly
    modified. What if other
    libraries do the same
    thing at runtime?
    Collisions.
    Tuesday, June 11, 13

    View Slide

  18. Monkey patching in
    Scala
    • Implicit conversions are a safe compile-time
    enrichment for a particular type
    • Only one implicit conversion can be used
    for a given call site
    • IDE can inform you which implicit
    conversion is in use for a particular method
    Tuesday, June 11, 13

    View Slide

  19. Scala Approach
    package com.twitter.conversions
    object storage {
    implicit class RichWholeNumber(wrapped: Long) {
    def byte = bytes
    def bytes = new StorageUnit(wrapped)
    def kilobyte = kilobytes
    def kilobytes = new StorageUnit(wrapped * 1024)
    def megabyte = megabytes
    def megabytes = new StorageUnit(wrapped * 1024 * 1024)
    def gigabyte = gigabytes
    def gigabytes = new StorageUnit(wrapped * 1024 * 1024 * 1024)
    }
    // user code: bring in the implicit conversion at compile-time, not run-time
    import com.twitter.conversions.storage._
    println(s"1 gigabyte is ${1.gigabyte.inBytes} bytes")
    Long numeric type
    isn’t directly modified.
    Namespace stays clean!
    Tuesday, June 11, 13

    View Slide

  20. will_paginate library
    Pagination behavior mixed
    directly into ActiveRecord
    base class
    # mix everything into Active Record
    ::ActiveRecord::Base.extend PerPage
    ::ActiveRecord::Base.extend Pagination
    ::ActiveRecord::Base.extend BaseMethods
    # a new Post model now has pagination support
    class Post < ActiveRecord::Base
    end
    Post.page(params[:page]).order('created_at DESC')
    Tuesday, June 11, 13

    View Slide

  21. Type classes
    • Compiler discovers and injects the new
    behavior for you
    • Safer alternative to Ruby monkey patching
    when providing new functionality for
    existing types
    • Namespace isn’t further bloated with new
    methods / behavior since they live in type
    class instances
    Tuesday, June 11, 13

    View Slide

  22. Pagination with Type Classes
    trait Pager[A] {
    def page(model: A, page: Int): Seq[A]
    }
    object Pager {
    implicit object ModelPager extends Pager[Model] {
    def page(model: Model, page: Int): Seq[Model] = {
    // default DB pagination
    }
    }
    }
    object Example {
    def process[A](model: A, page: Int)(implicit pager: Pager[A]) {
    val records = pager.page(model, page)
    // do something with records
    }
    }
    Pager typeclass
    Default
    implementation
    Compiler supplies
    our Pager instance
    Tuesday, June 11, 13

    View Slide

  23. Extending core
    language libraries
    • Ruby’s approach usually involves injecting a
    custom module into core collection
    modules / classes (e.g., Enumerable) or
    direct modification
    • Scala’s approach: implicit conversions (safer,
    less invasive than direct modification)
    Tuesday, June 11, 13

    View Slide

  24. Ruby: Adding Hash#except and
    String#underscore
    {'a' => 1, 'b' => 2, 'c' => 3}.except('a', 'c')
    {'b' => 2}
    'HelloThereEveryone'.underscore
    "hello_there_everyone"
    Tuesday, June 11, 13

    View Slide

  25. Extending Ruby collections
    # example from will_paginate gem
    unless Hash.method_defined? :except
    Hash.class_eval do
    def except(*keys)
    has_convert = respond_to?(:convert_key)
    rejected = Set.new(has_convert ? keys.map { |k| convert_key(k) } : keys)
    reject { |key,| rejected.include?(key) }
    end
    end
    end
    unless String.method_defined? :underscore
    String.class_eval do
    def underscore
    self.to_s.gsub(/::/, '/').
    gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
    gsub(/([a-z\d])([A-Z])/,'\1_\2').
    tr("-", "_").
    downcase
    end
    end
    end
    Brittle approach: Use of
    class_eval and
    respond_to? to directly
    modify classes at
    runtime
    Tuesday, June 11, 13

    View Slide

  26. Scala: cycle & sample
    Vector(1,2,3).cycle.take(10).mkString(",")
    => 1,2,3,1,2,3,1,2,3,1
    val nums = 0 to 1000
    nums.sample(5)
    => IndexedSeq[Int] = WrappedArray(675, 510, 795, 169, 886)
    Tuesday, June 11, 13

    View Slide

  27. cycle
    implicit class RichSeq[A, C[A] <: Seq[A]](underlying: C[A]) {
    def cycle: Iterator[A] = {
    lazy val circular: Stream[A] = underlying.toStream #::: circular
    circular.iterator
    }
    }
    Tuesday, June 11, 13

    View Slide

  28. sample
    implicit class RichSeq[A: ClassManifest, C[A] <: Seq[A]](underlying: C[A]) {
    // see https://en.wikipedia.org/wiki/Reservoir_sampling
    def sample(k: Int): IndexedSeq[A] = {
    val rnd = new Random
    val elems = new Array[A](math.min(k, underlying.size))
    // avoiding zipWithIndex below for performance
    var idx = 0
    underlying.foreach { elt =>
    if (idx < k) elems(idx) = elt
    else {
    val nextIdx = rnd.nextInt(idx + 1)
    if (nextIdx < k) elems(nextIdx) = elt
    }
    idx += 1
    }
    elems
    }
    }
    Tuesday, June 11, 13

    View Slide

  29. Better Abstractions
    • Tail-recursive methods
    • For expressions
    • Pattern matching & custom extractors
    • Stackable traits
    Tuesday, June 11, 13

    View Slide

  30. @tailrec methods
    groupRuns(Vector(1, 1, 3, 3, 4, 2, 2, 5, 6))
    List(Vector(1, 1), Vector(3, 3), Vector(4), Vector(2, 2), Vector(5),Vector(6))
    @tailrec
    def groupRuns[A](c: Seq[A], acc: Seq[Seq[A]] = Seq.empty): Seq[Seq[A]] = {
    c match {
    case Seq() => acc
    case xs =>
    val (same, rest) = xs.span { _ == xs.head }
    groupRuns(rest, acc :+ same)
    }
    }
    Tuesday, June 11, 13

    View Slide

  31. For expressions
    import scala.io.Source
    import scala.util.control.Exception.allCatch
    case class Record(blogUrl: String, postId: String, userId: String)
    // build lazy stream of parsed & valid records
    def parseRecords(path: String): Stream[Record] = {
    for {
    line <- Source.fromFile(path).getLines.toStream
    data <- allCatch.opt { parse[Map[String, String]](line) }
    text <- data.get("text") if text.trim.nonEmpty
    postId <- data.get("postId")
    blogUrl <- data.get("blogUrl")
    userId <- data.get("userId")
    } yield Record(blogUrl, postId, userId)
    }
    // retrieve first 20 records for a specific user
    parseRecords(“records.txt”).filter { case Record(_, _, “ryan”) }.take(20)
    Desugars to filter, map, and
    flatMap calls. Your own classes
    can work with for expressions.
    Tuesday, June 11, 13

    View Slide

  32. Pattern matching &
    extractors
    // extractor
    object Reachable {
    private val SeenByThreshold = 5
    def unapply(node: Node): Option[Node] = {
    if (node.seenBy >= SeenByThreshold) Some(node)
    else None
    }
    }
    // extract host for the first reachable node
    val reachable = nodes.collectFirst { case Reachable(node) => node.address }
    Reusable
    matching logic
    Tuesday, June 11, 13

    View Slide

  33. Stackable traits
    (Scalatra)
    trait Handler {
    // Handles a request and writes to the response.
    def handle(request: HttpServletRequest, res: HttpServletResponse)
    }
    Tuesday, June 11, 13

    View Slide

  34. // Scalatra handler for gzipped responses.
    trait GZipSupport extends Handler {
    abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) {
    if (isGzip) {
    // ... perform gzip handling, then delegate ...
    super.handle(req, response)
    } else super.handle(req, res)
    }
    }
    }
    // Redirects unsecured requests to the corresponding secure URL.
    trait SslRequirement extends Handler {
    abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) {
    if (!req.isSecure) {
    // build new uri and redirect
    res.redirect(uri)
    } else super.handle(req, res)
    }
    }
    Tuesday, June 11, 13

    View Slide

  35. Composing stackable
    traits
    // pick which traits we want to use/stack for our service
    class MyService extends ScalatraServlet with SslRequirement with GZipSupport {
    // custom service logic goes here (unaware of surrounding functionality!)
    }
    Tuesday, June 11, 13

    View Slide

  36. Refactoring
    • Developers often spend their time
    refactoring existing components
    • Ability to easily refactor is extremely
    important for large code bases
    • Without constant refactoring, code bases
    become cluttered with increased technical
    debt over time as new features are added
    Tuesday, June 11, 13

    View Slide

  37. Refactoring in Ruby
    • Refactoring only effective when integration
    tests are plentiful
    • Difficult to ensure correctness in a large
    code base
    • Tests may pass, but a lurking
    NoMethodError can await you in
    production
    Tuesday, June 11, 13

    View Slide

  38. Refactoring in Scala
    • Type system & compiler always have your
    back
    • Immediate feedback when performing
    invasive refactoring
    • IDE not necessary, but extremely helpful
    for revealing types in old or new code
    Tuesday, June 11, 13

    View Slide

  39. Types Revealed
    Tuesday, June 11, 13

    View Slide

  40. Conclusion
    • In my opinion, Ruby (and other dynamic
    languages) are great for small-scale
    development
    • Building complex systems with a highly
    dynamic language like Ruby is challenging
    • Scala empowers you to be efficient both at
    small-scale and large-scale development
    Tuesday, June 11, 13

    View Slide

  41. We’re hiring!
    Tuesday, June 11, 13

    View Slide

  42. Thanks!
    • http://github.com/ryanlecompte
    • Twitter: @ryanlecompte
    [email protected]find.com
    Tuesday, June 11, 13

    View Slide