Confessions of a Ruby Developer Whose Heart was Stolen by Scala

Confessions of a Ruby
Developer Whose Heart was
Stolen by Scala
Ryan LeCompte
@ryanlecompte
Scala Days 2013
Tuesday, June 11, 13

View Slide

Background
• Scala developer at Quantiﬁnd
• Author of redis_failover
• http://github.com/ryanlecompte
• Twitter: @ryanlecompte
• [email protected]ﬁnd.com

View Slide

“Ruby and JavaScript are dynamically typed, and types
and objects can change shape at runtime. This is a
confluence of all the hardest-to-optimize language
characteristics. In both cases, the best we can do is to
attempt to predict common type and object shapes and
insert guards for when we're wrong, but it's not possible to
achieve the performance of a system with fully-predictable
type and object shapes. Prove me wrong.”
- Charles Nutter (JRuby Lead Developer)

View Slide

#Scala IRC channel
“I think there is a threshold beyond which
it is impossible to seriously change any
program written in a dynamic language"

View Slide

#Scala IRC channel
“You end up patching around the edges to avoid
unintended consequences, and eventually
it's a big ball of sticky tape surrounding the
original program"

View Slide

#Scala IRC channel
“Tests, grep, prayer."

View Slide

Ruby
• Dynamic, object-oriented
• Expressive, concise, flexible
• Multiple implementations (MRI, JRuby, Rubinius)
• Objects, classes, modules (mixins), blocks
(closures)
• Some functional constructs in collections library
(each_cons, map, flat_map, filter)

View Slide

Scala
• Object-oriented / functional hybrid
• Immutability encouraged
• Concurrency ﬁrst-class citizen
• Statically typed with powerful type
inference
• Objects, classes, traits (mixins), functions

View Slide

Ruby’s Philosophy
• Programmer should always be “happy”
when programming in Ruby
• Duck typing (i.e., who cares what your
actual “type” is)
• Malleable language (class deﬁnitions
modiﬁable at runtime, methods can be
injected virtually anywhere)

View Slide

Ruby Modules vs. Scala
Traits
• Ruby’s modules are akin to Scala’s traits
• Modules can be mixed into classes and
other modules
• Modules can be mixed into instances after
they are created
• Encapsulate cross-cutting concerns

View Slide

Ruby Module Example
module RedisFailover
# Base class for strategies that determine which
# node is used during failover.
module FailoverStrategy
include Util
# Returns a candidate node as determined by this strategy.
#
# @param [Hash] the node snapshots
# @return [Node] the candidate node or nil if one couldn't be found
def find_candidate(snapshots)
raise NotImplementedError
end
end
end No way to deﬁne
abstract methods
Types encoded in
comments
(tomdoc)

View Slide

Module Mixin Abuse
module ActiveRecord
class Base
extend ActiveModel::Naming
extend ActiveSupport::Benchmarkable
extend ActiveSupport::DescendantsTracker
extend ConnectionHandling
extend QueryCache::ClassMethods
extend Querying
extend Translation
extend DynamicMatchers
extend Explain
include Persistence
include ReadonlyAttributes
include ModelSchema
include Inheritance
include Scoping
include Sanitization
include AttributeAssignment
# 19 more modules included ...
end
end
Separate concerns
are jammed together
in a single
namespace

View Slide

Where is the method
deﬁned?
module M1
def foo; puts "M1 foo"; end
end
module M2
end
module M3
end
class A
def foo; puts "A foo"; end
end
class B < A
include M1
include M2
def foo; puts "B foo"; end
end
b = B.new
b.extend(M3)
b.foo
Which foo implementation
gets used? No IDE to help you.
Must rely on runtime
introspection (i.e.,
method_locator gem)

View Slide

Scala Traits
• Never guess where a method is deﬁned
• Avoid mixin abuse with better abstractions
(e.g. type classes, stackable traits, monads)
• Encode dependencies and constraints using
the type system (e.g., cake pattern)

View Slide

Monkey patching in
Ruby
• Very common practice in the Ruby/Rails
community for providing new behavior for
existing classes
• Often takes the form of modules that are
injected / mixed into existing classes from
other libraries
• Often invasive and munges together
disparate concerns

View Slide

Example: Storage units
250.bytes
1.megabyte
1.petabyte
Integers don’t have storage unit
methods by default. Goal is to
somehow extend the numeric
type to support them.

View Slide

Ruby on Rails Approach
class Numeric
KILOBYTE = 1024
MEGABYTE = KILOBYTE * 1024
GIGABYTE = MEGABYTE * 1024
def kilobytes
self * KILOBYTE
end
alias :kilobyte :kilobytes
def megabytes
self * MEGABYTE
end
alias :megabyte :megabytes
def gigabytes
self * GIGABYTE
end
alias :gigabyte :gigabytes
end
Ruby’s Numeric class is
re-opened and directly
modiﬁed. What if other
libraries do the same
thing at runtime?
Collisions.

View Slide

Monkey patching in
Scala
• Implicit conversions are a safe compile-time
enrichment for a particular type
• Only one implicit conversion can be used
for a given call site
• IDE can inform you which implicit
conversion is in use for a particular method

View Slide

Scala Approach
package com.twitter.conversions
object storage {
implicit class RichWholeNumber(wrapped: Long) {
def byte = bytes
def bytes = new StorageUnit(wrapped)
def kilobyte = kilobytes
def kilobytes = new StorageUnit(wrapped * 1024)
def megabyte = megabytes
def megabytes = new StorageUnit(wrapped * 1024 * 1024)
def gigabyte = gigabytes
def gigabytes = new StorageUnit(wrapped * 1024 * 1024 * 1024)
}
// user code: bring in the implicit conversion at compile-time, not run-time
import com.twitter.conversions.storage._
println(s"1 gigabyte is ${1.gigabyte.inBytes} bytes")
Long numeric type
isn’t directly modiﬁed.
Namespace stays clean!

View Slide

will_paginate library
Pagination behavior mixed
directly into ActiveRecord
base class
# mix everything into Active Record
::ActiveRecord::Base.extend PerPage
::ActiveRecord::Base.extend Pagination
::ActiveRecord::Base.extend BaseMethods
# a new Post model now has pagination support
class Post < ActiveRecord::Base
end
Post.page(params[:page]).order('created_at DESC')

View Slide

Type classes
• Compiler discovers and injects the new
behavior for you
• Safer alternative to Ruby monkey patching
when providing new functionality for
existing types
• Namespace isn’t further bloated with new
methods / behavior since they live in type
class instances

View Slide

Pagination with Type Classes
trait Pager[A] {
def page(model: A, page: Int): Seq[A]
}
object Pager {
implicit object ModelPager extends Pager[Model] {
def page(model: Model, page: Int): Seq[Model] = {
// default DB pagination
}
}
}
object Example {
def process[A](model: A, page: Int)(implicit pager: Pager[A]) {
val records = pager.page(model, page)
// do something with records
}
}
Pager typeclass
Default
implementation
Compiler supplies
our Pager instance

View Slide

Extending core
language libraries
• Ruby’s approach usually involves injecting a
custom module into core collection
modules / classes (e.g., Enumerable) or
direct modiﬁcation
• Scala’s approach: implicit conversions (safer,
less invasive than direct modiﬁcation)

View Slide

Ruby: Adding Hash#except and
String#underscore
{'a' => 1, 'b' => 2, 'c' => 3}.except('a', 'c')
{'b' => 2}
'HelloThereEveryone'.underscore
"hello_there_everyone"

View Slide

Extending Ruby collections
# example from will_paginate gem
unless Hash.method_defined? :except
Hash.class_eval do
def except(*keys)
has_convert = respond_to?(:convert_key)
rejected = Set.new(has_convert ? keys.map { |k| convert_key(k) } : keys)
reject { |key,| rejected.include?(key) }
end
end
end
unless String.method_defined? :underscore
String.class_eval do
def underscore
self.to_s.gsub(/::/, '/').
gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
gsub(/([a-z\d])([A-Z])/,'\1_\2').
tr("-", "_").
downcase
end
end
end
Brittle approach: Use of
class_eval and
respond_to? to directly
modify classes at
runtime

View Slide

Scala: cycle & sample
Vector(1,2,3).cycle.take(10).mkString(",")
=> 1,2,3,1,2,3,1,2,3,1
val nums = 0 to 1000
nums.sample(5)
=> IndexedSeq[Int] = WrappedArray(675, 510, 795, 169, 886)

View Slide

cycle
implicit class RichSeq[A, C[A] <: Seq[A]](underlying: C[A]) {
def cycle: Iterator[A] = {
lazy val circular: Stream[A] = underlying.toStream #::: circular
circular.iterator
}
}

View Slide

sample
implicit class RichSeq[A: ClassManifest, C[A] <: Seq[A]](underlying: C[A]) {
// see https://en.wikipedia.org/wiki/Reservoir_sampling
def sample(k: Int): IndexedSeq[A] = {
val rnd = new Random
val elems = new Array[A](math.min(k, underlying.size))
// avoiding zipWithIndex below for performance
var idx = 0
underlying.foreach { elt =>
if (idx < k) elems(idx) = elt
else {
val nextIdx = rnd.nextInt(idx + 1)
if (nextIdx < k) elems(nextIdx) = elt
}
idx += 1
}
elems
}
}

View Slide

Better Abstractions
• Tail-recursive methods
• For expressions
• Pattern matching & custom extractors
• Stackable traits

View Slide

@tailrec methods
groupRuns(Vector(1, 1, 3, 3, 4, 2, 2, 5, 6))
List(Vector(1, 1), Vector(3, 3), Vector(4), Vector(2, 2), Vector(5),Vector(6))
@tailrec
def groupRuns[A](c: Seq[A], acc: Seq[Seq[A]] = Seq.empty): Seq[Seq[A]] = {
c match {
case Seq() => acc
case xs =>
val (same, rest) = xs.span { _ == xs.head }
groupRuns(rest, acc :+ same)
}
}

View Slide

For expressions
import scala.io.Source
import scala.util.control.Exception.allCatch
case class Record(blogUrl: String, postId: String, userId: String)
// build lazy stream of parsed & valid records
def parseRecords(path: String): Stream[Record] = {
for {
line <- Source.fromFile(path).getLines.toStream
data <- allCatch.opt { parse[Map[String, String]](line) }
text <- data.get("text") if text.trim.nonEmpty
postId <- data.get("postId")
blogUrl <- data.get("blogUrl")
userId <- data.get("userId")
} yield Record(blogUrl, postId, userId)
}
// retrieve first 20 records for a specific user
parseRecords(“records.txt”).filter { case Record(_, _, “ryan”) }.take(20)
Desugars to ﬁlter, map, and
ﬂatMap calls. Your own classes
can work with for expressions.

View Slide

Pattern matching &
extractors
// extractor
object Reachable {
private val SeenByThreshold = 5
def unapply(node: Node): Option[Node] = {
if (node.seenBy >= SeenByThreshold) Some(node)
else None
}
}
// extract host for the first reachable node
val reachable = nodes.collectFirst { case Reachable(node) => node.address }
Reusable
matching logic

View Slide

Stackable traits
(Scalatra)
trait Handler {
// Handles a request and writes to the response.
def handle(request: HttpServletRequest, res: HttpServletResponse)
}

View Slide

// Scalatra handler for gzipped responses.
trait GZipSupport extends Handler {
abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) {
if (isGzip) {
// ... perform gzip handling, then delegate ...
super.handle(req, response)
} else super.handle(req, res)
}
}
}
// Redirects unsecured requests to the corresponding secure URL.
trait SslRequirement extends Handler {
abstract override def handle(req: HttpServletRequest, res: HttpServletResponse) {
if (!req.isSecure) {
// build new uri and redirect
res.redirect(uri)
} else super.handle(req, res)
}
}

View Slide

Composing stackable
traits
// pick which traits we want to use/stack for our service
class MyService extends ScalatraServlet with SslRequirement with GZipSupport {
// custom service logic goes here (unaware of surrounding functionality!)
}

View Slide

Refactoring
• Developers often spend their time
refactoring existing components
• Ability to easily refactor is extremely
important for large code bases
• Without constant refactoring, code bases
become cluttered with increased technical
debt over time as new features are added

View Slide

Refactoring in Ruby
• Refactoring only effective when integration
tests are plentiful
• Difﬁcult to ensure correctness in a large
code base
• Tests may pass, but a lurking
NoMethodError can await you in
production

View Slide

Refactoring in Scala
• Type system & compiler always have your
back
• Immediate feedback when performing
invasive refactoring
• IDE not necessary, but extremely helpful
for revealing types in old or new code

View Slide

Types Revealed

View Slide

Conclusion
• In my opinion, Ruby (and other dynamic
languages) are great for small-scale
development
• Building complex systems with a highly
dynamic language like Ruby is challenging
• Scala empowers you to be efﬁcient both at
small-scale and large-scale development

View Slide

We’re hiring!

View Slide

Thanks!
• http://github.com/ryanlecompte
• Twitter: @ryanlecompte
• [email protected]ﬁnd.com

View Slide

Confessions of a Ruby Developer Whose Heart was Stolen by Scala

Confessions of a Ruby Developer Whose Heart was Stolen by Scala

Ryan LeCompte

More Decks by Ryan LeCompte

Other Decks in Programming

Featured

Transcript