Ruby: A Family History (Railsconf 2018)

A talk I gave at Railsconf 2018. Experimenting with including some lightly edited speaker notes to accompany the slides.

Rails is made possible by Ruby’s unique combination of deep dynamism and pragmatic elegance. In turn, Ruby inherited many of its core ideas from older languages including Lisp, Smalltalk, and Perl.

In this talk, we’ll take a tour of these languages and their myriad influences on Ruby, to better understand the foundations of our tools, improve our ability to use them to their full potential, and imagine how they might be further improved.

Transcript

1. I have a confession to make: I
   love Ruby.
   Something just clicks, it feels
   expressive, powerful, flexible
   and I suspect many of you feel
   the same way
   Ruby:
   A Family History
   Geoffrey Litt
   @geoffreylitt
   Panorama Education
   

   View Slide

2. * But recently I've been thinking about: why? This
   isn't just a matter of intellectual curiosity.
   * When we see successful systems like Ruby/Rails,
   and we want to make successful systems ourselves,
   it can be useful to deeply understand WHY other
   systems are successful.
   To some extent, this is a personal question, but there
   are still ways to systematically analyze this question.
   One technique is to compare Ruby with other
   programming languages.
   Why do I Ruby?
   

   View Slide

3. One approach is to compare ruby with
   modern programming languages:
   Python, or Rust, or Java.
   To me, this is like comparing a person
   to their friends and peers, which is
   OK...
   But if you really want to understand
   someone...
   By learning
   other languages,
   we can broaden
   our horizons and
   also more deeply
   understand
   Ruby.
   — matz
   

   View Slide

4. You gotta meet their parents.
   Today we're going to meet
   Ruby's parents!
   !
   

   View Slide

5. Today we're going to explore 3 prominent languages that
   contributed heavily to Ruby: Lisp, Smalltalk, Perl.
   We'll approach these languages through 3 lenses:
   1) Understand the core ideas of these languages, to deepen our
   understanding of key principles in Ruby.
   2) Rediscover lost and forgotten ideas from these languages
   3) Most importantly: Not just explore these languages in
   isolation, but see how Matz combined them into a single
   coherent language that we love.
   We'll see a lot of subtle balance in his design decisions, that not
   only can help us answer "why do we love Ruby?", but can also
   teach us valuable lessons designing any system, whether a
   language, framework, or product.
   

   View Slide

6. * Let's start with LISP.
   LISP
   

   View Slide

7. According to Matz, Ruby
   actually originated as a Lisp!
   Ruby was a Lisp originally,
   in theory.
   Let's call it MatzLisp from
   now on. ;-)
   — matz
   

   View Slide

8. * Created in 1958-1960 by computer scientist John
   McCarthy.
   * His goal was NOT to create a programming language.
   He wanted to create a set of fundamental axioms for
   computing, a model of describing algorithms
   * Out of that work came Lisp: List Processing
   language.
   * Had a data structure of lists, and a small number of
   simple operators on lists: chopping them up, joining
   back together. Plus a keyword lambda for defining
   functions.
   

   View Slide

9. * Out of that small set of axioms, we
   can define arbitrary computation!!!
   * One example: given those
   primitives, we can define eval, a
   Lisp interpreter. Only about 30 lines!
   * to really appreciate this, let's place
   it in context
   An interpreter
   (defun eval (e a)
   (cond
   ((atom e) (assoc. e a))
   ((atom (car e))
   (cond
   ((eq (car e) 'quote) (cadr e))
   ((eq (car e) 'atom) (atom (eval (cadr e) a)))
   ((eq (car e) 'eq) (eq (eval (cadr e) a)
   (eval (caddr e) a)))
   ((eq (car e) 'car) (car (eval (cadr e) a)))
   ((eq (car e) 'cdr) (cdr (eval (cadr e) a)))
   ((eq (car e) 'cons) (cons (eval (cadr e) a)
   (eval (caddr e) a)))
   ((eq (car e) 'cond) (evcon. (cdr e) a))
   ;...
   

   View Slide

10. We can think of programming
    languages on a spectrum from
    low-level to high-level.
    

    View Slide

11. * Previously we were adding
    layers of abstraction on top of
    machine code.
    

    View Slide

12. * Lisp wasn't just another layer. It came from a totally different direction!
    * McCarthy was creating abstract description of computation, and how
    computers work is decoupled from that project.
    * When people say "Lisp was discovered, not invented", this is what they
    mean.
    * We still see this lineage in Ruby and Rails: human-centered tools that
    prioritize the way we think.
    * Optimizing for programmer productivity, not machine productivity.
    * To put in terms of David's keynote yesterday: these tools do conceptual
    compression
    Because Lisp was the first high-level language, Ruby got many many things
    from it:
    conditionals, dynamic typing, garbage collection, symbol types
    But one area that's particularly instructive: how Ruby incorporated Lisp's
    handling of functions
    

    View Slide

13. Lisp introduced idea of
    functions as values, created
    and manipulated at runtime.
    ^ Here's how you define an
    anonymous function in Lisp,
    using the lambda keyword.
    Functions
    (lambda (x) (* x 2))
    

    View Slide

14. * Lisp also introduced higher-order functions
    like map that take a function as an argument.
    * As Ruby developers, you probably
    understand how powerful things like map
    are.
    * In Ruby, these ideas of function values and
    higher-order functions are directly inherited
    from Lisp.
    Functions
    (map
    (lambda (x) (* x 2))
    (1 2 3))
    

    View Slide

15. * We can translate this Lisp code
    directly into Ruby using Ruby's lambda
    keyword
    * But that's not how we normally do
    things in ruby, right? We have blocks.
    * We use blocks every day, but have
    you ever asked, why do they exist?
    And what's the point?
    Functions
    (map
    (lambda (x) (* x 2))
    (1 2 3))
    [1, 2, 3].map(
    &lambda { |x| x * 2 }
    )
    

    View Slide

16. We say "everything is an
    object" in ruby, but a block isn't
    a valid object
    Functions
    { |x| x * 2 } # => SyntaxError
    

    View Slide

17. You can only pass a block as an implicit argument.
    ^ This adds quite a lot of complexity. In Lisp, only
    need to know about lambdas. In Ruby, procs,
    lambdas, blocks...
    ^ What do we gain from this added complexity?
    What this gives us: more concise syntax for
    special cases where function takes just one
    anonymous function as an argument.
    Turns out this is useful in many different
    situations!
    Functions
    { |x| x * 2 } # => SyntaxError
    def takes_a_block
    yield 2
    end
    takes_a_block { |x| x * 2 } # => 4
    

    View Slide

18. Lisp had the idea of pipelines
    of functional transformations...
    ^ but Ruby arguably offers a
    cleaner way to write them than
    Lisp itself, with blocks.
    Functions
    (remove-if (lambda (n) (< n 4))
    (map (lambda (x) (* x 2))
    (1 2 3)))
    [1, 2, 3].
    map { |x| x * 2 }.
    reject{ |n| n < 4 }
    

    View Slide

19. Here I have some fake Rspec syntax in lisp.
    Internal DSLs are more reasonable to write in Ruby.
    We see this in Rails migrations, rake tasks, and
    more
    Now what if you want to pass in two anonymous
    functions?
    Can't do it with blocks, you have to resort to a
    totally different system.
    Blocks arguably add inconsistency and complexity
    to the language.
    Functions
    (describe "my machine" (lambda () (
    (it "produces widgets" (lambda () (
    ;...
    ))))))
    describe "my machine" do
    it "produces widgets" do
    #...
    end
    end
    

    View Slide

20. * Often people say "consistency is really important in design"
    * But, in this case: inconsistency has a payoff: helps make our
    code easier to read/write in a very common "special case"
    * You see this in Rails too! Idea of convention over configuration:
    introduce implementation complexity to make the common case
    easy
    * Different users will want different tradeoffs here, but the point
    is that consistency can be sacrificed in service of other goals.
    By the way, fun to see concise lambdas catching on...
    In last 10 years: Stabby lambda syntax in Ruby, fat arrow syntax
    in ES6.
    Language designers realizing this concise lambda definition is a
    valuable feature.
    Thoughtful inconsistency
    

    View Slide

21. * LISP's syntax is called "s-expressions"
    Fun tidbit of computing history:
    Originally an accident, s-expressions were intended
    as the internal representation.
    McCarthy had a plan for a whole other syntax called
    m-expressions, with fewer parens.
    But a researcher in his lab wrote an interpreter for s-
    expressions, that got popular, and the rest was
    history.
    McCarthy has a great quote reflecting on the m-
    expressions project:
    Syntax
    (
    (let a 1)
    (let b 2)
    (+ a b))
    

    View Slide

22. The project...was neither
    finalized nor explicitly
    abandoned. It just
    receded into the indefinite
    future...
    — John McCarthy
    

    View Slide

23. * Lisp syntax mirrors the abstract
    syntax tree and has tons of beautiful
    properties:
    * very little parsing
    * macros for manipulating code as
    data
    * and Lisp hackers will say you can
    totally get used to the parentheses
    LISP Syntax
    (
    (let a 1)
    (let b 2)
    (+ a b))
    

    View Slide

24. * This is syntax of a different language, FORTRAN
    * Matz moved from s-expression syntax to Fortran
    style. Kinda looks like Ruby.
    * Requires much more complex parsing, infix
    operators, etc...so if Ruby is a Lisp, why use Fortran
    syntax?
    * I can't read Matz's mind. Maybe he preferred this.
    But either way, it was the right thing to do.
    * This syntax had already won and was familiar to
    everybody. It was a necessary choice to drive
    adoption.
    FORTRAN Syntax
    a = 1
    b = 2
    result = a + b
    

    View Slide

25. * If you want widespread usage, sometimes you
    have to go along with the popular trend to
    remain relevant.
    * Ruby went with the prevailing trend in syntax.
    * Rails has a similar approach to the modern
    Javascript ecosystem with webpacker, going
    along with trends to remain relevant
    * Fun question: Lisp lets you manipulate your
    code as data. What if we want to do that in ruby?
    Pick your battles.
    

    View Slide

26. Using the parser gem, we can
    take a string of code and parse
    it...
    Ruby parsing/unparsing
    code = "2 + 3 * 4"
    ast = Parser::CurrentRuby.parse(code)
    

    View Slide

27. and we get access to that
    code as a tree of data!
    Ruby parsing/unparsing
    code = "2 + 3 * 4"
    ast = Parser::CurrentRuby.parse(code)
    # => [s(:send,
    # s(:int, 2), :+,
    # s(:send,
    # s(:int, 3), :*,
    # s(:int, 4))), []]
    

    View Slide

28. and we can then unparse it back into text.
    ^ You might be thinking, this is a terrible idea, and you're
    mostly right :)
    ^ But some interesting real world use cases of this. Gem
    called "mutant" that does mutation testing, where it
    randomly changes your code and makes sure your tests fail
    when your code changes. Uses this gem under the hood.
    But of course we don't need to resort to this often.
    ^ Reason: most of the time in Lisp, code manipulation is used
    for metaprogramming: code that creates code. In Ruby we
    don't need to use this approach because we have a variety
    of powerful tools, which come from another language:
    Ruby parsing/unparsing
    code = "2 + 3 * 4"
    ast = Parser::CurrentRuby.parse(code)
    # => [s(:send,
    # s(:int, 2), :+,
    # s(:send,
    # s(:int, 3), :*,
    # s(:int, 4))), []]
    2.1.5 :005 > Unparser.unparse(ast)
    # => "2 + (3 * 4)"
    

    View Slide

29. * Smalltalk!
    * The first OO language
    * Huge influence on Ruby
    Smalltalk
    

    View Slide

30. * Early 70s, led by Alan Kay,
    lots of work from Adele
    Goldberg and Dan Ingalls at
    legendary Xerox PARC lab
    

    View Slide

31. Fundamental goal: expand computing to everyone, let
    anyone make their own programs. Architects making
    architecture software, artists making drawing
    software...
    Smalltalk also had a focus on kids
    This resonates with DHH's keynote from yesterday.
    Widening the group that has access.
    This is part of why it's named Smalltalk: focus on kids
    Also a joke: Kay was tired of systems with big names
    like Zeus/Thor doing nothing, so he made a system
    with a small, cute name but lots of power
    

    View Slide

32. Not just the first OO language, also an editing
    environment for that language.
    They invented the modern GUI and overlapping windows
    Model-View-Controller was also invented there for
    Smalltalk
    Basically everything we do today invented at this one
    lab!
    By the way, other labs down the hall were inventing
    things like Ethernet...
    Great book called Dealers of Lightning that recounts the
    history
    

    View Slide

33. We talk a lot about OO, and
    sometimes this question
    comes up, "what is OO?"
    What is "object-oriented
    programming"?
    

    View Slide

34. Maybe there's not a single answer to that
    question, but we CAN ask, what was the
    intention of the creator of OO when it was
    invented?
    This is a question that can yield some
    insight. To answer it, we look to Smalltalk,
    the first OO language
    Similarly to McCarthy, Alan Kay was looking
    for a consistent model of computation.
    What is "object-oriented
    programming"?
    What was the original
    intention of object-
    oriented programming?
    

    View Slide

35. * He was a fan of Lisp... but had problems
    with it
    * Amazingly, thought lisp wasn't consistent
    enough!
    * example: "lambda" isn't a function. Too
    much of lisp isn't implemented with functions.
    * He was looking for a more purely recursive
    approach to designing a model of
    computation.
    I could hardly believe how
    beautiful and wonderful
    the idea of LISP was...but
    there were deep flaws in
    its logical foundations.
    — Alan Kay
    

    View Slide

36. View Slide

37. * People had been splitting up
    a computer into functions and
    data separately.
    

    View Slide

38. * What if we split into little
    computers instead?
    

    View Slide

39. * Each has state and process
    encapsulated. Like millions of
    little computers
    In computer terms,
    Smalltalk is a recursion on
    the notion of computer
    itself.
    — Alan Kay
    

    View Slide

40. Let's see what that looks like in
    practice.
    Message passing
    3 + 4
    "=> 7
    

    View Slide

41. View Slide

42. View Slide

43. * almost like making a remote API call, a
    post request to 3.com
    * Revolutionary idea: give RECEIVER so
    much control, extremely DYNAMIC behavior
    * Very very different from how functions
    work in C
    Much of Ruby's OO side comes very
    directly from Smalltalk:
    

    View Slide

44. Message passing similarities
    Smalltalk:
    3 perform: '+' asSymbol with: 4
    Ruby:
    3.send(:+, 4)
    

    View Slide

45. Message passing similarities
    Smalltalk:
    3 isKindOf: Integer
    Ruby:
    3.is_a? Integer
    

    View Slide

46. classes are just objects
    Message passing similarities
    Smalltalk:
    Integer isKindOf: Class
    Ruby:
    Integer.is_a? Class
    

    View Slide

47. * have you ever thought about
    WHY ruby pushes us to do
    iteration over arrays this way?
    * having more things be
    message-based makes the
    language more CONSISTENT
    Control flow with message sending
    Smalltalk:
    array do: [ :element | Transcript show: element ]
    Ruby:
    array.each { |element| puts element }
    

    View Slide

48. In this example: changing what happens when an integer
    receives a method it doesn't understand.
    The Smalltalk and Ruby look nearly identical.
    Smalltalk has precedents for 2 key ideas in Ruby:
    * Opening classes, including built-ins
    * Handling missing methods
    * This all flows from the core mental model: extreme
    encapsulation.
    Rails uses these metaprogramming tools extensively
    ^ Opening up classes, dynamically defining/undefining methods,
    handling missing methods
    ^ Fair to debate how to use these powerful tools wisely, but at
    least we see their origins
    Metaprogramming
    Integer extend [
    doesNotUnderstand: msg [
    'method not defined' printNl
    ]
    ]
    class Integer
    def method_missing(msg)
    puts 'method not defined'
    end
    end
    

    View Slide

49. * Obviously we got a lot
    * But how did Ruby DIVERGE from
    Smalltalk?
    There are no conditionals in Smalltalk.
    You saw Kay was committed to the
    purity of the recursion.
    Conditional logic in Smalltalk: Just a
    message send
    Message passing purity
    (2 + 2 == 5)
    ifTrue: [ Transcript show: 'true'. ]
    ifFalse: [ Transcript show: 'false'. ].
    

    View Slide

50. You can implement similar things in Ruby,
    but instead we got conditionals as a
    language feature.
    not sure why Matz chose this, but one
    theory: if statements just seem more
    straightforward to use.
    And he was willing to add complexity/
    inconsistency to the language to
    accommodate that
    Thoughtful inconsistency
    

    View Slide

51. * Smalltalk had a different
    approach to code organization
    * This is a forgotten idea: a
    super highly integrated IDE
    * Smalltalk: no text files, always
    operating on a live image
    

    View Slide

52. * In this example, you can
    provide example input output
    and it'll find the corresponding
    method for you.
    

    View Slide

53. * Ruby didn't do image-based
    editing, too radical. Almost
    certainly a good idea for adoption.
    * Ruby/Rails is a text editor +
    terminal community, but we have
    rich editor heritage. Bring it back?
    Pick your battles.
    

    View Slide

54. * Perl: the "Swiss army
    chainsaw" of programming
    languages
    Perl
    

    View Slide

55. * You can tell this one is going to be a little
    different
    * Developed in late 80s by Larry Wall
    * linguist working as sysadmin for NASA
    * Larry Wall wasn't solving a deep
    theoretical problem.
    * He was trying to do sysadmin stuff.
    * Wanted to combine text processing (sed,
    awk) w/ Real Programming (C)
    

    View Slide

56. Perl provided many mantras
    that Ruby adopted
    A focus on pragmatism,
    productivity, just getting the
    job done somehow, getting out
    of your way.
    "A program is correct if it
    gets the job done before
    you get fired."
    

    View Slide

57. * One of Ruby's core principles!
    * in contrast w/ Python, in contrast
    with more academic communities.
    * Larry Wall thinks of
    programming as human language
    more than a mathematical
    system.
    "There's more than one
    way to do it"
    

    View Slide

58. * This brings us back to
    introducing thoughtful
    inconsistency, making special
    cases when it helps.
    "Easy things should be
    easy and hard things
    should be possible"
    

    View Slide

59. A good car has a solid chassis + transmission +
    airbags...
    but it also has to have a nice interior. The seats
    and interior details matter.
    Academic languages often focus on getting the
    chassis + transmission right, but often don't pay
    as much attention to this stuff
    Ruby is a polished language, feels nice to use.
    Like a car with a nice interior.
    And we got a lot of those things from Perl.
    

    View Slide

60. String interpolation is super super easy
    in Perl.
    ^ Ruby we don't have it quite this easy,
    but still have a good syntax for it
    ^ Javascript had nothing good until very
    recently, many languages have nothing
    ^ Wouldn't be surprised if we have Perl
    to thank
    Effortless string interpolation
    $x = 5;
    $msg = "The value is $x now.";
    

    View Slide

61. A lot of what makes Ruby feel
    delightful is this detailed polish
    work.
    ^ Rails has this too:
    2.days.ago is the same
    type of thing
    More little details
    —native regex syntax
    —%w array syntax
    —heredocs
    —1_000_000 number syntax
    

    View Slide

62. Also got some things that might be
    more controversial.
    global variables: not common in
    many langs now, but came from Perl.
    Bad idea most of the time, but
    sometimes pragmatic and useful for
    hacking stuff together
    Globals
    $global_variable = 1
    

    View Slide

63. not only that, we got weird
    built-in global variables
    Globals
    $$
    

    View Slide

64. Cryptic and probably bad for
    production code...
    ^ but maybe useful if you're
    just hacking together a script
    ^ Very pragmatic!
    Globals
    $$ # => 19936
    Process ID!
    

    View Slide

65. Some parts of Perl are too far
    in this direction so Ruby
    doesn't incorporate
    ^ In Perl, guess what this
    produces?
    Weak typing
    print "8" + "1";
    

    View Slide

66. Yuck. Contrast with Ruby,
    which has a more strongly
    typed core.
    Weak typing
    print "8" + "1"; # => 9
    

    View Slide

67. Weak typing
    @cities = qw( Berlin Tokyo London Boston );
    # Assign the array to a scalar
    $cities_count = @cities;
    

    View Slide

68. * Perl tilts heavily towards implicit
    type coercion
    * Reminiscent of some of most
    frustrating parts of Javascript
    * Ruby has a more consistent core.
    * We got the elegance and
    ergonomics of Perl, without the core
    Weak typing
    @cities = qw( Berlin Tokyo London Boston );
    # Assign the array to a scalar
    $cities_count = @cities;
    # => 4
    

    View Slide

69. Alright, brings us back to this
    question: why do we love
    Ruby?
    Why do I Ruby?
    

    View Slide

70. Ruby Achieves a delicate balance on many dimensions, by picking best parts of
    different solutions
    Consistency of Smalltalk/Lisp computation models. But willing to add exceptions to the
    rule for usability.
    ^ Academic flavor of Smalltalk/Lisp, layers on nice leather detailing from Perl
    ^ Takes some stances, like being very object-oriented. But, shies away from radical
    departures like Lisp's syntax or Smalltalk's editing paradigm.
    This is hard to pin down because Ruby isn't the most anything. Not most OO or most
    functional. Not the fastest, or the highest level.
    When we're designing systems, there's often pressure to stand out on some dimension.
    You see the cell networks competing for "most coverage" or "fastest network"
    Even Apple, a company that understands design, plays the game of making the
    "thinnest laptop ever"
    But sometimes, a beautiful system doesn't need to be the most anything.
    It can just be a perfect, well-balanced combination of many different ideas.
    Maybe next time you're designing a system, you can look to Ruby for inspiration for
    designing a well-balanced system.
    

    View Slide

71. Thanks!
    Twitter: @geoffreylitt
    

    View Slide