$30 off During Our Annual Pro Sale. View Details »

Metaprogramming? Not good enough!

Justin Weiss
November 12, 2016

Metaprogramming? Not good enough!

If you know how to metaprogram in Ruby, you can create methods and objects on the fly, build Domain Specific Languages, or just save yourself a lot of typing. But can you change how methods are dispatched? Can you decide that the normal inheritance rules don't apply to some object?

In order to change those core parts of the language, there can't be much difference between how a language is implemented and how it's used. In this talk, you'll make that difference smaller, building a totally extensible object model on top of Ruby, using less than a dozen new classes and methods.

Justin Weiss

November 12, 2016
Tweet

More Decks by Justin Weiss

Other Decks in Programming

Transcript

  1. Metaprogramming?
    Not good enough!
    Justin Weiss
    @justinweiss | #tinyobj

    View Slide

  2. Metaprogramming?
    Not good enough!
    Justin Weiss
    @justinweiss | #tinyobj

    View Slide

  3. @justinweiss | #tinyobj

    View Slide

  4. @justinweiss | #tinyobj

    View Slide

  5. @justinweiss | #tinyobj

    View Slide

  6. @justinweiss | #tinyobj

    View Slide

  7. → 2005: Ruby
    @justinweiss | #tinyobj

    View Slide

  8. → 2005: Ruby
    → 2006: JavaScript Ruby
    @justinweiss | #tinyobj

    View Slide

  9. → 2005: Ruby
    → 2006: JavaScript Ruby
    → 2007: Erlang Ruby
    @justinweiss | #tinyobj

    View Slide

  10. → 2005: Ruby
    → 2006: JavaScript Ruby
    → 2007: Erlang Ruby
    → ...
    → 2015: Swift Ruby
    @justinweiss | #tinyobj

    View Slide

  11. What makes Ruby, Ruby?
    @justinweiss | #tinyobj

    View Slide

  12. @justinweiss | #tinyobj

    View Slide

  13. Erlang = Actors
    @justinweiss | #tinyobj

    View Slide

  14. Erlang = Actors
    Haskell = Monads
    @justinweiss | #tinyobj

    View Slide

  15. Erlang = Actors
    Haskell = Monads
    Lisp = Parentheses
    @justinweiss | #tinyobj

    View Slide

  16. Erlang = Actors
    Haskell = Monads
    Lisp = Parentheses Macros
    @justinweiss | #tinyobj

    View Slide

  17. Erlang = Actors
    Haskell = Monads
    Lisp = Parentheses Macros
    Ruby = Metaprogramming?
    @justinweiss | #tinyobj

    View Slide

  18. @justinweiss | #tinyobj

    View Slide

  19. Metaprogramming → !?
    @justinweiss | #tinyobj

    View Slide

  20. @justinweiss | #tinyobj

    View Slide

  21. @justinweiss | #tinyobj

    View Slide

  22. @justinweiss | #tinyobj

    View Slide

  23. @justinweiss | #tinyobj

    View Slide

  24. @justinweiss | #tinyobj

    View Slide

  25. A brand new
    object model
    (on top of Ruby)
    @justinweiss | #tinyobj

    View Slide

  26. Object model
    The concepts, data structures, and methods
    you use to build things in your language
    @justinweiss | #tinyobj

    View Slide

  27. @justinweiss | #tinyobj

    View Slide

  28. I. Piumarta and A. Warth
    Open, Extensible Object Models
    http://piumarta.com/software/cola/
    objmodel2.pdf
    @justinweiss | #tinyobj

    View Slide

  29. When you have a method
    name and arguments,
    which code do you use?
    @justinweiss | #tinyobj

    View Slide

  30. When you have a method
    name and arguments,
    which code do you use?
    @justinweiss | #tinyobj

    View Slide

  31. def lookup(object, method_name)
    # Look up a method by name in object's methods list
    # If it exists, return it
    # Otherwise, if you have a parent class, recurse
    end
    @justinweiss | #tinyobj

    View Slide

  32. @justinweiss | #tinyobj

    View Slide

  33. @justinweiss | #tinyobj

    View Slide

  34. What is behavior?
    @justinweiss | #tinyobj

    View Slide

  35. @justinweiss | #tinyobj

    View Slide

  36. @justinweiss | #tinyobj

    View Slide

  37. @justinweiss | #tinyobj

    View Slide

  38. @justinweiss | #tinyobj

    View Slide

  39. @justinweiss | #tinyobj

    View Slide

  40. @justinweiss | #tinyobj

    View Slide

  41. @justinweiss | #tinyobj

    View Slide

  42. Object = a thing
    Class = a thing that builds objects, holds
    methods, has a parent
    Behavior = a thing that holds methods like
    lookup and add_method
    @justinweiss | #tinyobj

    View Slide

  43. Object
    add_method
    lookup
    @justinweiss | #tinyobj

    View Slide

  44. How do we get an object
    from a class?
    @justinweiss | #tinyobj

    View Slide

  45. Object
    add_method
    lookup
    build_object
    @justinweiss | #tinyobj

    View Slide

  46. What about new classes,
    sub-classes, or sub-
    behaviors?
    @justinweiss | #tinyobj

    View Slide

  47. Object
    add_method
    lookup
    build_object
    delegate
    @justinweiss | #tinyobj

    View Slide

  48. How do you call a
    method?
    @justinweiss | #tinyobj

    View Slide

  49. Object
    add_method
    lookup
    build_object
    delegate
    send
    @justinweiss | #tinyobj

    View Slide

  50. Let's build it in Ruby!
    @justinweiss | #tinyobj

    View Slide

  51. class TinyObject < BasicObject
    attr_accessor :state
    attr_accessor :behavior
    end
    @justinweiss | #tinyobj

    View Slide

  52. class TinyObject < BasicObject
    attr_accessor :state
    attr_accessor :behavior
    end
    @justinweiss | #tinyobj

    View Slide

  53. Create the core methods
    @justinweiss | #tinyobj

    View Slide

  54. behavior_add_method = lambda do |behavior, method_name, method|
    behavior.state[:methods][method_name] = method
    end
    @justinweiss | #tinyobj

    View Slide

  55. behavior_add_method = lambda do |behavior, method_name, method|
    # -------^
    behavior.state[:methods][method_name] = method
    end
    @justinweiss | #tinyobj

    View Slide

  56. behavior_lookup = lambda do |behavior, method_name|
    # ...
    end
    @justinweiss | #tinyobj

    View Slide

  57. behavior_lookup = lambda do |behavior, method_name|
    # Look up a method by name in the object's methods list
    method = behavior.state[:methods][method_name]
    # ...
    method
    end
    @justinweiss | #tinyobj

    View Slide

  58. behavior_lookup = lambda do |behavior, method_name|
    # ...
    # If you can't find it, but have a parent class...
    if !method && behavior.state[:parent]
    # call this method recursively, using the parent instead
    method = behavior_lookup.call(behavior.state[:parent], method_name)
    end
    method
    end
    @justinweiss | #tinyobj

    View Slide

  59. behavior_lookup = lambda do |behavior, method_name|
    # Look up a method by name in the object's methods list
    method = behavior.state[:methods][method_name]
    # If you can't find it, but have a parent class...
    if !method && behavior.state[:parent]
    # call this method recursively, using the parent instead
    method = behavior_lookup.call(behavior.state[:parent], method_name)
    end
    method
    end
    @justinweiss | #tinyobj

    View Slide

  60. behavior_build_object = lambda do |behavior|
    # Allocate an object
    obj = TinyObject.new
    obj.state = {}
    # set the object's class / behavior
    # to yourself (the class that called this method)
    obj.behavior = behavior
    # return the object
    obj
    end
    @justinweiss | #tinyobj

    View Slide

  61. behavior_delegate = lambda do |parent_class|
    # ...
    end
    @justinweiss | #tinyobj

    View Slide

  62. @justinweiss | #tinyobj

    View Slide

  63. subclass = behavior_build_object.call(parent_class.behavior)
    @justinweiss | #tinyobj

    View Slide

  64. # set the object's parent (in its state) to ourselves
    subclass.state[:parent] = parent_class
    # initialize the object's state with an empty set of methods
    subclass.state[:methods] ||= {}
    subclass
    @justinweiss | #tinyobj

    View Slide

  65. behavior_delegate = lambda do |parent_class|
    parent_class_behavior = parent_class && parent_class.behavior
    subclass = behavior_build_object.call(parent_class_behavior)
    # set the object's parent (in its state) to ourselves
    subclass.state[:parent] = parent_class
    # initialize the object's state with an empty set of methods
    subclass.state[:methods] ||= {}
    subclass
    end
    @justinweiss | #tinyobj

    View Slide

  66. TinyObject: Our object structure, containing behavior and state
    behavior_add_method: Add a method to a class
    behavior_lookup: Find an implementation, from a method name
    behavior_build_object: Like .new, build an object from a class /
    behavior
    behavior_delegate: Inherit from a class / behavior
    @justinweiss | #tinyobj

    View Slide

  67. @justinweiss | #tinyobj

    View Slide

  68. default_behavior = behavior_delegate.call(nil)
    # ...
    @justinweiss | #tinyobj

    View Slide

  69. default_behavior = behavior_delegate.call(nil)
    default_behavior.behavior = default_behavior
    @justinweiss | #tinyobj

    View Slide

  70. root_object_class = behavior_delegate.call(nil)
    root_object_class.behavior = default_behavior
    @justinweiss | #tinyobj

    View Slide

  71. root_object_class = behavior_delegate.call(nil)
    root_object_class.behavior = default_behavior
    @justinweiss | #tinyobj

    View Slide

  72. # ...
    default_behavior.state[:parent] = root_object_class
    @justinweiss | #tinyobj

    View Slide

  73. default_behavior = behavior_delegate.call(nil)
    default_behavior.behavior = default_behavior
    root_object_class = behavior_delegate.call(nil)
    root_object_class.behavior = default_behavior
    default_behavior.state[:parent] = root_object_class
    @justinweiss | #tinyobj

    View Slide

  74. behavior_add_method.call(default_behavior,
    "lookup",
    behavior_lookup)
    behavior_add_method.call(default_behavior,
    "add_method",
    behavior_add_method)
    behavior_add_method.call(default_behavior,
    "build_object",
    behavior_build_object)
    behavior_add_method.call(default_behavior,
    "delegate",
    behavior_delegate)
    @justinweiss | #tinyobj

    View Slide

  75. @justinweiss | #tinyobj

    View Slide

  76. @justinweiss | #tinyobj

    View Slide

  77. object_send = lambda do |object, method_name, *args|
    method = find_method.call(object, method_name)
    if method
    method.call(object, *args)
    else
    raise "No method #{method_name} found on object"
    end
    end
    @justinweiss | #tinyobj

    View Slide

  78. find_method = lambda do |object, method_name|
    object_send.call(object.behavior, "lookup", method_name)
    end
    @justinweiss | #tinyobj

    View Slide

  79. find_method = lambda do |object, method_name|
    if (object == default_behavior && method_name == "lookup")
    behavior_lookup.call(default_behavior, "lookup")
    else
    object_send.call(object.behavior, "lookup", method_name)
    end
    end
    @justinweiss | #tinyobj

    View Slide

  80. behavior_add_method: Add a method to a class
    behavior_lookup: Find an implementation, from a method name
    behavior_build_object: Like .new, build an object from a class /
    behavior
    behavior_delegate: Inherit from a class / behavior
    object_send: Call a method, by name, on an object
    @justinweiss | #tinyobj

    View Slide

  81. greeter_class = object_send.call(root_object_class, "delegate")
    @justinweiss | #tinyobj

    View Slide

  82. greeter_class = object_send.call(root_object_class, "delegate")
    object_send.call(greeter_class, "add_method", "hello",
    lambda { |object| puts "Hello, world!" })
    @justinweiss | #tinyobj

    View Slide

  83. greeter = object_send.call(greeter_class, "build_object")
    object_send.call(greeter, "hello") # =>
    @justinweiss | #tinyobj

    View Slide

  84. greeter = object_send.call(greeter_class, "build_object")
    object_send.call(greeter, "hello") # => "Hello, world!"
    @justinweiss | #tinyobj

    View Slide

  85. @justinweiss | #tinyobj

    View Slide

  86. greeter = object_send.call(greeter_class, "build_object")
    object_send.call(greeter, "hello") # => "Hello, world!"
    @justinweiss | #tinyobj

    View Slide

  87. @justinweiss | #tinyobj

    View Slide

  88. class TinyObject < BasicObject
    attr_accessor :state
    attr_accessor :behavior
    def method_missing(name, *args)
    object_send = nil
    behavior = self.behavior
    # Crawl our ancestors for an "object_send" implementation
    while !object_send && behavior
    object_send = behavior.state[:methods]["object_send"]
    behavior = behavior.state[:parent]
    end
    # Call object_send on ourselves
    object_send.call(self, name.to_s, *args)
    end
    end
    @justinweiss | #tinyobj

    View Slide

  89. object_send.call(greeter, "hello")
    @justinweiss | #tinyobj

    View Slide

  90. object_send.call(greeter, "hello")

    greeter.hello
    @justinweiss | #tinyobj

    View Slide

  91. behavior_add_method.call(root_object_class, "object_send",
    object_send)
    @justinweiss | #tinyobj

    View Slide

  92. greeter_class.add_method("hello_name",
    lambda { |object| puts "Hello, #{object.state[:name]}!" })
    # ----^
    @justinweiss | #tinyobj

    View Slide

  93. alice = greeter_class.build_object
    bob = greeter_class.build_object
    alice.state[:name] = "Alice"
    bob.state[:name] = "Bob"
    alice.hello_name # =>
    bob.hello_name # =>
    @justinweiss | #tinyobj

    View Slide

  94. alice = greeter_class.build_object
    bob = greeter_class.build_object
    alice.state[:name] = "Alice"
    bob.state[:name] = "Bob"
    alice.hello_name # => "Hello, Alice!"
    bob.hello_name # => "Hello, Bob!"
    @justinweiss | #tinyobj

    View Slide

  95. Now what?
    @justinweiss | #tinyobj

    View Slide

  96. Log method calls!
    @justinweiss | #tinyobj

    View Slide

  97. def lookup
    # Get the old lookup method from the parent of our behavior
    # Call it to find the real method implementation
    # Wrap the method and return it
    end
    @justinweiss | #tinyobj

    View Slide

  98. method_logger = lambda do |method_name, method, *args|
    puts "> Calling #{method_name}... "
    method.call(*args)
    puts "> Done!"
    end
    @justinweiss | #tinyobj

    View Slide

  99. @justinweiss | #tinyobj

    View Slide

  100. intercepting_behavior = default_behavior.delegate
    @justinweiss | #tinyobj

    View Slide

  101. intercepting_behavior.add_method(
    "lookup",
    lambda do |sender, method_name|
    # Get the old lookup method from the parent of our class's behavior
    lookup = object_send.call(sender.behavior.state[:parent], "lookup", "lookup")
    # Call it to find the real method implementation
    method = lookup.call(sender, method_name)
    # Wrap the method and return it
    if method
    original_method = method
    interceptor = sender.state[:interceptor_method]
    method = lambda { |*args| interceptor.call(method_name, original_method, *args) }
    end
    method
    end
    )
    @justinweiss | #tinyobj

    View Slide

  102. # Get the old lookup method
    # from the parent of our class's behavior
    super_behavior = ?
    lookup = super_behavior.lookup("lookup")
    @justinweiss | #tinyobj

    View Slide

  103. super_behavior = sender.behavior.state[:parent]
    lookup = super_behavior.lookup("lookup")
    @justinweiss | #tinyobj

    View Slide

  104. # Get the old lookup method
    # from the parent of our class's behavior
    super_behavior = sender.behavior.state[:parent]
    lookup = super_behavior.lookup("lookup")
    # Call lookup to find the real method implementation
    method = lookup.call(sender, method_name)
    @justinweiss | #tinyobj

    View Slide

  105. # Wrap the method and return it
    if method
    original_method = method
    interceptor = sender.state[:interceptor_method]
    method = lambda do |*args|
    interceptor.call(method_name, original_method, *args)
    end
    end
    method
    @justinweiss | #tinyobj

    View Slide

  106. @justinweiss | #tinyobj

    View Slide

  107. person_class = root_object_class.delegate
    # ...
    @justinweiss | #tinyobj

    View Slide

  108. person_class = root_object_class.delegate
    person_class.behavior = intercepting_behavior
    @justinweiss | #tinyobj

    View Slide

  109. person_class = root_object_class.delegate
    # ...
    @justinweiss | #tinyobj

    View Slide

  110. person_class = root_object_class.delegate
    person_class.behavior = intercepting_behavior
    @justinweiss | #tinyobj

    View Slide

  111. method_logger = lambda do |method_name, method, *args|
    puts "> Calling #{method_name}... "
    method.call(*args)
    puts "> Done!"
    end
    person_class.state[:interceptor_method] = method_logger
    @justinweiss | #tinyobj

    View Slide

  112. person_class.add_method "name", lambda { |_| puts "Justin" }
    person_class.add_method "location", lambda { |_| puts "Cincinnati, OH" }
    @justinweiss | #tinyobj

    View Slide

  113. person = person_class.build_object
    person.name
    #
    #
    #
    person.location
    #
    #
    #
    @justinweiss | #tinyobj

    View Slide

  114. person = person_class.build_object
    person.name
    # > Calling name...
    # Justin
    # > Done!
    person.location
    # > Calling location...
    # Cincinnati, OH
    # > Done!
    @justinweiss | #tinyobj

    View Slide

  115. @justinweiss | #tinyobj

    View Slide

  116. person_class.behavior = default_behavior
    person.name
    #
    @justinweiss | #tinyobj

    View Slide

  117. person_class.behavior = default_behavior
    person.name
    # Justin
    @justinweiss | #tinyobj

    View Slide

  118. Retry method calls!
    @justinweiss | #tinyobj

    View Slide

  119. person_class.add_method("flaky_method", lambda do |_|
    if rand(3) == 0
    puts "Success!"
    true
    else
    false
    end
    end)
    @justinweiss | #tinyobj

    View Slide

  120. retry_interceptor = lambda do |method_name, method, *args|
    until method.call(*args)
    puts "Method #{method_name} failed, retrying..."
    sleep 0.5
    end
    end
    @justinweiss | #tinyobj

    View Slide

  121. person_class.state[:interceptor_method] = retry_interceptor
    person.flaky_method
    @justinweiss | #tinyobj

    View Slide

  122. person_class.state[:interceptor_method] = retry_interceptor
    person.flaky_method
    # Method flaky_method failed, retrying...
    @justinweiss | #tinyobj

    View Slide

  123. person_class.state[:interceptor_method] = retry_interceptor
    person.flaky_method
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    @justinweiss | #tinyobj

    View Slide

  124. person_class.state[:interceptor_method] = retry_interceptor
    person.flaky_method
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    @justinweiss | #tinyobj

    View Slide

  125. person_class.state[:interceptor_method] = retry_interceptor
    person.flaky_method
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    @justinweiss | #tinyobj

    View Slide

  126. person_class.state[:interceptor_method] = retry_interceptor
    person.flaky_method
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    # Method flaky_method failed, retrying...
    # Success!
    @justinweiss | #tinyobj

    View Slide

  127. @justinweiss | #tinyobj

    View Slide

  128. Multiple Inheritance
    @justinweiss | #tinyobj

    View Slide

  129. @justinweiss | #tinyobj

    View Slide

  130. @justinweiss | #tinyobj

    View Slide

  131. @justinweiss | #tinyobj

    View Slide

  132. @justinweiss | #tinyobj

    View Slide

  133. multiple_parents_behavior = default_behavior.delegate
    @justinweiss | #tinyobj

    View Slide

  134. multiple_parents_behavior.add_method("add_parent",
    lambda do |behavior, new_parent|
    # Initialize `parents` with the original parent
    unless behavior.state[:parents]
    behavior.state[:parents] = []
    parent = behavior.state[:parent]
    behavior.state[:parents] << parent if parent
    end
    # Append the new parent to `parents`
    behavior.state[:parents] << new_parent
    end
    )
    @justinweiss | #tinyobj

    View Slide

  135. multiple_parents_behavior.add_method(
    "lookup",
    lambda do |behavior, method_name|
    method = behavior.state[:methods][method_name]
    parents = behavior.state[:parents] ||
    [behavior.state[:parent]]
    if !method && parents
    parents.each do |parent|
    break if method = parent.lookup(method_name)
    end
    end
    method
    end
    )
    @justinweiss | #tinyobj

    View Slide

  136. multiple_parents_behavior.add_method(
    "lookup",
    lambda do |behavior, method_name|
    method = behavior.state[:methods][method_name]
    parents = behavior.state[:parents] ||
    [behavior.state[:parent]]
    if !method && parents
    parents.each do |parent|
    # v---- -----^
    break if method = parent.lookup(method_name)
    end
    end
    method
    end
    )
    @justinweiss | #tinyobj

    View Slide

  137. person_class = root_object_class.delegate
    person_class.add_method "name", lambda { |_| puts "Justin" }
    greeter_class = root_object_class.delegate
    greeter_class.add_method "greeting", lambda { |_| puts "Hello! Hello! Hello!" }
    @justinweiss | #tinyobj

    View Slide

  138. social_person_class = person_class.delegate
    social_person_class.behavior = multiple_parents_behavior
    social_person_class.add_parent(greeter_class)
    @justinweiss | #tinyobj

    View Slide

  139. justin = social_person_class.build_object
    justin.name # =>
    justin.greeting # =>
    @justinweiss | #tinyobj

    View Slide

  140. justin = social_person_class.build_object
    justin.name # => "Justin"
    justin.greeting # => "Hello! Hello! Hello!"
    @justinweiss | #tinyobj

    View Slide

  141. → Single inheritance
    → Interception
    → Multiple Inheritance
    @justinweiss | #tinyobj

    View Slide

  142. → Object
    → add_method
    → lookup
    → build_object
    → delegate
    → send
    @justinweiss | #tinyobj

    View Slide

  143. Modules?
    method_missing?
    BasicObject?
    delegation?
    @justinweiss | #tinyobj

    View Slide

  144. Intent
    @justinweiss | #tinyobj

    View Slide

  145. What's normal?
    @justinweiss | #tinyobj

    View Slide

  146. @justinweiss | #tinyobj

    View Slide

  147. @justinweiss | #tinyobj

    View Slide

  148. What? Why!?
    @justinweiss | #tinyobj

    View Slide

  149. @justinweiss | #tinyobj

    View Slide

  150. The best way to
    understand a system is to
    break it.
    @justinweiss | #tinyobj

    View Slide

  151. symbols.each do |symbol|
    klass = Class.new(ActiveRecord::Base) do
    def method_missing(name, *args)
    # ...
    end
    end
    Object.const_set(symbol.to_s.capitalize, klass)
    end
    @justinweiss | #tinyobj

    View Slide

  152. @justinweiss | #tinyobj

    View Slide

  153. @justinweiss | #tinyobj

    View Slide

  154. Try it out
    @justinweiss | #tinyobj

    View Slide

  155. Thank you!
    Justin Weiss
    @justinweiss
    https://www.avvo.com
    [email protected]
    https://www.justinweiss.com/
    rubyconf-2016
    @justinweiss | #tinyobj

    View Slide

  156. Thank you!
    Justin Weiss
    @justinweiss
    https://www.avvo.com
    [email protected]
    https://www.justinweiss.com/
    rubyconf-2016
    @justinweiss | #tinyobj

    View Slide

  157. The Ruby logo is Copyright © 2006, Yukihiro Matsumoto. It is
    licensed under the terms of the Creative Commons Attribution-
    ShareAlike 2.5 License agreement.
    "Happy Programmers" by Jesper Rønn-Jensen, used under CC BY-
    SA 2.0 / Resized from original
    "Laptop Stickers" by Nate Angell, used under CC BY 2.0 / Resized
    from original
    "No Brain" by Pierre-Olivier Carles, used under CC BY 2.0
    @justinweiss | #tinyobj

    View Slide