Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Enumerator - Enumerable's Ugly Cousin

Enumerator - Enumerable's Ugly Cousin

Why don't more Rubyists use Enumerator? Learn more about this beautiful Ruby concept that could use a little more love.

B0169a78f851962058d63337ad0147d6?s=128

Ross Kaffenberger

March 31, 2016
Tweet

Transcript

  1. Enumerator Enumerable’s Ugly Cousin

  2. 2 @rossta NYC

  3. Pretend everything is normal NYC

  4. Choose one of two speeds: Move fast or… NYC

  5. GET OUT OF THE WAY NYC

  6. Question yourself at every turn… NYC

  7. NYC

  8. None
  9. What we say about Enumerable

  10. Powerful, simple, and elegant. How modules should be made! Learn

    it. Use it.
  11. Why I fell in love with Ruby

  12. What we say about Enumerator

  13. I don’t get it I would never write code this

    way That’s so ugly
  14. This seems like a big hack Why would I ever

    use this?
  15. OMFG

  16. None
  17. PRACTICAL EXPRESSIVE CLEAN ELEGANT MINASWAN READABLE SIMPLE SIMPLE HAPPINESS BEAUTIFUL

    CONCISE
  18. Conventions are great because Productivity!

  19. Conventions can limit our growth But…

  20. for loops are useful! C++ MATLAB Perl Algol BASIC PostScript

    C Pascal Ada Bash Haskell Python Lua Java JavaScript PHP ActionScript Go
  21. for n in [1, 2, 3] # ... end

  22. “Real” Rubyists use each, right? [1, 2, 3].each { |x|

    # ... }
  23. – Zed Shaw “When you teach people social norms as

    if they are universal truths you are actually indoctrinating them not educating them. .”
  24. “Code Morality” The “right” way and everything else

  25. None
  26. None
  27. None
  28. None
  29. None
  30. Social Norms + Conventions !== Universal truth

  31. Yes, follow conventions! but…

  32. Also explore unconventional ideas

  33. Unconventional Ugly

  34. What is Enumerator?

  35. Enumerator – Ruby Docs “A class which allows both internal

    and external iteration.”
  36. enum = [1, 2, 3].to_enum # => #<Enumerator: [1, 2,

    3]:each> enum.next # => 1 enum.next # => 2 enum.next # => 3 enum.next # => StopIteration enum.each { |x| # ... }
  37. Let’s explore

  38. Java What I learned from

  39. Iterator

  40. ArrayList list = list.add( list.add( list.add( ArrayList list = new

    ArrayList(); list.add(1); list.add(2); list.add(3); Iterator it = list.iterator(); while(it.hasNext()) { Object element = it.next(); System.out.println(element); };
  41. ArrayList list = list.add( list.add( list.add( Iterator it = list.iterator();

    while Object element = it.next(); System.out.println(element); }; it.next();
  42. Iterator an object that can suspend iteration and pass control

    back to its caller
  43. Iterator Like yield turned inside out

  44. def each yield 1 # to block yield 2 #

    to block yield 3 # to block end enum = [1, 2, 3].to_enum enum.next # yield 1 enum.next # yield 2 enum.next # yield 3
  45. Use case: Replace nested loops

  46. <table> <% projects.each do |project| %> <tr style="background: <%= colors.next

    %>"> <td><%= project.name %></td> </tr> <% end %> </table> <% colors = %w[aliceblue ghostwhite].to_enum(:cycle) %>
  47. <% colors = <table> <% <% </table> colors.next

  48. What I learned from JavaScript & Python

  49. Generator

  50. Iterators Generators

  51. Generators #next

  52. Generators produce data on the fly

  53. async infinite sequences fetching data list comprehensions concurrency lazy evaluation

    coroutines
  54. JavaScript Generators in

  55. None
  56. –AirBnB JavaScript Style Guide “Don't use generators.”

  57. None
  58. None
  59. None
  60. None
  61. Python Generators in

  62. Awesome Simple, expressive. Love them. List comprehensions… insanely readable and

    easy to maintain
  63. gen = python_generator() gen.next() # => 1 gen.next() # =>

    2 gen.next() # => 3 def python_generator(): yield 1 yield 2 yield 3
  64. def fibonacci(n): result = []

  65. def fibonacci(n): result = [] a = b = 1

    for i in xrange(n): result.append(a) a, b = b, a + b return result
  66. def fibonacci(n): result = [] a = b = 1

    for i in xrange(n): result.append(a) a, b = b, a + b return result for a in fibonacci(20): print a
  67. def fibonacci(n): a = b = 1 for i in

    xrange(n): a, b = b, a + b for a in fibonacci(20): print a
  68. def fibonacci(n): a = b = 1 for i in

    xrange(n): yield a a, b = b, a + b for a in fibonacci(20): print a
  69. gen = fibonacci(20) gen.next() gen.next() gen.next() def a = b

    = a, b = b, a + b
  70. Generator a function that can suspend its execution and pass

    control back to its caller
  71. Generator Provides ability to treat algorithms as collections

  72. Can we write generators in Ruby?* *yes, we can! Q:

  73. def fibonacci(n): a = b = 1 for i in

    xrange(n): yield a a, b = b, a + b for a in fibonacci(20): print a
  74. def fibonacci(n) a = b = 1 n.times do yield

    a a, b = b, a + b end end fibonacci(20) { |a| puts a } Not enumerable!
  75. Enumeratorize it!

  76. def fibonacci(n) a = b = 1 n.times do yield

    a a, b = b, a + b end end def a = b = n.times a, b = b, a + b end return to_enum(__method__, n) unless block_given?
  77. return to_enum(__method__) unless block_given? This seems like a big hack

    Too magical That’s ugly
  78. def fibonacci(n) a = b = 1 n.times do yield

    a a, b = b, a + b end end def a = b = n.times a, b = b, a + b end fibonacci(7).each { |a| puts a } fibonacci(25).map { |a| a * 3 }.select(&:odd?) fibonacci(100).find { |a| a > 50 } return to_enum(__method__, n) unless block_given? ] [ Enumerable!
  79. Enumerable + Generator == Enumerator

  80. to_enum

  81. to_enum is everywhere

  82. [1, 2, 3].to_enum(:each) # => #<Enumerator: [1, 2, 3]:each> [1,

    2, 3].to_enum(:map) # => #<Enumerator: [1, 2, 3]:map> [1, 2, 3].to_enum(:select) # => #<Enumerator: [1, 2, 3]:select> [1, 2, 3].to_enum(:group_by) # => #<Enumerator: [1, 2, 3]:group_by> [1, 2, 3].to_enum(:cycle) # => #<Enumerator: [1, 2, 3]:cycle>
  83. [1, 2, 3].each # => #<Enumerator: [1, 2, 3]:each> [1,

    2, 3].map # => #<Enumerator: [1, 2, 3]:map> [1, 2, 3].select # => #<Enumerator: [1, 2, 3]:select> [1, 2, 3].group_by # => #<Enumerator: [1, 2, 3]:group_by> [1, 2, 3].cycle # => #<Enumerator: [1, 2, 3]:cycle>
  84. Fixnum 4.times 1.upto(10) 10.upto(1) Range (1..10).each (1..10).step Struct struct.each struct.each_pair

    String "123".each_byte "123".each_char "123".each_codepoint "123".each_line "123".gsub File File.foreach file.each file.each_line file.each_byte file.each_codepoint ObjectSpace ObjectSpace.each_object loop
  85. Object#to_enum Not magic!

  86. We can re-implement to_enum in Ruby

  87. obj_to_enum(int argc, VALUE *argv, VALUE obj) { VALUE enumerator, meth

    = sym_each; if (argc > 0) { --argc; meth = *argv++; } enumerator = rb_enumeratorize_with_size(obj, meth, argc, argv, 0); if (rb_block_given_p()) { enumerator_ptr(enumerator)->size = rb_block_proc(); } return enumerator; } static VALUE enumerator.c
  88. obj_to_enum( { VALUE enumerator, meth = sym_each; --argc; meth =

    *argv++; } enumerator = rb_enumeratorize_with_size(obj, meth, argc, argv, enumerator_ptr(enumerator)->size = rb_block_proc(); } return enumerator; } rb_enumeratorize_with_size(obj, meth, static VALUE enumerator.c
  89. VALUE rb_enumeratorize_with_size(VALUE obj, VALUE meth, int *size_fn) { dispatching to

    either return lazy_to_enum_i(obj, meth, argc, argv, size_fn); return enumerator_init( enumerator_allocate(rb_cEnumerator), } return enumerator_init( enumerator_allocate(rb_cEnumerator), obj, meth, argc, argv, size_fn, Qnil); rb_enumeratorize_with_size
  90. def to_enum(method, *args) Enumerator.new(self, method, *args) end

  91. Enumerator Collection Enumerable method

  92. None
  93. static next_i { VALUE nil VALUE result; result next_ii e

    "iteration reached an end" return } next_i(VALUE curr, VALUE obj) return rb_fiber_yield(1, &nil); enumerator.c Fiber!
  94. We can re-implement Enumerator with Fiber

  95. Fiber.new fiber = Fiber.new do Fiber.yield 1 Fiber.yield 2 Fiber.yield

    3 end fiber.resume # => 1 fiber.resume # => 2 fiber.resume # => 3
  96. Look familiar? Q:

  97. gen = python_generator() gen.next() # => 1 gen.next() # =>

    2 gen.next() # => 3 def python_generator(): yield 1 yield 2 yield 3
  98. Fiber.new fiber = Fiber.new do Fiber.yield 1 Fiber.yield 2 Fiber.yield

    3 end fiber.resume # => 1 fiber.resume # => 2 fiber.resume # => 3
  99. a block that can suspend its execution and pass control

    back to its caller Fiber
  100. Fiber =~ Generator

  101. def to_enum(method, *args) Enumerator.new(self, method, *args) end

  102. class CustomEnumerator def initialize(collection, meth, *args) @collection = collection @fiber

    = Fiber.new do @collection.send(meth, *args) do |n| Fiber.yield(n) end raise StopIteration end end end class CustomEnumerator def initialize(collection, meth, *args) @collection = collection class CustomEnumerator
  103. @fiber = Fiber.new do @collection.send(meth, *args) do |n| Fiber.yield(n) end

    raise StopIteration end class def end
  104. Enumerator Fiber Collection Enumerable method

  105. class CustomEnumerator def next @fiber.resume end end

  106. class CustomEnumerator include Enumerable def next @fiber.resume end def each

    loop { yield self.next } end end # => StopIteration
  107. Enumerable + Fiber == Enumerator

  108. Should we “enumeratorize” by convention? Q:

  109. class PaginatedApiClient def posts(page=0) return to_enum(:posts, page) unless bg? #

    fetch page # yield each post # call posts(page+1) end end
  110. class Document include Enumerable def each # yield each end

    end paragraph word line class Document
  111. class Document def each_line return to_enum(:each_line) unless block_given? # yield

    each line end def each_word return to_enum(:each_word) unless block_given? # yield each word end def each_para return to_enum(:each_para) unless block_given? # yield each paragraph end end class Document def each_line return to_enum(:each_line) unless block_given? # yield each line end class Document def each_line return to_enum(:each_line) unless block_given? # yield each line end def each_word return to_enum(:each_word) unless block_given? # yield each word end class Document
  112. class BinaryTree def breadth_first return to_enum(__method__) unless block_given? # yield

    values in “breadth first” order end def pre_order # etc… end def post_order end def in_order end end
  113. tree = tree.breadth_first. with_index. partition { |n, i| i.odd? }.

    flat_map(&:join) # => “b1d3f5”, “a0c2e4”
  114. Ugly but effective return to_enum(__method__) unless block_given? tree.breadth_first. with_index. partition

    { #… } flat_map { #… }
  115. What I learned from Clojure & Elixir

  116. Infinite Sequence

  117. (take 2 (filter odd? [0 1 2 3 4 5]))

    [0, 1, 2, 3, 4, 5] |> Enum.filter(&Integer.is_odd/1) |> Enum.take(2)
  118. (take 2 (filter odd? (iterate inc 0))) Stream.iterate(0, &(&1 +

    1)) |> Stream.filter(&Integer.is_odd/1) |> Enum.take(2)
  119. We don’t have infinite sequences in Ruby… or do we?

  120. def fibonacci Enumerator.new do |y| a = b = 1

    loop do y.yield a a, b = b, a + b end end end like Fiber.yield!
  121. def a = b = y.yield a a, b =

    b, a + b end fibonacci.lazy. select(&:odd?). take(3)
  122. lazy augments how data is processed in an enumerator chain

  123. Eager Pipeline [] map filter sum

  124. Lazy Pipeline [] map filter sum

  125. Failed Project What I learned from a

  126. Recurrence

  127. 2008

  128. What’s the estimate? Oh, about 4 days

  129. 4 days weeks

  130. Modeling Recurrence

  131. { every: :month, on: { friday: 13 } }

  132. Present Day: Redemption

  133. Wish list enumerable lazily generate events infinite recurrence queryable

  134. i.e., an algorithm for an enumerable infinite recurring events

  135. Enumeratorize it!

  136. $ gem install montrose

  137. Recurrence Enumerator find next event yield stop or continue

  138. # Expressive Montrose.weekly(on: :monday, at: "9 am") # => #<Montrose::Recurrence...>

    # Flexible Montrose.hourly.interval(3) Montrose.every(3.hours) Montrose.r(every: 3.hours) # Chainable Montrose.monthly. starting(1.year.from_now). on(friday: 13). repeat(5)
  139. # Enumerable r = Montrose.monthly r.until(1.year.from_now).each do |event| # …

    end r.lazy.chunk(&:month).take(8).to_a # => [ [4, 2016-04-06 12:00:00 -0500], [5, 2016-05—06 12:00:00 -0500], [6, 2016-06—06 12:00:00 -0500], ...]
  140. Enumerator is beautiful

  141. fibers infinite sequences generators iterators lazy evaluation Enumerator Enumerable

  142. This wasn’t just a talk about Enumerator

  143. –Me “I wasn’t capable of writing Montrose a few years

    go. I needed to get out of my comfort zone.”
  144. MY RUBY GROWTH CYCLE Uninformed Optimism Informed Pessimism Informed Optimism

  145. None
  146. “WTF?” Instead of

  147. “What can I learn from this?” Try

  148. Sometimes, ugly code can teach us something

  149. Go forth and be curious

  150. @rossta rossta.net/talks Ross Kaffenberger