Lazy Enumera,on Eric Hodel — [email protected] 

Loop values = [1, 2, 3, 4] doubles = [] index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end 

Enumera,ng Where am I? values[index] Am I done? index < values.length What’s next? index += 1 

Enumerator API Where am I? next #=> nil or exception if done Am I done? nil, StopIteration What’s next? handled for you 

External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while order = result.next do # … end 

Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row do |order| # … end 

External vs Internal You write loop Impera,ve C, Ruby Loop built-in Func,onal Scheme, Ruby 

Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value = orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total } 

Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value = orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total } 

100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a => 800000040 800MB 400ms 

Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value = orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me 

Eager Processing 100M 100M .map .map 100M .map 100M 

Lazy Processing 1 1 .map .map 1 .map 1 2 2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M 

How does lazy work? Fibers! 

Fiber? •Story line for a program •One Fiber runs at a ,me •Scheduled by author •“Corou,ne” 

Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled 

Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value) Return output to #resume 

Ac,ve Fiber 1 1 .map .map 1 .map 1 2 2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber 

Example of Fiber ⃠ 

Lazy Enumera,on •Reduces memory used •Great for huge data sets •Processes one at a ,me •Uses Fiber (corou,ne)