Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
630
Introduction to Rake
drbrain
0
310
Lessons in Mentorship
drbrain
1
200
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
110
Open Source Maintenance — RailsClub Moscow
drbrain
1
160
drbdump
drbrain
2
480
Other Decks in Programming
See All in Programming
Git Sync を超える!OSS で実現する CDK Pull 型デプロイ / Deploying CDK with PipeCD in Pull-style
tkikuc
4
470
MCPを使ってイベントソーシングのAIコーディングを効率化する / Streamlining Event Sourcing AI Coding with MCP
tomohisa
0
190
状態遷移図を書こう / Sequence Chart vs State Diagram
orgachem
PRO
3
280
LLMは麻雀を知らなすぎるから俺が教育してやる
po3rin
2
1.2k
Vibe Codingの幻想を超えて-生成AIを現場で使えるようにするまでの泥臭い話.ai
fumiyakume
18
9.5k
バイブコーディング超えてバイブデプロイ〜CloudflareMCPで実現する、未来のアプリケーションデリバリー〜
azukiazusa1
2
730
AIのメモリー
watany
11
1k
Google I/O Extended Incheon 2025 ~ What's new in Android development tools
pluu
1
200
新しいモバイルアプリ勉強会(仮)について
uetyo
1
190
Googleの新しいコーディングAIエージェントJulesを使ってみた
tonionagauzzi
0
140
Understanding Kotlin Multiplatform
l2hyunwoo
0
230
QA x AIエコシステム段階構築作戦
osu
0
210
Featured
See All Featured
Bash Introduction
62gerente
613
210k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
331
22k
Testing 201, or: Great Expectations
jmmastey
43
7.6k
For a Future-Friendly Web
brad_frost
179
9.8k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
21
1.4k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
850
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
Optimizing for Happiness
mojombo
379
70k
How STYLIGHT went responsive
nonsquared
100
5.7k
How to Think Like a Performance Engineer
csswizardry
25
1.8k
Writing Fast Ruby
sferik
628
62k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.8k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)