Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
640
Introduction to Rake
drbrain
0
320
Lessons in Mentorship
drbrain
1
210
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
110
Open Source Maintenance — RailsClub Moscow
drbrain
1
160
drbdump
drbrain
2
490
Other Decks in Programming
See All in Programming
RDoc meets YARD
okuramasafumi
4
170
Ruby Parser progress report 2025
yui_knk
1
420
Ruby×iOSアプリ開発 ~共に歩んだエコシステムの物語~
temoki
0
270
Putting The Genie in the Bottle - A Crash Course on running LLMs on Android
iurysza
0
140
rage against annotate_predecessor
junk0612
0
160
Compose Multiplatform × AI で作る、次世代アプリ開発支援ツールの設計と実装
thagikura
0
130
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
320
実用的なGOCACHEPROG実装をするために / golang.tokyo #40
mazrean
1
250
AI Coding Agentのセキュリティリスク:PRの自己承認とメルカリの対策
s3h
0
200
為你自己學 Python - 冷知識篇
eddie
1
350
アセットのコンパイルについて
ojun9
0
120
アルテニア コンサル/ITエンジニア向け 採用ピッチ資料
altenir
0
100
Featured
See All Featured
Why Our Code Smells
bkeepers
PRO
339
57k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
A better future with KSS
kneath
239
17k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6k
Embracing the Ebb and Flow
colly
87
4.8k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.7k
Scaling GitHub
holman
463
140k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Measuring & Analyzing Core Web Vitals
bluesmoon
9
580
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.9k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1.1k
Bash Introduction
62gerente
615
210k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)