Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
600
Introduction to Rake
drbrain
0
290
Lessons in Mentorship
drbrain
1
180
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
95
Open Source Maintenance — RailsClub Moscow
drbrain
1
140
drbdump
drbrain
2
460
Other Decks in Programming
See All in Programming
SwiftUI移行のためのインプレッショントラッキング基盤の構築
kokihirokawa
0
150
SwiftUI Viewの責務分離
elmetal
PRO
2
280
CSS Linter による Baseline サポートの仕組み
ryo_manba
1
160
コードを読んで理解するko build
bells17
1
110
1年目の私に伝えたい!テストコードを怖がらなくなるためのヒント/Tips for not being afraid of test code
push_gawa
1
620
LINE messaging APIを使ってGoogleカレンダーと連携した予約ツールを作ってみた
takumakoike
0
120
5分で理解する SOLID 原則 #phpcon_nagoya
shogogg
1
350
2025.2.14_Developers Summit 2025_登壇資料
0101unite
0
210
読まないコードリーディング術
hisaju
0
110
CDKを使ったPagerDuty連携インフラのテンプレート化
shibuya_shogo
0
110
React 19アップデートのために必要なこと
uhyo
8
1.5k
Formの複雑さに立ち向かう
bmthd
1
940
Featured
See All Featured
Code Review Best Practice
trishagee
67
18k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.2k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
12
1k
Music & Morning Musume
bryan
46
6.4k
Rails Girls Zürich Keynote
gr2m
94
13k
Into the Great Unknown - MozCon
thekraken
35
1.6k
We Have a Design System, Now What?
morganepeng
51
7.4k
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
Docker and Python
trallard
44
3.3k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.1k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)