Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
120
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
640
Introduction to Rake
drbrain
0
320
Lessons in Mentorship
drbrain
1
210
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
120
Open Source Maintenance — RailsClub Moscow
drbrain
1
160
drbdump
drbrain
2
490
Other Decks in Programming
See All in Programming
Your Perfect Project Setup for Angular @BASTA! 2025 in Mainz
manfredsteyer
PRO
0
130
XP, Testing and ninja testing ZOZ5
m_seki
3
320
SpecKitでどこまでできる? コストはどれくらい?
leveragestech
0
550
フロントエンド開発に役立つクライアントプログラム共通のノウハウ / Universal client-side programming best practices for frontend development
nrslib
7
3.9k
非同期jobをtransaction内で 呼ぶなよ!絶対に呼ぶなよ!
alstrocrack
0
540
実践AIチャットボットUI実装入門
syumai
7
2.5k
AIで開発生産性を上げる個人とチームの取り組み
taniigo
0
130
プロダクト開発をAI 1stに変革する〜SaaS is dead時代で生き残るために〜 / AI 1st Product Development
kobakei
0
490
CSC509 Lecture 06
javiergs
PRO
0
240
メモリ不足との戦い〜大量データを扱うアプリでの実践例〜
kwzr
1
880
複雑化したリポジトリをなんとかした話 pipenvからuvによるモノレポ構成への移行
satoshi256kbyte
1
790
Reduxモダナイズ 〜コードのモダン化を通して、将来のライブラリ移行に備える〜
pvcresin
2
690
Featured
See All Featured
The World Runs on Bad Software
bkeepers
PRO
71
11k
Building an army of robots
kneath
306
46k
How to Ace a Technical Interview
jacobian
280
24k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
Thoughts on Productivity
jonyablonski
70
4.9k
Writing Fast Ruby
sferik
629
62k
Designing for humans not robots
tammielis
254
25k
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.9k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
45
2.5k
Automating Front-end Workflow
addyosmani
1371
200k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.7k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)