Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
600
Introduction to Rake
drbrain
0
290
Lessons in Mentorship
drbrain
1
180
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
95
Open Source Maintenance — RailsClub Moscow
drbrain
1
140
drbdump
drbrain
2
460
Other Decks in Programming
See All in Programming
Ça bouge du côté des animations CSS !
goetter
2
160
読まないコードリーディング術
hisaju
0
110
pylint custom ruleで始めるレビュー自動化
shogoujiie
0
160
AIプログラミング雑キャッチアップ
yuheinakasaka
19
5k
5分で理解する SOLID 原則 #phpcon_nagoya
shogogg
1
400
dbt Pythonモデルで実現するSnowflake活用術
trsnium
0
270
Jasprが凄い話
hyshu
0
180
LINE messaging APIを使ってGoogleカレンダーと連携した予約ツールを作ってみた
takumakoike
0
130
Honoとフロントエンドの 型安全性について
yodaka
7
1.5k
新宿駅構内を三人称視点で探索してみる
satoshi7190
2
120
Kotlinの開発でも AIをいい感じに使いたい / Making the Most of AI in Kotlin Development
kohii00
5
1.8k
CDKを使ったPagerDuty連携インフラのテンプレート化
shibuya_shogo
0
120
Featured
See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
49
2.3k
Building a Scalable Design System with Sketch
lauravandoore
461
33k
The Language of Interfaces
destraynor
156
24k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
7
650
Git: the NoSQL Database
bkeepers
PRO
428
65k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
13
1k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
Agile that works and the tools we love
rasmusluckow
328
21k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3k
Reflections from 52 weeks, 52 projects
jeffersonlam
348
20k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.2k
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
Transcript
Lazy Enumera,on Eric Hodel — drbrain@segment7.net
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)