Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
630
Introduction to Rake
drbrain
0
310
Lessons in Mentorship
drbrain
1
200
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
100
Open Source Maintenance — RailsClub Moscow
drbrain
1
150
drbdump
drbrain
2
480
Other Decks in Programming
See All in Programming
Cursor AI Agentと伴走する アプリケーションの高速リプレイス
daisuketakeda
1
130
Node-RED を(HTTP で)つなげる MCP サーバーを作ってみた
highu
0
100
エンジニア向け採用ピッチ資料
inusan
0
160
VS Code Update for GitHub Copilot
74th
1
410
Composerが「依存解決」のためにどんな工夫をしているか #phpcon
o0h
PRO
1
240
A2A プロトコルを試してみる
azukiazusa1
2
1.2k
明示と暗黙 ー PHPとGoの インターフェイスの違いを知る
shimabox
2
370
Benchmark
sysong
0
270
Deep Dive into ~/.claude/projects
hiragram
9
1.6k
PHPで始める振る舞い駆動開発(Behaviour-Driven Development)
ohmori_yusuke
2
220
GraphRAGの仕組みまるわかり
tosuri13
8
490
ruby.wasmで多人数リアルタイム通信ゲームを作ろう
lnit
2
280
Featured
See All Featured
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Facilitating Awesome Meetings
lara
54
6.4k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
A designer walks into a library…
pauljervisheath
207
24k
Practical Orchestrator
shlominoach
188
11k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.3k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Testing 201, or: Great Expectations
jmmastey
42
7.5k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
How to Ace a Technical Interview
jacobian
277
23k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)