Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
640
Introduction to Rake
drbrain
0
320
Lessons in Mentorship
drbrain
1
210
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
110
Open Source Maintenance — RailsClub Moscow
drbrain
1
160
drbdump
drbrain
2
490
Other Decks in Programming
See All in Programming
楽して成果を出すためのセルフリソース管理
clipnote
0
190
為你自己學 Python - 冷知識篇
eddie
1
350
デザイナーが Androidエンジニアに 挑戦してみた
874wokiite
0
550
RDoc meets YARD
okuramasafumi
4
170
GitHubとGitLabとAWS CodePipelineでCI/CDを組み比べてみた
satoshi256kbyte
4
250
個人軟體時代
ethanhuang13
0
330
複雑なドメインに挑む.pdf
yukisakai1225
5
1.2k
ファインディ株式会社におけるMCP活用とサービス開発
starfish719
0
2.1k
速いWebフレームワークを作る
yusukebe
5
1.7k
Things You Thought You Didn’t Need To Care About That Have a Big Impact On Your Job
hollycummins
0
110
はじめてのMaterial3 Expressive
ym223
2
900
意外と簡単!?フロントエンドでパスキー認証を実現する WebAuthn
teamlab
PRO
2
780
Featured
See All Featured
Six Lessons from altMBA
skipperchong
28
4k
How GitHub (no longer) Works
holman
315
140k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.7k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
It's Worth the Effort
3n
187
28k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)