Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
580
Introduction to Rake
drbrain
0
270
Lessons in Mentorship
drbrain
1
160
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
87
Open Source Maintenance — RailsClub Moscow
drbrain
1
140
drbdump
drbrain
2
440
Other Decks in Programming
See All in Programming
テストコード文化を0から作り、変化し続けた組織
kazatohiei
2
1.5k
複雑な仕様に立ち向かうアーキテクチャ
myohei
0
170
テスト自動化失敗から再挑戦しチームにオーナーシップを委譲した話/STAC2024 macho
ma_cho29
1
1.3k
nekko cloudにおけるProxmox VE利用事例
irumaru
3
420
Criando Commits Incríveis no Git
marcelgsantos
2
170
talk-with-local-llm-with-web-streams-api
kbaba1001
0
180
CSC305 Lecture 26
javiergs
PRO
0
140
Effective Signals in Angular 19+: Rules and Helpers @ngbe2024
manfredsteyer
PRO
0
130
Webエンジニア主体のモバイルチームの 生産性を高く保つためにやったこと
igreenwood
0
330
暇に任せてProxmoxコンソール 作ってみました
karugamo
1
720
KMP와 kotlinx.rpc로 서버와 클라이언트 동기화
kwakeuijin
0
140
あれやってみてー駆動から成長を加速させる / areyattemite-driven
nashiusagi
1
200
Featured
See All Featured
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
2
290
Measuring & Analyzing Core Web Vitals
bluesmoon
4
170
A Tale of Four Properties
chriscoyier
157
23k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
111
49k
Writing Fast Ruby
sferik
628
61k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
232
17k
Keith and Marios Guide to Fast Websites
keithpitt
410
22k
Building an army of robots
kneath
302
44k
How GitHub (no longer) Works
holman
311
140k
Facilitating Awesome Meetings
lara
50
6.1k
Large-scale JavaScript Application Architecture
addyosmani
510
110k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
7k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)