Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
490
Introduction to Rake
drbrain
0
240
Lessons in Mentorship
drbrain
1
150
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
80
Open Source Maintenance — RailsClub Moscow
drbrain
1
130
drbdump
drbrain
2
420
Other Decks in Programming
See All in Programming
#phpcon_odawara オープン・クローズドなテストフィクスチャを求めて / open closed test fixtures
77web
3
220
Front-end application development, Symfony-style(s)
dunglas
2
1.9k
スクラムチームと認知負荷 - ニフティのスクラムトーク Vol2. / NIFTY Tech Talk #18
niftycorp
PRO
1
120
TYPO3 v13 – The road to LTS: What's new and new APIs
luisasofie_xoxo
0
180
ADRを一年運用してみた/adr_after_a_year
hanhan1978
7
2.2k
Netty Chicago Java User Group 2024-04-17
sullis
0
130
1인 개발자로 행복하게 살기 - GDG 송도 헬로월드 2024
benjaminkim
1
5.6k
ログラスを支える設計標準について / loglass-design-standards
urmot
10
2.1k
今、知っておきたい! 生成AIエージェントの世界
elith
3
340
0→1と1→10の狭間で Javaという技術選定を振り返る/Reflecting on the Decision to Choose Java Between Scaling from 0 to 1 and 1 to 10
jaguar_imo
2
370
Git Lint
bkuhlmann
4
740
try!Swift Tokyo 2024 参加報告 LT
akidon0000
1
190
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
186
16k
Building Adaptive Systems
keathley
30
1.8k
Clear Off the Table
cherdarchuk
83
310k
Documentation Writing (for coders)
carmenintech
59
3.9k
Visualization
eitanlees
135
14k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
39
4.4k
A better future with KSS
kneath
231
16k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
29
6k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
24
2.3k
Writing Fast Ruby
sferik
620
60k
Principles of Awesome APIs and How to Build Them.
keavy
120
16k
Design by the Numbers
sachag
274
18k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)