Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Lazy Enumeration
Search
Eric Hodel
April 27, 2016
Programming
0
110
Lazy Enumeration
An introduction to lazy enumeration in ruby
Eric Hodel
April 27, 2016
Tweet
Share
More Decks by Eric Hodel
See All by Eric Hodel
Building maintainable command-line tools with MRuby
drbrain
0
610
Introduction to Rake
drbrain
0
300
Lessons in Mentorship
drbrain
1
190
Open Source Maintenance — Ruby on Ales 2014
drbrain
1
99
Open Source Maintenance — RailsClub Moscow
drbrain
1
150
drbdump
drbrain
2
470
Other Decks in Programming
See All in Programming
AI Coding Agent Enablement - エージェントを自走させよう
yukukotani
13
5.8k
パスキーのすべて / 20250324 iddance Lesson.5
kuralab
0
150
PHPで書いたAPIをGoに書き換えてみた 〜パフォーマンス改善の可能性を探る実験レポート〜
koguuum
0
130
PHPバージョンアップから始めるOSSコントリビュート / how2oss-contribute
dmnlk
1
990
サービスクラスのありがたみを発見したときの思い出 #phpcon_odawara
77web
4
630
SQL Server ベクトル検索
odashinsuke
0
160
Kubernetesで実現できるPlatform Engineering の現在地
nwiizo
3
1.9k
Qiita Bash
mercury_dev0517
1
190
Youtube Lofier - Chrome拡張開発
ninikoko
0
2.4k
Bedrock×MCPで社内ブログ執筆文化を育てたい!
har1101
6
890
Boost Your Performance and Developer Productivity with Jakarta EE 11
ivargrimstad
0
1.2k
「影響が少ない」を自分の目でみてみる
o0h
PRO
2
960
Featured
See All Featured
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.5k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.1k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
34
2.2k
Statistics for Hackers
jakevdp
798
220k
Writing Fast Ruby
sferik
628
61k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Scaling GitHub
holman
459
140k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
Building Applications with DynamoDB
mza
94
6.3k
For a Future-Friendly Web
brad_frost
176
9.7k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
30
2.3k
Transcript
Lazy Enumera,on Eric Hodel —
[email protected]
Loop values = [1, 2, 3, 4] doubles = []
index = 0 while index < values.length do doubles << values[index] * 2 index += 1 end
Enumera,ng Where am I? values[index] Am I done? index <
values.length What’s next? index += 1
Enumerator API Where am I? next #=> nil or exception
if done Am I done? nil, StopIteration What’s next? handled for you
External Enumerator result = db_conn.exec ‘SELECT * FROM orders’ while
order = result.next do # … end
Internal Enumerator result = db_conn.exec ‘SELECT * FROM orders’ result.each_row
do |order| # … end
External vs Internal You write loop Impera,ve C, Ruby Loop
built-in Func,onal Scheme, Ruby
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 10,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
Eager Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.map { |order| # 100,000,000 values order[‘total’] }.reduce { |sum, order_total| sum + order_total }
100,000,000 Objects >> a = Array.new 100_000_000 >> ObjectSpace.memsize_of a
=> 800000040 800MB 400ms
Lazy Enumera,on orders = db_conn.exec ‘SELECT total FROM orders’ total_order_value
= orders.lazy.map { |order| order[‘total’] }.reduce { |sum, order_total| sum + order_total } 10MB similar ,me
Eager Processing 100M 100M .map .map 100M .map 100M
Lazy Processing 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M
How does lazy work? Fibers!
Fiber? •Story line for a program •One Fiber runs at
a ,me •Scheduled by author •“Corou,ne”
Hierarchy Process ↳Thread ↳Fiber OS scheduled Manually scheduled
Scheduling Fibers resume(input) #=> output Run a specific Fiber Fiber.yield(value)
Return output to #resume
Ac,ve Fiber 1 1 .map .map 1 .map 1 2
2 2 2 3 3 3 3 … … … … 100M 100M 100M 100M Fiber Fiber Fiber Fiber
Example of Fiber ⃠
Lazy Enumera,on •Reduces memory used •Great for huge data sets
•Processes one at a ,me •Uses Fiber (corou,ne)