Arrrr Camp 2012: Simulating the World with Ruby

Transcript

1. ®
2. The (Ruby) Sims (part 2)
3. | @bryanl | @thunderboltlabs | Sim(foo) Introduction
4. | @bryanl | @thunderboltlabs | Sim(foo) @bryanl
5. | @bryanl | @thunderboltlabs | Sim(foo)
6. | @bryanl | @thunderboltlabs | Sim(foo)
7. | @bryanl | @thunderboltlabs | Sim(foo)
8. | @bryanl | @thunderboltlabs | Sim(foo)
9. | @bryanl | @thunderboltlabs | Sim(foo)
10. | @bryanl | @thunderboltlabs | Sim(foo)
11. | @bryanl | @thunderboltlabs | Sim(foo) CootieSim2012 v2.0
12. | @bryanl | @thunderboltlabs | Sim(foo) Why RUBY?
13. | @bryanl | @thunderboltlabs | Sim(foo) Stochastic vs Deterministic
14. | @bryanl | @thunderboltlabs | Sim(foo) Define inputs Compute Analyze output
15. | @bryanl | @thunderboltlabs | Sim(foo) SIMULATION_LENGTH = 100 PEOPLE_COUNT = 100 def create_people(people_count); end def report; end def simulate_to(count) 1.upto(count) {|i| yield i } end people = create_people(PEOPLE_COUNT) simulate_to(SIMULATION_LENGTH) do |iteration| # mish and mash people end report
16. | @bryanl | @thunderboltlabs | Sim(foo) class Simulator def initialize(length, count) # ... end end s = Simulator.new 3650, 100 Define inputs
17. | @bryanl | @thunderboltlabs | Sim(foo) class Simulator def simulate(iteration) create_pairs(iteration) transmit(iteration) progress_infections(iteration) end end Compute
18. | @bryanl | @thunderboltlabs | Sim(foo) s = Simulator.new 3650, 100 s.run s.infected_people.each do |p| i = p.infections.to_a d = p.diseases.to_a puts "#{p} #{i} #{d}" end Analyze output
20. | @bryanl | @thunderboltlabs | Sim(foo) Breakdown
21. | @bryanl | @thunderboltlabs | Sim(foo) What’s in a day?
22. | @bryanl | @thunderboltlabs | Sim(foo) Create new pair
23. | @bryanl | @thunderboltlabs | Sim(foo) def create_pairs(iteration) @pairs += people_not_in_pairs(iteration) .shuffle .each_slice(2) .map{|x| Pair.new iteration, x} end
24. | @bryanl | @thunderboltlabs | Sim(foo) Transmit cooties
25. | @bryanl | @thunderboltlabs | Sim(foo) class Simulation def transmit(iteration) current_pairs(iteration).each{|pair| pair.contact(iteration)} end end class Pair def contact(iteration) @alpha.infect(iteration, @beta) @beta.infect(iteration, @alpha) end end class Person def infect(iteration, person); end def acquire(infection); end end
26. | @bryanl | @thunderboltlabs | Sim(foo) Track infection progress
27. | @bryanl | @thunderboltlabs | Sim(foo) class Person def progress_infections(iteration) @infections.each do |infection| if rand <= infection.chance_to_acquire_disease unless has_disease? infection.disease @diseases << infection.disease.new(iteration) end end end end end
28. | @bryanl | @thunderboltlabs | Sim(foo) ... This isn’t going to be fast
29. | @bryanl | @thunderboltlabs | Sim(foo) 28.23s user 0.10s system 99% cpu 28.333 total
30. | @bryanl | @thunderboltlabs | Sim(foo) but why?
31. | @bryanl | @thunderboltlabs | Sim(foo) %self total self wait child calls name 30.32 386.26 117.53 0.00 268.72 21904 Array#each 28.08 180.05 108.88 0.00 71.18 78738195 Comparable#between? 21.74 84.29 84.29 0.00 0.00 78738195 Pair#duration 18.36 71.18 71.18 0.00 0.00 157476390 Fixnum#<=> 0.29 2.11 1.11 0.00 1.06 414957 Set#each 0.13 1.15 0.50 0.00 0.65 365100 Person#infected? 0.13 2.36 0.50 0.00 1.86 365000 Person#infect 0.12 0.65 0.48 0.00 0.17 365100 Set#empty?
32. | @bryanl | @thunderboltlabs | Sim(foo) module Enumerable # create a thread and execute in parallel def parallel_each(&block) # create thread yield # cleanup end end
33. | @bryanl | @thunderboltlabs | Sim(foo)
34. | @bryanl | @thunderboltlabs | Sim(foo) 100 people
35. | @bryanl | @thunderboltlabs | Sim(foo) 3650 iterations
36. | @bryanl | @thunderboltlabs | Sim(foo) 365,000 “actions”
37. | @bryanl | @thunderboltlabs | Sim(foo)
38. | @bryanl | @thunderboltlabs | Sim(foo) 3,650,000 @ 1,000 36,500,000 @ 10,000 365,000,000 @ 100,000 3,650,000,000 @ 1,000,000
39. | @bryanl | @thunderboltlabs | Sim(foo) How do we make it faster?
40. | @bryanl | @thunderboltlabs | Sim(foo) Get a faster CPU?
41. | @bryanl | @thunderboltlabs | Sim(foo) 1. Run less code 2. Do more things at once
42. | @bryanl | @thunderboltlabs | Sim(foo) Can’t really make it “faster”
43. | @bryanl | @thunderboltlabs | Sim(foo) MRI 1.9
44. | @bryanl | @thunderboltlabs | Sim(foo)
45. | @bryanl | @thunderboltlabs | Sim(foo) Thread.new
46. | @bryanl | @thunderboltlabs | Sim(foo)
47. | @bryanl | @thunderboltlabs | Sim(foo) JRuby
48. | @bryanl | @thunderboltlabs | Sim(foo)
49. | @bryanl | @thunderboltlabs | Sim(foo) Thread.new
50. | @bryanl | @thunderboltlabs | Sim(foo)
51. | @bryanl | @thunderboltlabs | Sim(foo) JRuby’s sweetspot is Hotspot
52. | @bryanl | @thunderboltlabs | Sim(foo) 11.34s user 0.37s system 116% cpu 10.064 total
53. | @bryanl | @thunderboltlabs | Sim(foo) Doing more than one thing at once with MRI
54. | @bryanl | @thunderboltlabs | Sim(foo) fork
55. | @bryanl | @thunderboltlabs | Sim(foo) external processes
56. | @bryanl | @thunderboltlabs | Sim(foo) statistics
57. | @bryanl | @thunderboltlabs | Sim(foo) mean median variance standard deviation σ
58. | @bryanl | @thunderboltlabs | Sim(foo) How do i randomly pick an integer from 1-6?
59. | @bryanl | @thunderboltlabs | Sim(foo) rand(1..6) rand(6)+1
60. | @bryanl | @thunderboltlabs | Sim(foo) gem install vose
61. | @bryanl | @thunderboltlabs | Sim(foo) v = Vose::AliasMethod.new [1,1,1,1,1,1] (1..10000).map{v.next}.inject(Hash.new{0}){ |m,n| m[n]+=1; m} (1..10000).map{rand(1..6)} .inject(Hash.new{0}){|m,n| m[n]+=1; m} VS
62. | @bryanl | @thunderboltlabs | Sim(foo) What if the die is weighted?
63. | @bryanl | @thunderboltlabs | Sim(foo) v = Vose::AliasMethod.new [1.2,1,1,1,1,1] (1..10000).map{v.next}.inject(Hash.new{0}){ |m,n| m[n]+=1; m} ????? VS
64. | @bryanl | @thunderboltlabs | Sim(foo) why is this important?
65. | @bryanl | @thunderboltlabs | Sim(foo)
66. | @bryanl | @thunderboltlabs | Sim(foo)
67. | @bryanl | @thunderboltlabs | Sim(foo) graphing
68. | @bryanl | @thunderboltlabs | Sim(foo) Trends
71. | @bryanl | @thunderboltlabs | Sim(foo) library(ggplot2) runtime.csv <- read.csv("runtime.csv", header=T) mygg<- qplot(iteration, time, data=runtime.csv); mygg<- mygg + stat_smooth(method="lm") ggsave("runtime.png") dev.off()
72. | @bryanl | @thunderboltlabs | Sim(foo) library(ggplot2) runtime.csv <- read.csv("runtime.csv", header=T) mygg<- qplot(iteration, time, data=runtime.csv); mygg<- mygg + stat_smooth(method="lm") ggsave("runtime.png") dev.off() R
73. | @bryanl | @thunderboltlabs | Sim(foo) Visualization
74. | @bryanl | @thunderboltlabs | Sim(foo)
75. | @bryanl | @thunderboltlabs | Sim(foo) digraph g { "person59" -> "person5"; "person23" -> "person18"; "person72" -> "person23"; "person78" -> "person44"; "person78" -> "person59"; "person59" -> "person72"; "person1" -> "person78"; "person83" -> "person81"; "person59" -> "person83"; "person59" -> "person92"; };
76. | @bryanl | @thunderboltlabs | Sim(foo) digraph g { "person59" -> "person5"; "person23" -> "person18"; "person72" -> "person23"; "person78" -> "person44"; "person78" -> "person59"; "person59" -> "person72"; "person1" -> "person78"; "person83" -> "person81"; "person59" -> "person83"; "person59" -> "person92"; }; Graphviz
77. | @bryanl | @thunderboltlabs | Sim(foo) Building a simulator (v1.0)
78. | @bryanl | @thunderboltlabs | Sim(foo)
79. | @bryanl | @thunderboltlabs | Sim(foo)
80. | @bryanl | @thunderboltlabs | Sim(foo)
81. | @bryanl | @thunderboltlabs | Sim(foo) ?!
82. | @bryanl | @thunderboltlabs | Sim(foo) Shared Cooties?
83. | @bryanl | @thunderboltlabs | Sim(foo) Making it fast
84. | @bryanl | @thunderboltlabs | Sim(foo) Be kind to MRI’s GC
85. | @bryanl | @thunderboltlabs | Sim(foo) Run more than one process
86. | @bryanl | @thunderboltlabs | Sim(foo) Look outside of MRI
87. | @bryanl | @thunderboltlabs | Sim(foo) irb(main):003:0> rand 1..7 TypeError: can't convert Range into Integer " from org/jruby/RubyKernel.java:1580:in `rand' " from (irb):3:in `evaluate' " from org/jruby/RubyKernel.java:1070:in `eval' " from org/jruby/RubyKernel.java:1395:in `loop' " from org/jruby/RubyKernel.java:1178:in `catch' " from org/jruby/RubyKernel.java:1178:in `catch' " from /Users/bryan/.rbenv/versions/jruby-1.7.0-preview2/bin/irb:13:in `(root)' JRuby 1.7 pre2
88. | @bryanl | @thunderboltlabs | Sim(foo) irb(main):001:0> rand 1..7 => 3 MRI 1.9.3
89. | @bryanl | @thunderboltlabs | Sim(foo) other languages?
90. | @bryanl | @thunderboltlabs | Sim(foo) Building a simulator (v2.0)
91. | @bryanl | @thunderboltlabs | Sim(foo) #!/usr/bin/env ruby require 'benchmark' n,m = [1_000_000, 100] Benchmark.bm do |x| x.report do m.times do a = []; n.times{|i| a[i] = i}; n.times{ j=a[rand(n)]; j*j} end end x.report do m.times do h = {}; n.times{|i| h[i] = i}; n.times{j=h[rand(n)]; j*j} end end end
92. | @bryanl | @thunderboltlabs | Sim(foo) -> ❸ % ruby big_arrays.rb user system total real 41.920000 0.330000 42.250000 ( 42.258538) 138.270000 2.860000 141.130000 (141.366728) [~/Development/talks/arrrrcamp2012/src/lib] -> ♢ % ruby big_arrays.rb user system total real 88.320000 0.570000 88.890000 ( 41.623000) 191.970000 0.460000 192.430000 ( 73.113000) [~/Development/talks/arrrrcamp2012/src/lib]
93. | @bryanl | @thunderboltlabs | Sim(foo) -> ❸ % ruby discrete.rb 36500 1000 creating 1000 people inoculating 50% of the population circle circle dot dot. now you have a cootie shot starting simulation 3650 7300 10950 14600 18250 21900 25550 29200 32850 36500 runtime: 129.49060773849487 Infected: 479 Noninfected: 521
94. | @bryanl | @thunderboltlabs | Sim(foo) -> ♢ % ruby discrete.rb 36500 1000 creating 1000 people inoculating 50% of the population circle circle dot dot. now you have a cootie shot starting simulation 3650 7300 10950 14600 18250 21900 25550 29200 32850 36500 runtime: 129.88899993896484 Infected: 526 Noninfected: 474
95. | @bryanl | @thunderboltlabs | Sim(foo) Rails won’t scale
96. | @bryanl | @thunderboltlabs | Sim(foo) Ruby won’t scale
97. | @bryanl | @thunderboltlabs | Sim(foo) *this* won’t scale
98. | @bryanl | @thunderboltlabs | Sim(foo)
99. | @bryanl | @thunderboltlabs | Sim(foo) Ready to pair?
100. | @bryanl | @thunderboltlabs | Sim(foo) Ready to pair?
101. | @bryanl | @thunderboltlabs | Sim(foo) Ready to pair?
102. | @bryanl | @thunderboltlabs | Sim(foo) Will something happen?
103. | @bryanl | @thunderboltlabs | Sim(foo) Will something happen?
104. | @bryanl | @thunderboltlabs | Sim(foo) 1000 people 636 pairs 1636 choices ___________ +
105. | @bryanl | @thunderboltlabs | Sim(foo) class Event def schedule; end def run # do stuff end end e = Event.new e.run e.schedule
106. | @bryanl | @thunderboltlabs | Sim(foo) class PairEvent < Event def initialize(person) @person = person end def run # add person to “available # to pair pool” end end
107. | @bryanl | @thunderboltlabs | Sim(foo) class ContactEvent < Event def schedule # create future contact event end def run # perform a contact end end
108. | @bryanl | @thunderboltlabs | Sim(foo) how do we keep track of events?
109. | @bryanl | @thunderboltlabs | Sim(foo) event_pool = [] event_pool << event def do_step(current_step) current_events = event_pool.find_all do |event| event.on?(current_step) end current_events.each do |event| event.run event_pool << event.schedule end end
110. | @bryanl | @thunderboltlabs | Sim(foo) event_pool.insert(x, event.schedule)
111. | @bryanl | @thunderboltlabs | Sim(foo) n = 100_000 Benchmark.bm do |x| x.report {a = [1] * 10; n.times{|i| a << i}} x.report {a = [1] * 10; n.times{|i| a.insert(1, i)}} end
112. | @bryanl | @thunderboltlabs | Sim(foo) user system total real 0.010000 0.000000 0.010000 ( 0.010555) 1.670000 0.000000 1.670000 ( 1.667457)
113. | @bryanl | @thunderboltlabs | Sim(foo) we can do better
114. | @bryanl | @thunderboltlabs | Sim(foo) gem install algorithms
115. | @bryanl | @thunderboltlabs | Sim(foo) event_pool = Containers::PriorityQueue.new event_pool.push event, (length-current_step) while (event_pool.size > 0) event = event_pool.pop event.run(current_step) event.schedule(event_pool) end
116. | @bryanl | @thunderboltlabs | Sim(foo) a.k.a. Discrete Event Simulation
117. | @bryanl | @thunderboltlabs | Sim(foo) but wait...
118. | @bryanl | @thunderboltlabs | Sim(foo) n = 1000 Benchmark.bm do |x| x.report {a = [1] * 10; n.times{|i| a << i}} x.report {a = [1] * 10; n.times{|i| a.insert(1, i)}} x.report do pq = PriorityQueue.new n.times{|i| pq.push(i, 2)} } end end
119. | @bryanl | @thunderboltlabs | Sim(foo) user system total real 0.000000 0.000000 0.000000 ( 0.000134) 0.000000 0.000000 0.000000 ( 0.000408) 0.080000 0.000000 0.080000 ( 0.082884)
120. | @bryanl | @thunderboltlabs | Sim(foo) The Ruby compromise
121. | @bryanl | @thunderboltlabs | Sim(foo) To be continued...
122. | @bryanl | @thunderboltlabs | Sim(foo) Thanks! @thunderboltlabs @bryanl