Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Arrrr Camp 2012: Simulating the World with Ruby

Arrrr Camp 2012: Simulating the World with Ruby

Ba4b0b302231bffb0f685bbef25db0d6?s=128

Bryan Liles

October 05, 2012
Tweet

More Decks by Bryan Liles

Other Decks in Programming

Transcript

  1. ®

  2. The (Ruby) Sims (part 2)

  3. | @bryanl | @thunderboltlabs | Sim(foo) Introduction

  4. | @bryanl | @thunderboltlabs | Sim(foo) @bryanl

  5. | @bryanl | @thunderboltlabs | Sim(foo)

  6. | @bryanl | @thunderboltlabs | Sim(foo)

  7. | @bryanl | @thunderboltlabs | Sim(foo)

  8. | @bryanl | @thunderboltlabs | Sim(foo)

  9. | @bryanl | @thunderboltlabs | Sim(foo)

  10. | @bryanl | @thunderboltlabs | Sim(foo)

  11. | @bryanl | @thunderboltlabs | Sim(foo) CootieSim2012 v2.0

  12. | @bryanl | @thunderboltlabs | Sim(foo) Why RUBY?

  13. | @bryanl | @thunderboltlabs | Sim(foo) Stochastic vs Deterministic

  14. | @bryanl | @thunderboltlabs | Sim(foo) Define inputs Compute Analyze

    output
  15. | @bryanl | @thunderboltlabs | Sim(foo) SIMULATION_LENGTH = 100 PEOPLE_COUNT

    = 100 def create_people(people_count); end def report; end def simulate_to(count) 1.upto(count) {|i| yield i } end people = create_people(PEOPLE_COUNT) simulate_to(SIMULATION_LENGTH) do |iteration| # mish and mash people end report
  16. | @bryanl | @thunderboltlabs | Sim(foo) class Simulator def initialize(length,

    count) # ... end end s = Simulator.new 3650, 100 Define inputs
  17. | @bryanl | @thunderboltlabs | Sim(foo) class Simulator def simulate(iteration)

    create_pairs(iteration) transmit(iteration) progress_infections(iteration) end end Compute
  18. | @bryanl | @thunderboltlabs | Sim(foo) s = Simulator.new 3650,

    100 s.run s.infected_people.each do |p| i = p.infections.to_a d = p.diseases.to_a puts "#{p} #{i} #{d}" end Analyze output
  19. None
  20. | @bryanl | @thunderboltlabs | Sim(foo) Breakdown

  21. | @bryanl | @thunderboltlabs | Sim(foo) What’s in a day?

  22. | @bryanl | @thunderboltlabs | Sim(foo) Create new pair

  23. | @bryanl | @thunderboltlabs | Sim(foo) def create_pairs(iteration) @pairs +=

    people_not_in_pairs(iteration) .shuffle .each_slice(2) .map{|x| Pair.new iteration, x} end
  24. | @bryanl | @thunderboltlabs | Sim(foo) Transmit cooties

  25. | @bryanl | @thunderboltlabs | Sim(foo) class Simulation def transmit(iteration)

    current_pairs(iteration).each{|pair| pair.contact(iteration)} end end class Pair def contact(iteration) @alpha.infect(iteration, @beta) @beta.infect(iteration, @alpha) end end class Person def infect(iteration, person); end def acquire(infection); end end
  26. | @bryanl | @thunderboltlabs | Sim(foo) Track infection progress

  27. | @bryanl | @thunderboltlabs | Sim(foo) class Person def progress_infections(iteration)

    @infections.each do |infection| if rand <= infection.chance_to_acquire_disease unless has_disease? infection.disease @diseases << infection.disease.new(iteration) end end end end end
  28. | @bryanl | @thunderboltlabs | Sim(foo) ... This isn’t going

    to be fast
  29. | @bryanl | @thunderboltlabs | Sim(foo) 28.23s user 0.10s system

    99% cpu 28.333 total
  30. | @bryanl | @thunderboltlabs | Sim(foo) but why?

  31. | @bryanl | @thunderboltlabs | Sim(foo) %self total self wait

    child calls name 30.32 386.26 117.53 0.00 268.72 21904 Array#each 28.08 180.05 108.88 0.00 71.18 78738195 Comparable#between? 21.74 84.29 84.29 0.00 0.00 78738195 Pair#duration 18.36 71.18 71.18 0.00 0.00 157476390 Fixnum#<=> 0.29 2.11 1.11 0.00 1.06 414957 Set#each 0.13 1.15 0.50 0.00 0.65 365100 Person#infected? 0.13 2.36 0.50 0.00 1.86 365000 Person#infect 0.12 0.65 0.48 0.00 0.17 365100 Set#empty?
  32. | @bryanl | @thunderboltlabs | Sim(foo) module Enumerable # create

    a thread and execute in parallel def parallel_each(&block) # create thread yield # cleanup end end
  33. | @bryanl | @thunderboltlabs | Sim(foo)

  34. | @bryanl | @thunderboltlabs | Sim(foo) 100 people

  35. | @bryanl | @thunderboltlabs | Sim(foo) 3650 iterations

  36. | @bryanl | @thunderboltlabs | Sim(foo) 365,000 “actions”

  37. | @bryanl | @thunderboltlabs | Sim(foo)

  38. | @bryanl | @thunderboltlabs | Sim(foo) 3,650,000 @ 1,000 36,500,000

    @ 10,000 365,000,000 @ 100,000 3,650,000,000 @ 1,000,000
  39. | @bryanl | @thunderboltlabs | Sim(foo) How do we make

    it faster?
  40. | @bryanl | @thunderboltlabs | Sim(foo) Get a faster CPU?

  41. | @bryanl | @thunderboltlabs | Sim(foo) 1. Run less code

    2. Do more things at once
  42. | @bryanl | @thunderboltlabs | Sim(foo) Can’t really make it

    “faster”
  43. | @bryanl | @thunderboltlabs | Sim(foo) MRI 1.9

  44. | @bryanl | @thunderboltlabs | Sim(foo)

  45. | @bryanl | @thunderboltlabs | Sim(foo) Thread.new

  46. | @bryanl | @thunderboltlabs | Sim(foo)

  47. | @bryanl | @thunderboltlabs | Sim(foo) JRuby

  48. | @bryanl | @thunderboltlabs | Sim(foo)

  49. | @bryanl | @thunderboltlabs | Sim(foo) Thread.new

  50. | @bryanl | @thunderboltlabs | Sim(foo)

  51. | @bryanl | @thunderboltlabs | Sim(foo) JRuby’s sweetspot is Hotspot

  52. | @bryanl | @thunderboltlabs | Sim(foo) 11.34s user 0.37s system

    116% cpu 10.064 total
  53. | @bryanl | @thunderboltlabs | Sim(foo) Doing more than one

    thing at once with MRI
  54. | @bryanl | @thunderboltlabs | Sim(foo) fork

  55. | @bryanl | @thunderboltlabs | Sim(foo) external processes

  56. | @bryanl | @thunderboltlabs | Sim(foo) statistics

  57. | @bryanl | @thunderboltlabs | Sim(foo) mean median variance standard

    deviation σ
  58. | @bryanl | @thunderboltlabs | Sim(foo) How do i randomly

    pick an integer from 1-6?
  59. | @bryanl | @thunderboltlabs | Sim(foo) rand(1..6) rand(6)+1

  60. | @bryanl | @thunderboltlabs | Sim(foo) gem install vose

  61. | @bryanl | @thunderboltlabs | Sim(foo) v = Vose::AliasMethod.new [1,1,1,1,1,1]

    (1..10000).map{v.next}.inject(Hash.new{0}){ |m,n| m[n]+=1; m} (1..10000).map{rand(1..6)} .inject(Hash.new{0}){|m,n| m[n]+=1; m} VS
  62. | @bryanl | @thunderboltlabs | Sim(foo) What if the die

    is weighted?
  63. | @bryanl | @thunderboltlabs | Sim(foo) v = Vose::AliasMethod.new [1.2,1,1,1,1,1]

    (1..10000).map{v.next}.inject(Hash.new{0}){ |m,n| m[n]+=1; m} ????? VS
  64. | @bryanl | @thunderboltlabs | Sim(foo) why is this important?

  65. | @bryanl | @thunderboltlabs | Sim(foo)

  66. | @bryanl | @thunderboltlabs | Sim(foo)

  67. | @bryanl | @thunderboltlabs | Sim(foo) graphing

  68. | @bryanl | @thunderboltlabs | Sim(foo) Trends

  69. None
  70. None
  71. | @bryanl | @thunderboltlabs | Sim(foo) library(ggplot2) runtime.csv <- read.csv("runtime.csv",

    header=T) mygg<- qplot(iteration, time, data=runtime.csv); mygg<- mygg + stat_smooth(method="lm") ggsave("runtime.png") dev.off()
  72. | @bryanl | @thunderboltlabs | Sim(foo) library(ggplot2) runtime.csv <- read.csv("runtime.csv",

    header=T) mygg<- qplot(iteration, time, data=runtime.csv); mygg<- mygg + stat_smooth(method="lm") ggsave("runtime.png") dev.off() R
  73. | @bryanl | @thunderboltlabs | Sim(foo) Visualization

  74. | @bryanl | @thunderboltlabs | Sim(foo)

  75. | @bryanl | @thunderboltlabs | Sim(foo) digraph g { "person59"

    -> "person5"; "person23" -> "person18"; "person72" -> "person23"; "person78" -> "person44"; "person78" -> "person59"; "person59" -> "person72"; "person1" -> "person78"; "person83" -> "person81"; "person59" -> "person83"; "person59" -> "person92"; };
  76. | @bryanl | @thunderboltlabs | Sim(foo) digraph g { "person59"

    -> "person5"; "person23" -> "person18"; "person72" -> "person23"; "person78" -> "person44"; "person78" -> "person59"; "person59" -> "person72"; "person1" -> "person78"; "person83" -> "person81"; "person59" -> "person83"; "person59" -> "person92"; }; Graphviz
  77. | @bryanl | @thunderboltlabs | Sim(foo) Building a simulator (v1.0)

  78. | @bryanl | @thunderboltlabs | Sim(foo)

  79. | @bryanl | @thunderboltlabs | Sim(foo)

  80. | @bryanl | @thunderboltlabs | Sim(foo)

  81. | @bryanl | @thunderboltlabs | Sim(foo) ?!

  82. | @bryanl | @thunderboltlabs | Sim(foo) Shared Cooties?

  83. | @bryanl | @thunderboltlabs | Sim(foo) Making it fast

  84. | @bryanl | @thunderboltlabs | Sim(foo) Be kind to MRI’s

    GC
  85. | @bryanl | @thunderboltlabs | Sim(foo) Run more than one

    process
  86. | @bryanl | @thunderboltlabs | Sim(foo) Look outside of MRI

  87. | @bryanl | @thunderboltlabs | Sim(foo) irb(main):003:0> rand 1..7 TypeError:

    can't convert Range into Integer " from org/jruby/RubyKernel.java:1580:in `rand' " from (irb):3:in `evaluate' " from org/jruby/RubyKernel.java:1070:in `eval' " from org/jruby/RubyKernel.java:1395:in `loop' " from org/jruby/RubyKernel.java:1178:in `catch' " from org/jruby/RubyKernel.java:1178:in `catch' " from /Users/bryan/.rbenv/versions/jruby-1.7.0-preview2/bin/irb:13:in `(root)' JRuby 1.7 pre2
  88. | @bryanl | @thunderboltlabs | Sim(foo) irb(main):001:0> rand 1..7 =>

    3 MRI 1.9.3
  89. | @bryanl | @thunderboltlabs | Sim(foo) other languages?

  90. | @bryanl | @thunderboltlabs | Sim(foo) Building a simulator (v2.0)

  91. | @bryanl | @thunderboltlabs | Sim(foo) #!/usr/bin/env ruby require 'benchmark'

    n,m = [1_000_000, 100] Benchmark.bm do |x| x.report do m.times do a = []; n.times{|i| a[i] = i}; n.times{ j=a[rand(n)]; j*j} end end x.report do m.times do h = {}; n.times{|i| h[i] = i}; n.times{j=h[rand(n)]; j*j} end end end
  92. | @bryanl | @thunderboltlabs | Sim(foo) -> ❸ % ruby

    big_arrays.rb user system total real 41.920000 0.330000 42.250000 ( 42.258538) 138.270000 2.860000 141.130000 (141.366728) [~/Development/talks/arrrrcamp2012/src/lib] -> ♢ % ruby big_arrays.rb user system total real 88.320000 0.570000 88.890000 ( 41.623000) 191.970000 0.460000 192.430000 ( 73.113000) [~/Development/talks/arrrrcamp2012/src/lib]
  93. | @bryanl | @thunderboltlabs | Sim(foo) -> ❸ % ruby

    discrete.rb 36500 1000 creating 1000 people inoculating 50% of the population circle circle dot dot. now you have a cootie shot starting simulation 3650 7300 10950 14600 18250 21900 25550 29200 32850 36500 runtime: 129.49060773849487 Infected: 479 Noninfected: 521
  94. | @bryanl | @thunderboltlabs | Sim(foo) -> ♢ % ruby

    discrete.rb 36500 1000 creating 1000 people inoculating 50% of the population circle circle dot dot. now you have a cootie shot starting simulation 3650 7300 10950 14600 18250 21900 25550 29200 32850 36500 runtime: 129.88899993896484 Infected: 526 Noninfected: 474
  95. | @bryanl | @thunderboltlabs | Sim(foo) Rails won’t scale

  96. | @bryanl | @thunderboltlabs | Sim(foo) Ruby won’t scale

  97. | @bryanl | @thunderboltlabs | Sim(foo) *this* won’t scale

  98. | @bryanl | @thunderboltlabs | Sim(foo)

  99. | @bryanl | @thunderboltlabs | Sim(foo) Ready to pair?

  100. | @bryanl | @thunderboltlabs | Sim(foo) Ready to pair?

  101. | @bryanl | @thunderboltlabs | Sim(foo) Ready to pair?

  102. | @bryanl | @thunderboltlabs | Sim(foo) Will something happen?

  103. | @bryanl | @thunderboltlabs | Sim(foo) Will something happen?

  104. | @bryanl | @thunderboltlabs | Sim(foo) 1000 people 636 pairs

    1636 choices ___________ +
  105. | @bryanl | @thunderboltlabs | Sim(foo) class Event def schedule;

    end def run # do stuff end end e = Event.new e.run e.schedule
  106. | @bryanl | @thunderboltlabs | Sim(foo) class PairEvent < Event

    def initialize(person) @person = person end def run # add person to “available # to pair pool” end end
  107. | @bryanl | @thunderboltlabs | Sim(foo) class ContactEvent < Event

    def schedule # create future contact event end def run # perform a contact end end
  108. | @bryanl | @thunderboltlabs | Sim(foo) how do we keep

    track of events?
  109. | @bryanl | @thunderboltlabs | Sim(foo) event_pool = [] event_pool

    << event def do_step(current_step) current_events = event_pool.find_all do |event| event.on?(current_step) end current_events.each do |event| event.run event_pool << event.schedule end end
  110. | @bryanl | @thunderboltlabs | Sim(foo) event_pool.insert(x, event.schedule)

  111. | @bryanl | @thunderboltlabs | Sim(foo) n = 100_000 Benchmark.bm

    do |x| x.report {a = [1] * 10; n.times{|i| a << i}} x.report {a = [1] * 10; n.times{|i| a.insert(1, i)}} end
  112. | @bryanl | @thunderboltlabs | Sim(foo) user system total real

    0.010000 0.000000 0.010000 ( 0.010555) 1.670000 0.000000 1.670000 ( 1.667457)
  113. | @bryanl | @thunderboltlabs | Sim(foo) we can do better

  114. | @bryanl | @thunderboltlabs | Sim(foo) gem install algorithms

  115. | @bryanl | @thunderboltlabs | Sim(foo) event_pool = Containers::PriorityQueue.new event_pool.push

    event, (length-current_step) while (event_pool.size > 0) event = event_pool.pop event.run(current_step) event.schedule(event_pool) end
  116. | @bryanl | @thunderboltlabs | Sim(foo) a.k.a. Discrete Event Simulation

  117. | @bryanl | @thunderboltlabs | Sim(foo) but wait...

  118. | @bryanl | @thunderboltlabs | Sim(foo) n = 1000 Benchmark.bm

    do |x| x.report {a = [1] * 10; n.times{|i| a << i}} x.report {a = [1] * 10; n.times{|i| a.insert(1, i)}} x.report do pq = PriorityQueue.new n.times{|i| pq.push(i, 2)} } end end
  119. | @bryanl | @thunderboltlabs | Sim(foo) user system total real

    0.000000 0.000000 0.000000 ( 0.000134) 0.000000 0.000000 0.000000 ( 0.000408) 0.080000 0.000000 0.080000 ( 0.082884)
  120. | @bryanl | @thunderboltlabs | Sim(foo) The Ruby compromise

  121. | @bryanl | @thunderboltlabs | Sim(foo) To be continued...

  122. | @bryanl | @thunderboltlabs | Sim(foo) Thanks! @thunderboltlabs @bryanl