Don't Fear the Threads: Simplify Your Life with JRuby

Don't Fear the Threads: Simplify Your Life with JRuby

Slides form my talk at RubyNation 2012

F74253f4a099258870157426b4cdb2dc?s=128

David Copeland

March 23, 2012
Tweet

Transcript

  1. don’t fear the threads simplify your life with JRuby david.copeland@livingsocial.com

    @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com 1
  2. { :me => { :tech_lead => “LivingSocial” } } 2

  3. First Assignment 3

  4. Write a 4

  5. Command Line App Write a 4

  6. 5

  7. Command Line App Write a 6

  8. Command Line App Write a To charge credit cards faster

    6
  9. NO NEW HARDWARE 6

  10. 7

  11. THREADS! 7

  12. Are we maximizing our resources? 8

  13. Are we maximizing our resources? 9

  14. What problem are we solving? 10

  15. Do some I/O Compute Stuff 11

  16. Do some I/O 12

  17. Do some I/O Can I do I/O? Read/Write Block YES

    NO 13
  18. Do some I/O Block 14

  19. Block 15

  20. Let someone else work Block 15

  21. Maximize Resources Block 16

  22. 17

  23. Will our code be 17

  24. Will our code be easy to write? 17

  25. easy to understand? Will our code be easy to write?

    17
  26. easy to understand? easy to test? Will our code be

    easy to write? 17
  27. Are we maximizing our resources? Are we maximizing our resources?

    18
  28. Run lots of processes 19

  29. 20

  30. fork { # your code } 20

  31. easy to understand 21

  32. doesn’t maximize resources 22

  33. Parent 23

  34. Parent Parent’s Memory 23

  35. Parent Parent’s Memory Child fork { Parent’s Memory (copy of)

    23
  36. Event-based I/O 24

  37. 25

  38. require 'em-http' include EM::HttpRequest EM.run { # some of your

    code that does IO some_callback { |results_of_io| # the rest of your code that # uses the results } } 25
  39. require 'em-http' include EM::HttpRequest EM.run { # some of your

    code that does IO some_callback { |results_of_io| # the rest of your code that # uses the results } } 25
  40. Need to do I/O again? 26

  41. 27

  42. require 'em-http' include EM::HttpRequest EM.run { # some of your

    code that does SOME of your I/O some_callback { |results_of_io| # use the results # now some more code that does some MORE I/O some_other_callback { |results_of_more_io| # use the results of THIS I/O } } } 27
  43. 28

  44. require 'em-http' include EM::HttpRequest EM.run { # some of your

    code that does SOME of your I/O some_callback { |results_of_io| # use the results # now some more code that does some MORE I/O some_other_callback { |results_of_more_io| # use the results of THIS I/O # SERIOUSLY, more I/O?!??!@ How much do you need? yet_more_callbacks { |moar_resultz| # How many levels deep are we now?! } } } } 28
  45. require 'em-http' include EM::HttpRequest EM.run { # some of your

    code that does SOME of your I/O some_callback { |results_of_io| # use the results # now some more code that does some MORE I/O some_other_callback { |results_of_more_io| # use the results of THIS I/O # SERIOUSLY, more I/O?!??!@ How much do you need? yet_more_callbacks { |moar_resultz| # How many levels deep are we now?! } } } } 28
  46. NOT easy to understand 29

  47. kinda maximizes resources 30

  48. Entire call chain must be evented 31

  49. Not true parallelism 32

  50. Not true parallelism unless you run lots of processes 32

  51. What’s parallelism? 33

  52. Concurrency 34

  53. Parallelism 35

  54. Parallelism Code running at the same time as other code

    35
  55. Parallelism Impossible with only one CPU Code running at the

    same time as other code 35
  56. Event-based I/O 36

  57. NOT easy to understand kinda maximizes resources 37

  58. Threads 38

  59. 39

  60. Thread.new { # your code } 39

  61. fork { # your code } 40

  62. Thread.new { # your code } 41

  63. Need to do I/O? 42

  64. Need to do I/O? Not a problem 42

  65. Need to do I/O THREE TIMES? 43

  66. Need to do I/O THREE TIMES? Not a problem 43

  67. What if I need to calculate π to the 2,345,123rd

    decimal place? 44
  68. NOT A PROBLEM 44

  69. Threads achieve true parallelism 45

  70. easy to understand 46

  71. maximizes resources 47

  72. Threads don’t work in Ruby 48

  73. Threads don’t work in Ruby…right? 48

  74. MRI (aka C Ruby) 49

  75. MRI (aka C Ruby) Thread is an OS thread (except

    in 1.8) 49
  76. MRI (aka C Ruby) Thread is an OS thread GIL

    :( (except in 1.8) 49
  77. MRI (aka C Ruby) Thread is an OS thread GIL

    :( I/O will cause a context switch (except in 1.8) 49
  78. Rubinius 50

  79. Rubinius Thread is an OS thread 50

  80. Rubinius Thread is an OS thread No GIL! 50

  81. Rubinius Thread is an OS thread No GIL! True parallelism

    50
  82. JRuby 51

  83. JRuby Thread is a JVM Thread (which is an OS

    thread) 51
  84. JRuby Thread is a JVM Thread (which is an OS

    thread) Context switch for variety of reasons 51
  85. JRuby Thread is a JVM Thread (which is an OS

    thread) Context switch for variety of reasons True parallelism 51
  86. JRuby BATTLE TESTED 51

  87. So, you’ve decided to use Threads… 52

  88. You need to know four things 53

  89. 54

  90. Start & Manage 54

  91. Start & Manage (Dealing with) Shared State 54

  92. Start & Manage (Dealing with) Shared State Using Third Party

    Libraries 54
  93. Start & Manage (Dealing with) Shared State Using Third Party

    Libraries Context Switching 54
  94. Start & Manage Threads 55

  95. Chaos 56

  96. Thread.new { # your code } Thread.new { # moar

    code } # etc 57
  97. Hand-Roll 58

  98. threads = [] threads << Thread.new { # your code

    } threads << Thread.new { # moar code } # etc threads.each(&:join) # All threads have completed exit 0 59
  99. java.util.concurrent 60

  100. import java.util.concurrent # JRuby only service = Executors.new_fixed_thread_pool(100) service.execute {

    # your code } service.execute { # some other code } 61
  101. 62

  102. 63

  103. <T> 63

  104. ExecutorService 64

  105. ExecutorService 65

  106. ExecutorService execute() 65

  107. ExecutorService execute() shutdown() 65

  108. ExecutorService execute() shutdown() await_termination() 65

  109. ExecutorService execute() shutdown() await_termination() shutdown_now() 65

  110. service = Executors.new_fixed_thread_pool(10) tcp_erver = TCPServer.new("127.0.0.1",8080) loop do { s

    = tcp_server.accept service.execute { s.puts calculate_pi() s.close } } 66
  111. service = Executors.new_fixed_thread_pool(10) tcp_server = TCPServer.new("127.0.0.1",8080) Signal.trap('SIGINT') { service.shutdown }

    loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } break if service.is_shutdown } 67
  112. service = Executors.new_fixed_thread_pool(10) tcp_server = TCPServer.new("127.0.0.1",8080) Signal.trap('SIGINT') { service.shutdown }

    loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } break if service.is_shutdown } service.await_termination(10,TimeUnit.SECONDS) service.shutdown_now 68
  113. So much more 69

  114. ScheduledExecutorService So much more 69

  115. ScheduledExecutorService So much more ThreadFactory 69

  116. Read the javadocs ScheduledExecutorService So much more ThreadFactory 69

  117. Shared State 70

  118. results = [] Thread.new { results << some_result() } Thread.new

    { results << other_result() } 71
  119. mutex = Mutex.new results = [] Thread.new { mutex.synchronize {

    results << some_result() } } Thread.new { mutex.synchronize { results << other_result() } } 72
  120. Avoid explicit locking/sync 73

  121. results = ConcurrentLinkedQueue.new Thread.new { results.offer(some_result()) } Thread.new { results.offer(some_result())

    } 74
  122. import java.util.concurrent.atomic flag = AtomicBoolean.new(true) Thread.new { if flag.get {

    flag.set(false) } } Thread.new { flag.set(true) } 75
  123. So much more 76

  124. ConcurrentHashMap So much more 76

  125. ConcurrentHashMap So much more AtomicReference 76

  126. Read the javadocs ConcurrentHashMap So much more AtomicReference 76

  127. Third Party Libraries 77

  128. Are they thread safe? 78

  129. Probably 79

  130. Probably …but to be sure 79

  131. variables 80

  132. variables Global 80

  133. variables Global Class 80

  134. Most are fine 81

  135. Context Switching 82

  136. Amdahl’s Law 83

  137. Performance Gains Increase in # of Threads Speedup 84

  138. Performance Gains with Context Switching Increase in # of Threads

    Speedup Cost of Context Switching Takes Over 85
  139. Thread Pools 86

  140. Worker Worker Worker Worker Thread Thread Thread Thread Pool Worker

    Worker Worker Worker 87
  141. Worker Worker Worker Worker Thread Thread Thread Thread Pool Worker

    Worker Worker Worker 87
  142. Worker Worker Worker Worker Thread Thread Thread Thread Pool Worker

    Worker Worker 87
  143. Thread.new { # your code } 88

  144. service.execute { # your code } 89

  145. Not usually a concern 90

  146. Always use Threads, right? 91

  147. Thread UNsafe Libraries 92

  148. Evented 93

  149. Simple, I/O bound task 94

  150. Evented 95

  151. Cannot use JRuby 96

  152. Evented 97

  153. Otherwise, Threads will simplify your code and maximize your resources

    98
  154. Semantic layers on top of evented I/O 99

  155. 100

  156. You’re using Threads 100

  157. You’re using Threads (in degenerate form) 100

  158. Exercises 101

  159. Echo Server •Listen on a port •Respond to each request

    in a new Thread •Extra Credit: Record stats on requests in a shared data structure 102
  160. Connection Pool •Allow N clients to access X shared instances

    of, say, Redis (where N > X) •Clients “check out” a connection and get exclusive access •Clients “check in” when done •Instances get re-used 103
  161. THANKS! david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com 104