Don't Fear the Threads: Simplify Your Life with JRuby

Slides form my talk at RubyNation 2012

Transcript

1. don’t fear the
   threads
   simplify your life with JRuby
   david.copeland@livingsocial.com
   @davetron5000
   www.naildrivin5.com
   www.awesomecommandlineapps.com
   1
   

   View Slide

2. {
   :me => {
   :tech_lead =>
   “LivingSocial”
   }
   }
   2
   

   View Slide

3. First Assignment
   3
   

   View Slide

4. Write a
   4
   

   View Slide

5. Command
   Line
   App
   Write a
   4
   

   View Slide

6. View Slide

7. Command
   Line
   App
   Write a
   6
   

   View Slide

8. Command
   Line
   App
   Write a
   To charge
   credit cards
   faster
   6
   

   View Slide

9. NO NEW
   HARDWARE
   6
   

   View Slide

10. View Slide

11. THREADS!
    7
    

    View Slide

12. Are we
    maximizing
    our
    resources?
    8
    

    View Slide

13. Are we
    maximizing
    our
    resources?
    9
    

    View Slide

14. What problem
    are we solving?
    10
    

    View Slide

15. Do some I/O
    Compute
    Stuff
    11
    

    View Slide

16. Do some I/O
    12
    

    View Slide

17. Do some I/O
    Can I
    do I/O?
    Read/Write
    Block
    YES
    NO
    13
    

    View Slide

18. Do some I/O
    Block
    14
    

    View Slide

19. Block
    15
    

    View Slide

20. Let someone
    else work
    Block
    15
    

    View Slide

21. Maximize
    Resources
    Block
    16
    

    View Slide

22. View Slide

23. Will our
    code
    be
    17
    

    View Slide

24. Will our
    code
    be easy to write?
    17
    

    View Slide

25. easy to understand?
    Will our
    code
    be
    easy to write?
    17
    

    View Slide

26. easy to understand?
    easy to test?
    Will our
    code
    be
    easy to write?
    17
    

    View Slide

27. Are we
    maximizing
    our
    resources?
    Are we
    maximizing
    our
    resources?
    18
    

    View Slide

28. Run lots of
    processes
    19
    

    View Slide

29. View Slide

30. fork {
    # your code
    }
    20
    

    View Slide

31. easy to
    understand
    21
    

    View Slide

32. doesn’t
    maximize resources
    22
    

    View Slide

33. Parent
    23
    

    View Slide

34. Parent
    Parent’s
    Memory
    23
    

    View Slide

35. Parent
    Parent’s
    Memory
    Child
    fork {
    Parent’s
    Memory
    (copy
    of)
    23
    

    View Slide

36. Event-based I/O
    24
    

    View Slide

37. View Slide

38. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does IO
    some_callback { |results_of_io|
    # the rest of your code that
    # uses the results
    }
    }
    25
    

    View Slide

39. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does IO
    some_callback { |results_of_io|
    # the rest of your code that
    # uses the results
    }
    }
    25
    

    View Slide

40. Need to do
    I/O again?
    26
    

    View Slide

41. View Slide

42. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does SOME of your I/O
    some_callback { |results_of_io|
    # use the results
    # now some more code that does some MORE I/O
    some_other_callback { |results_of_more_io|
    # use the results of THIS I/O
    }
    }
    }
    27
    

    View Slide

43. View Slide

44. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does SOME of your I/O
    some_callback { |results_of_io|
    # use the results
    # now some more code that does some MORE I/O
    some_other_callback { |results_of_more_io|
    # use the results of THIS I/O
    # SERIOUSLY, more I/O?!??!@ How much do you need?
    yet_more_callbacks { |moar_resultz|
    # How many levels deep are we now?!
    }
    }
    }
    }
    28
    

    View Slide

45. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does SOME of your I/O
    some_callback { |results_of_io|
    # use the results
    # now some more code that does some MORE I/O
    some_other_callback { |results_of_more_io|
    # use the results of THIS I/O
    # SERIOUSLY, more I/O?!??!@ How much do you need?
    yet_more_callbacks { |moar_resultz|
    # How many levels deep are we now?!
    }
    }
    }
    }
    28
    

    View Slide

46. NOT easy to
    understand
    29
    

    View Slide

47. kinda
    maximizes resources
    30
    

    View Slide

48. Entire call
    chain must
    be evented
    31
    

    View Slide

49. Not true
    parallelism
    32
    

    View Slide

50. Not true
    parallelism
    unless you run lots
    of processes
    32
    

    View Slide

51. What’s
    parallelism?
    33
    

    View Slide

52. Concurrency
    34
    

    View Slide

53. Parallelism
    35
    

    View Slide

54. Parallelism
    Code running at the same time
    as other code
    35
    

    View Slide

55. Parallelism
    Impossible with only one CPU
    Code running at the same time
    as other code
    35
    

    View Slide

56. Event-based I/O
    36
    

    View Slide

57. NOT easy to
    understand
    kinda
    maximizes resources
    37
    

    View Slide

58. Threads
    38
    

    View Slide

59. View Slide

60. Thread.new {
    # your code
    }
    39
    

    View Slide

61. fork {
    # your code
    }
    40
    

    View Slide

62. Thread.new {
    # your code
    }
    41
    

    View Slide

63. Need to do
    I/O?
    42
    

    View Slide

64. Need to do
    I/O?
    Not a problem
    42
    

    View Slide

65. Need to do
    I/O THREE
    TIMES?
    43
    

    View Slide

66. Need to do
    I/O THREE
    TIMES?
    Not a problem
    43
    

    View Slide

67. What if I need
    to calculate π
    to the
    2,345,123rd
    decimal
    place?
    44
    

    View Slide

68. NOT A
    PROBLEM
    44
    

    View Slide

69. Threads achieve
    true
    parallelism
    45
    

    View Slide

70. easy to
    understand
    46
    

    View Slide

71. maximizes
    resources
    47
    

    View Slide

72. Threads
    don’t work in
    Ruby
    48
    

    View Slide

73. Threads
    don’t work in
    Ruby…right?
    48
    

    View Slide

74. MRI (aka C Ruby)
    49
    

    View Slide

75. MRI (aka C Ruby)
    Thread is an OS thread
    (except in 1.8)
    49
    

    View Slide

76. MRI (aka C Ruby)
    Thread is an OS thread
    GIL :( (except in 1.8)
    49
    

    View Slide

77. MRI (aka C Ruby)
    Thread is an OS thread
    GIL :(
    I/O will cause a context
    switch
    (except in 1.8)
    49
    

    View Slide

78. Rubinius
    50
    

    View Slide

79. Rubinius
    Thread is an OS thread
    50
    

    View Slide

80. Rubinius
    Thread is an OS thread
    No GIL!
    50
    

    View Slide

81. Rubinius
    Thread is an OS thread
    No GIL!
    True parallelism
    50
    

    View Slide

82. JRuby
    51
    

    View Slide

83. JRuby
    Thread is a JVM Thread
    (which is an OS thread)
    51
    

    View Slide

84. JRuby
    Thread is a JVM Thread
    (which is an OS thread)
    Context switch for variety of
    reasons
    51
    

    View Slide

85. JRuby
    Thread is a JVM Thread
    (which is an OS thread)
    Context switch for variety of
    reasons
    True parallelism
    51
    

    View Slide

86. JRuby
    BATTLE TESTED
    51
    

    View Slide

87. So, you’ve
    decided to
    use
    Threads…
    52
    

    View Slide

88. You need to
    know four
    things
    53
    

    View Slide

89. View Slide

90. Start & Manage
    54
    

    View Slide

91. Start & Manage
    (Dealing with) Shared State
    54
    

    View Slide

92. Start & Manage
    (Dealing with) Shared State
    Using Third Party Libraries
    54
    

    View Slide

93. Start & Manage
    (Dealing with) Shared State
    Using Third Party Libraries
    Context Switching
    54
    

    View Slide

94. Start &
    Manage
    Threads
    55
    

    View Slide

95. Chaos
    56
    

    View Slide

96. Thread.new {
    # your code
    }
    Thread.new {
    # moar code
    }
    # etc
    57
    

    View Slide

97. Hand-Roll
    58
    

    View Slide

98. threads = []
    threads << Thread.new {
    # your code
    }
    threads << Thread.new {
    # moar code
    }
    # etc
    threads.each(&:join)
    # All threads have completed
    exit 0
    59
    

    View Slide

99. java.util.concurrent
    60
    

    View Slide

100. import java.util.concurrent # JRuby only
     service =
     Executors.new_fixed_thread_pool(100)
     service.execute {
     # your code
     }
     service.execute {
     # some other code
     }
     61
     

     View Slide

101. View Slide

102. View Slide

103. View Slide

104. ExecutorService
     64
     

     View Slide

105. ExecutorService
     65
     

     View Slide

106. ExecutorService
     execute()
     65
     

     View Slide

107. ExecutorService
     execute()
     shutdown()
     65
     

     View Slide

108. ExecutorService
     execute()
     shutdown()
     await_termination()
     65
     

     View Slide

109. ExecutorService
     execute()
     shutdown()
     await_termination()
     shutdown_now()
     65
     

     View Slide

110. service = Executors.new_fixed_thread_pool(10)
     tcp_erver = TCPServer.new("127.0.0.1",8080)
     loop do {
     s = tcp_server.accept
     service.execute {
     s.puts calculate_pi()
     s.close
     }
     }
     66
     

     View Slide

111. service = Executors.new_fixed_thread_pool(10)
     tcp_server = TCPServer.new("127.0.0.1",8080)
     Signal.trap('SIGINT') {
     service.shutdown
     }
     loop do {
     s = tcp_server.accept
     service.execute {
     s.puts calculate_pi()
     s.close
     }
     break if service.is_shutdown
     }
     67
     

     View Slide

112. service = Executors.new_fixed_thread_pool(10)
     tcp_server = TCPServer.new("127.0.0.1",8080)
     Signal.trap('SIGINT') {
     service.shutdown
     }
     loop do {
     s = tcp_server.accept
     service.execute {
     s.puts calculate_pi()
     s.close
     }
     break if service.is_shutdown
     }
     service.await_termination(10,TimeUnit.SECONDS)
     service.shutdown_now
     68
     

     View Slide

113. So much more
     69
     

     View Slide

114. ScheduledExecutorService
     So much more
     69
     

     View Slide

115. ScheduledExecutorService
     So much more
     ThreadFactory
     69
     

     View Slide

116. Read the
     javadocs
     ScheduledExecutorService
     So much more
     ThreadFactory
     69
     

     View Slide

117. Shared State
     70
     

     View Slide

118. results = []
     Thread.new {
     results << some_result()
     }
     Thread.new {
     results << other_result()
     }
     71
     

     View Slide

119. mutex = Mutex.new
     results = []
     Thread.new {
     mutex.synchronize {
     results << some_result()
     }
     }
     Thread.new {
     mutex.synchronize {
     results << other_result()
     }
     }
     72
     

     View Slide

120. Avoid explicit
     locking/sync
     73
     

     View Slide

121. results = ConcurrentLinkedQueue.new
     Thread.new {
     results.offer(some_result())
     }
     Thread.new {
     results.offer(some_result())
     }
     74
     

     View Slide

122. import java.util.concurrent.atomic
     flag = AtomicBoolean.new(true)
     Thread.new {
     if flag.get {
     flag.set(false)
     }
     }
     Thread.new {
     flag.set(true)
     }
     75
     

     View Slide

123. So much more
     76
     

     View Slide

124. ConcurrentHashMap
     So much more
     76
     

     View Slide

125. ConcurrentHashMap
     So much more
     AtomicReference
     76
     

     View Slide

126. Read the
     javadocs
     ConcurrentHashMap
     So much more
     AtomicReference
     76
     

     View Slide

127. Third Party
     Libraries
     77
     

     View Slide

128. Are they
     thread safe?
     78
     

     View Slide

129. Probably
     79
     

     View Slide

130. Probably
     …but to be sure
     79
     

     View Slide

131. variables
     80
     

     View Slide

132. variables
     Global
     80
     

     View Slide

133. variables
     Global
     Class
     80
     

     View Slide

134. Most are fine
     81
     

     View Slide

135. Context Switching
     82
     

     View Slide

136. Amdahl’s
     Law
     83
     

     View Slide

137. Performance Gains
     Increase in # of Threads
     Speedup
     84
     

     View Slide

138. Performance Gains with Context Switching
     Increase in # of Threads
     Speedup
     Cost of Context
     Switching Takes
     Over
     85
     

     View Slide

139. Thread Pools
     86
     

     View Slide

140. Worker
     Worker
     Worker
     Worker
     Thread
     Thread
     Thread
     Thread Pool
     Worker
     Worker
     Worker
     Worker
     87
     

     View Slide

141. Worker
     Worker
     Worker
     Worker
     Thread
     Thread
     Thread
     Thread Pool
     Worker
     Worker
     Worker
     Worker
     87
     

     View Slide

142. Worker
     Worker
     Worker
     Worker
     Thread
     Thread
     Thread
     Thread Pool
     Worker
     Worker
     Worker
     87
     

     View Slide

143. Thread.new {
     # your code
     }
     88
     

     View Slide

144. service.execute {
     # your code
     }
     89
     

     View Slide

145. Not usually a
     concern
     90
     

     View Slide

146. Always use
     Threads, right?
     91
     

     View Slide

147. Thread
     UNsafe
     Libraries
     92
     

     View Slide

148. Evented
     93
     

     View Slide

149. Simple, I/O
     bound task
     94
     

     View Slide

150. Evented
     95
     

     View Slide

151. Cannot use
     JRuby
     96
     

     View Slide

152. Evented
     97
     

     View Slide

153. Otherwise,
     Threads will
     simplify your code
     and maximize
     your resources
     98
     

     View Slide

154. Semantic
     layers on top
     of evented I/O
     99
     

     View Slide

155. View Slide

156. You’re using
     Threads
     100
     

     View Slide

157. You’re using
     Threads
     (in degenerate form)
     100
     

     View Slide

158. Exercises
     101
     

     View Slide

159. Echo Server
     •Listen on a port
     •Respond to each request in a
     new Thread
     •Extra Credit: Record stats on
     requests in a shared data
     structure
     102
     

     View Slide

160. Connection Pool
     •Allow N clients to access X shared
     instances of, say, Redis (where N > X)
     •Clients “check out” a connection and
     get exclusive access
     •Clients “check in” when done
     •Instances get re-used
     103
     

     View Slide

161. THANKS!
     david.copeland@livingsocial.com
     @davetron5000
     www.naildrivin5.com
     www.awesomecommandlineapps.com
     104
     

     View Slide