Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Don't Fear the Threads: Simplify Your Life with JRuby

Don't Fear the Threads: Simplify Your Life with JRuby

Slides form my talk at RubyNation 2012

David Copeland

March 23, 2012
Tweet

More Decks by David Copeland

Other Decks in Programming

Transcript

  1. don’t fear the
    threads
    simplify your life with JRuby
    [email protected]
    @davetron5000
    www.naildrivin5.com
    www.awesomecommandlineapps.com
    1

    View Slide

  2. {
    :me => {
    :tech_lead =>
    “LivingSocial”
    }
    }
    2

    View Slide

  3. First Assignment
    3

    View Slide

  4. Write a
    4

    View Slide

  5. Command
    Line
    App
    Write a
    4

    View Slide

  6. 5

    View Slide

  7. Command
    Line
    App
    Write a
    6

    View Slide

  8. Command
    Line
    App
    Write a
    To charge
    credit cards
    faster
    6

    View Slide

  9. NO NEW
    HARDWARE
    6

    View Slide

  10. 7

    View Slide

  11. THREADS!
    7

    View Slide

  12. Are we
    maximizing
    our
    resources?
    8

    View Slide

  13. Are we
    maximizing
    our
    resources?
    9

    View Slide

  14. What problem
    are we solving?
    10

    View Slide

  15. Do some I/O
    Compute
    Stuff
    11

    View Slide

  16. Do some I/O
    12

    View Slide

  17. Do some I/O
    Can I
    do I/O?
    Read/Write
    Block
    YES
    NO
    13

    View Slide

  18. Do some I/O
    Block
    14

    View Slide

  19. Block
    15

    View Slide

  20. Let someone
    else work
    Block
    15

    View Slide

  21. Maximize
    Resources
    Block
    16

    View Slide

  22. 17

    View Slide

  23. Will our
    code
    be
    17

    View Slide

  24. Will our
    code
    be easy to write?
    17

    View Slide

  25. easy to understand?
    Will our
    code
    be
    easy to write?
    17

    View Slide

  26. easy to understand?
    easy to test?
    Will our
    code
    be
    easy to write?
    17

    View Slide

  27. Are we
    maximizing
    our
    resources?
    Are we
    maximizing
    our
    resources?
    18

    View Slide

  28. Run lots of
    processes
    19

    View Slide

  29. 20

    View Slide

  30. fork {
    # your code
    }
    20

    View Slide

  31. easy to
    understand
    21

    View Slide

  32. doesn’t
    maximize resources
    22

    View Slide

  33. Parent
    23

    View Slide

  34. Parent
    Parent’s
    Memory
    23

    View Slide

  35. Parent
    Parent’s
    Memory
    Child
    fork {
    Parent’s
    Memory
    (copy
    of)
    23

    View Slide

  36. Event-based I/O
    24

    View Slide

  37. 25

    View Slide

  38. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does IO
    some_callback { |results_of_io|
    # the rest of your code that
    # uses the results
    }
    }
    25

    View Slide

  39. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does IO
    some_callback { |results_of_io|
    # the rest of your code that
    # uses the results
    }
    }
    25

    View Slide

  40. Need to do
    I/O again?
    26

    View Slide

  41. 27

    View Slide

  42. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does SOME of your I/O
    some_callback { |results_of_io|
    # use the results
    # now some more code that does some MORE I/O
    some_other_callback { |results_of_more_io|
    # use the results of THIS I/O
    }
    }
    }
    27

    View Slide

  43. 28

    View Slide

  44. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does SOME of your I/O
    some_callback { |results_of_io|
    # use the results
    # now some more code that does some MORE I/O
    some_other_callback { |results_of_more_io|
    # use the results of THIS I/O
    # SERIOUSLY, more I/O?!??!@ How much do you need?
    yet_more_callbacks { |moar_resultz|
    # How many levels deep are we now?!
    }
    }
    }
    }
    28

    View Slide

  45. require 'em-http'
    include EM::HttpRequest
    EM.run {
    # some of your code that does SOME of your I/O
    some_callback { |results_of_io|
    # use the results
    # now some more code that does some MORE I/O
    some_other_callback { |results_of_more_io|
    # use the results of THIS I/O
    # SERIOUSLY, more I/O?!??!@ How much do you need?
    yet_more_callbacks { |moar_resultz|
    # How many levels deep are we now?!
    }
    }
    }
    }
    28

    View Slide

  46. NOT easy to
    understand
    29

    View Slide

  47. kinda
    maximizes resources
    30

    View Slide

  48. Entire call
    chain must
    be evented
    31

    View Slide

  49. Not true
    parallelism
    32

    View Slide

  50. Not true
    parallelism
    unless you run lots
    of processes
    32

    View Slide

  51. What’s
    parallelism?
    33

    View Slide

  52. Concurrency
    34

    View Slide

  53. Parallelism
    35

    View Slide

  54. Parallelism
    Code running at the same time
    as other code
    35

    View Slide

  55. Parallelism
    Impossible with only one CPU
    Code running at the same time
    as other code
    35

    View Slide

  56. Event-based I/O
    36

    View Slide

  57. NOT easy to
    understand
    kinda
    maximizes resources
    37

    View Slide

  58. Threads
    38

    View Slide

  59. 39

    View Slide

  60. Thread.new {
    # your code
    }
    39

    View Slide

  61. fork {
    # your code
    }
    40

    View Slide

  62. Thread.new {
    # your code
    }
    41

    View Slide

  63. Need to do
    I/O?
    42

    View Slide

  64. Need to do
    I/O?
    Not a problem
    42

    View Slide

  65. Need to do
    I/O THREE
    TIMES?
    43

    View Slide

  66. Need to do
    I/O THREE
    TIMES?
    Not a problem
    43

    View Slide

  67. What if I need
    to calculate π
    to the
    2,345,123rd
    decimal
    place?
    44

    View Slide

  68. NOT A
    PROBLEM
    44

    View Slide

  69. Threads achieve
    true
    parallelism
    45

    View Slide

  70. easy to
    understand
    46

    View Slide

  71. maximizes
    resources
    47

    View Slide

  72. Threads
    don’t work in
    Ruby
    48

    View Slide

  73. Threads
    don’t work in
    Ruby…right?
    48

    View Slide

  74. MRI (aka C Ruby)
    49

    View Slide

  75. MRI (aka C Ruby)
    Thread is an OS thread
    (except in 1.8)
    49

    View Slide

  76. MRI (aka C Ruby)
    Thread is an OS thread
    GIL :( (except in 1.8)
    49

    View Slide

  77. MRI (aka C Ruby)
    Thread is an OS thread
    GIL :(
    I/O will cause a context
    switch
    (except in 1.8)
    49

    View Slide

  78. Rubinius
    50

    View Slide

  79. Rubinius
    Thread is an OS thread
    50

    View Slide

  80. Rubinius
    Thread is an OS thread
    No GIL!
    50

    View Slide

  81. Rubinius
    Thread is an OS thread
    No GIL!
    True parallelism
    50

    View Slide

  82. JRuby
    51

    View Slide

  83. JRuby
    Thread is a JVM Thread
    (which is an OS thread)
    51

    View Slide

  84. JRuby
    Thread is a JVM Thread
    (which is an OS thread)
    Context switch for variety of
    reasons
    51

    View Slide

  85. JRuby
    Thread is a JVM Thread
    (which is an OS thread)
    Context switch for variety of
    reasons
    True parallelism
    51

    View Slide

  86. JRuby
    BATTLE TESTED
    51

    View Slide

  87. So, you’ve
    decided to
    use
    Threads…
    52

    View Slide

  88. You need to
    know four
    things
    53

    View Slide

  89. 54

    View Slide

  90. Start & Manage
    54

    View Slide

  91. Start & Manage
    (Dealing with) Shared State
    54

    View Slide

  92. Start & Manage
    (Dealing with) Shared State
    Using Third Party Libraries
    54

    View Slide

  93. Start & Manage
    (Dealing with) Shared State
    Using Third Party Libraries
    Context Switching
    54

    View Slide

  94. Start &
    Manage
    Threads
    55

    View Slide

  95. Chaos
    56

    View Slide

  96. Thread.new {
    # your code
    }
    Thread.new {
    # moar code
    }
    # etc
    57

    View Slide

  97. Hand-Roll
    58

    View Slide

  98. threads = []
    threads << Thread.new {
    # your code
    }
    threads << Thread.new {
    # moar code
    }
    # etc
    threads.each(&:join)
    # All threads have completed
    exit 0
    59

    View Slide

  99. java.util.concurrent
    60

    View Slide

  100. import java.util.concurrent # JRuby only
    service =
    Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    61

    View Slide

  101. 62

    View Slide

  102. 63

    View Slide


  103. 63

    View Slide

  104. ExecutorService
    64

    View Slide

  105. ExecutorService
    65

    View Slide

  106. ExecutorService
    execute()
    65

    View Slide

  107. ExecutorService
    execute()
    shutdown()
    65

    View Slide

  108. ExecutorService
    execute()
    shutdown()
    await_termination()
    65

    View Slide

  109. ExecutorService
    execute()
    shutdown()
    await_termination()
    shutdown_now()
    65

    View Slide

  110. service = Executors.new_fixed_thread_pool(10)
    tcp_erver = TCPServer.new("127.0.0.1",8080)
    loop do {
    s = tcp_server.accept
    service.execute {
    s.puts calculate_pi()
    s.close
    }
    }
    66

    View Slide

  111. service = Executors.new_fixed_thread_pool(10)
    tcp_server = TCPServer.new("127.0.0.1",8080)
    Signal.trap('SIGINT') {
    service.shutdown
    }
    loop do {
    s = tcp_server.accept
    service.execute {
    s.puts calculate_pi()
    s.close
    }
    break if service.is_shutdown
    }
    67

    View Slide

  112. service = Executors.new_fixed_thread_pool(10)
    tcp_server = TCPServer.new("127.0.0.1",8080)
    Signal.trap('SIGINT') {
    service.shutdown
    }
    loop do {
    s = tcp_server.accept
    service.execute {
    s.puts calculate_pi()
    s.close
    }
    break if service.is_shutdown
    }
    service.await_termination(10,TimeUnit.SECONDS)
    service.shutdown_now
    68

    View Slide

  113. So much more
    69

    View Slide

  114. ScheduledExecutorService
    So much more
    69

    View Slide

  115. ScheduledExecutorService
    So much more
    ThreadFactory
    69

    View Slide

  116. Read the
    javadocs
    ScheduledExecutorService
    So much more
    ThreadFactory
    69

    View Slide

  117. Shared State
    70

    View Slide

  118. results = []
    Thread.new {
    results << some_result()
    }
    Thread.new {
    results << other_result()
    }
    71

    View Slide

  119. mutex = Mutex.new
    results = []
    Thread.new {
    mutex.synchronize {
    results << some_result()
    }
    }
    Thread.new {
    mutex.synchronize {
    results << other_result()
    }
    }
    72

    View Slide

  120. Avoid explicit
    locking/sync
    73

    View Slide

  121. results = ConcurrentLinkedQueue.new
    Thread.new {
    results.offer(some_result())
    }
    Thread.new {
    results.offer(some_result())
    }
    74

    View Slide

  122. import java.util.concurrent.atomic
    flag = AtomicBoolean.new(true)
    Thread.new {
    if flag.get {
    flag.set(false)
    }
    }
    Thread.new {
    flag.set(true)
    }
    75

    View Slide

  123. So much more
    76

    View Slide

  124. ConcurrentHashMap
    So much more
    76

    View Slide

  125. ConcurrentHashMap
    So much more
    AtomicReference
    76

    View Slide

  126. Read the
    javadocs
    ConcurrentHashMap
    So much more
    AtomicReference
    76

    View Slide

  127. Third Party
    Libraries
    77

    View Slide

  128. Are they
    thread safe?
    78

    View Slide

  129. Probably
    79

    View Slide

  130. Probably
    …but to be sure
    79

    View Slide

  131. variables
    80

    View Slide

  132. variables
    Global
    80

    View Slide

  133. variables
    Global
    Class
    80

    View Slide

  134. Most are fine
    81

    View Slide

  135. Context Switching
    82

    View Slide

  136. Amdahl’s
    Law
    83

    View Slide

  137. Performance Gains
    Increase in # of Threads
    Speedup
    84

    View Slide

  138. Performance Gains with Context Switching
    Increase in # of Threads
    Speedup
    Cost of Context
    Switching Takes
    Over
    85

    View Slide

  139. Thread Pools
    86

    View Slide

  140. Worker
    Worker
    Worker
    Worker
    Thread
    Thread
    Thread
    Thread Pool
    Worker
    Worker
    Worker
    Worker
    87

    View Slide

  141. Worker
    Worker
    Worker
    Worker
    Thread
    Thread
    Thread
    Thread Pool
    Worker
    Worker
    Worker
    Worker
    87

    View Slide

  142. Worker
    Worker
    Worker
    Worker
    Thread
    Thread
    Thread
    Thread Pool
    Worker
    Worker
    Worker
    87

    View Slide

  143. Thread.new {
    # your code
    }
    88

    View Slide

  144. service.execute {
    # your code
    }
    89

    View Slide

  145. Not usually a
    concern
    90

    View Slide

  146. Always use
    Threads, right?
    91

    View Slide

  147. Thread
    UNsafe
    Libraries
    92

    View Slide

  148. Evented
    93

    View Slide

  149. Simple, I/O
    bound task
    94

    View Slide

  150. Evented
    95

    View Slide

  151. Cannot use
    JRuby
    96

    View Slide

  152. Evented
    97

    View Slide

  153. Otherwise,
    Threads will
    simplify your code
    and maximize
    your resources
    98

    View Slide

  154. Semantic
    layers on top
    of evented I/O
    99

    View Slide

  155. 100

    View Slide

  156. You’re using
    Threads
    100

    View Slide

  157. You’re using
    Threads
    (in degenerate form)
    100

    View Slide

  158. Exercises
    101

    View Slide

  159. Echo Server
    •Listen on a port
    •Respond to each request in a
    new Thread
    •Extra Credit: Record stats on
    requests in a shared data
    structure
    102

    View Slide

  160. Connection Pool
    •Allow N clients to access X shared
    instances of, say, Redis (where N > X)
    •Clients “check out” a connection and
    get exclusive access
    •Clients “check in” when done
    •Instances get re-used
    103

    View Slide

  161. THANKS!
    [email protected]
    @davetron5000
    www.naildrivin5.com
    www.awesomecommandlineapps.com
    104

    View Slide