Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Threads & JRuby: the simple alternative to evented

Threads & JRuby: the simple alternative to evented

Although event-driven solutions for handling concurrency are getting a lot of attention, they are not without compromise. The JVM provides a robust, battle-tested threading model, along with powerful concurrency libraries that make it easy to solve a wide-variety of problems that need a concurrent solution. By using JRuby, developers can get the advantages of dynamic languages like JavaScript, but wield the full power of the JVM, all while producing clean, comprehensible, testable code.

This talk will relate a real-world problem faced by LivingSocial, and its solution, to a more general set of guidelines, tools, and techniques that any developer can use to deliver a highly performant, concurrent system that is easy to understand.

David Copeland

July 19, 2012
Tweet

More Decks by David Copeland

Other Decks in Programming

Transcript

  1. Threads & JRuby
    the simple alternative to evented
    [email protected]
    @davetron5000
    www.naildrivin5.com
    www.awesomecommandlineapps.com
    Thursday, July 19, 12

    View full-size slide

  2. {
    :me => {
    :twitter => “@davetron5000”,
    :author => “Build Awesome
    Command-Line
    Applications
    in Ruby”,
    }
    }
    Thursday, July 19, 12

    View full-size slide

  3. Thursday, July 19, 12

    View full-size slide

  4. {
    :me => {
    :twitter => “@davetron5000”,
    :author => “Build Awesome
    Command Line
    Applications
    in Ruby”,
    :techlead => “LivingSocial”,
    }
    }
    Thursday, July 19, 12

    View full-size slide

  5. First Assignment
    Thursday, July 19, 12

    View full-size slide

  6. Command
    Line
    App
    Write a
    Thursday, July 19, 12

    View full-size slide

  7. Command
    Line
    App
    Write a
    To charge credit
    cards faster
    Thursday, July 19, 12

    View full-size slide

  8. NO NEW
    HARDWARE
    Thursday, July 19, 12

    View full-size slide

  9. Thursday, July 19, 12

    View full-size slide

  10. THREADS!
    Thursday, July 19, 12

    View full-size slide

  11. Are we
    maximizing our
    resources?
    Thursday, July 19, 12

    View full-size slide

  12. Are we
    maximizing our
    resources?
    Thursday, July 19, 12

    View full-size slide

  13. What problem
    are we solving?
    Thursday, July 19, 12

    View full-size slide

  14. Do some I/O
    Compute Stuff
    Thursday, July 19, 12

    View full-size slide

  15. Do some I/O
    Thursday, July 19, 12

    View full-size slide

  16. Do some I/O
    Can I do I/O? Read/Write
    Block
    YES
    NO
    Thursday, July 19, 12

    View full-size slide

  17. Do some I/O
    Block
    Thursday, July 19, 12

    View full-size slide

  18. Block
    Thursday, July 19, 12

    View full-size slide

  19. Let someone else
    work
    Block
    Thursday, July 19, 12

    View full-size slide

  20. Maximize
    Resources
    Block
    Thursday, July 19, 12

    View full-size slide

  21. Thursday, July 19, 12

    View full-size slide

  22. Will our
    code
    be
    Thursday, July 19, 12

    View full-size slide

  23. Will our
    code
    be easy to write?
    Thursday, July 19, 12

    View full-size slide

  24. easy to understand?
    Will our
    code
    be
    easy to write?
    Thursday, July 19, 12

    View full-size slide

  25. easy to understand?
    easy to test?
    Will our
    code
    be
    easy to write?
    Thursday, July 19, 12

    View full-size slide

  26. Are we
    maximizing our
    resources?
    Are we
    maximizing our
    resources?
    Thursday, July 19, 12

    View full-size slide

  27. Run lots of
    processes
    Thursday, July 19, 12

    View full-size slide

  28. Thursday, July 19, 12

    View full-size slide

  29. fork {
    # your code
    }
    Thursday, July 19, 12

    View full-size slide

  30. fork {
    # your code
    }
    not available on JVM or in
    JavaScript
    Thursday, July 19, 12

    View full-size slide

  31. Thursday, July 19, 12

    View full-size slide

  32. for i in {1..10}
    do
    ./your_app
    done
    Thursday, July 19, 12

    View full-size slide

  33. easy to
    understand
    Thursday, July 19, 12

    View full-size slide

  34. doesn’t maximize
    resources
    Thursday, July 19, 12

    View full-size slide

  35. Parent
    Thursday, July 19, 12

    View full-size slide

  36. Parent
    Parent’s
    Memory
    Thursday, July 19, 12

    View full-size slide

  37. Parent
    Parent’s
    Memory
    Child
    fork {
    Parent’s
    Memory
    (copy
    of)
    Thursday, July 19, 12

    View full-size slide

  38. Event-based I/O
    Thursday, July 19, 12

    View full-size slide

  39. var http = require('http');
    http.createServer(function (req, res) {
    res.writeHead(200,
    {'Content-Type':
    'text/plain'});
    res.end('Hello World\n');
    }).listen(1337, '127.0.0.1');
    Thursday, July 19, 12

    View full-size slide

  40. // bring in special classes and setup
    eventedThing.whenIOIsReady(function(results) {
    // do something with results
    }).startEventLoop();
    Thursday, July 19, 12

    View full-size slide

  41. Need to do I/O
    again?
    Thursday, July 19, 12

    View full-size slide

  42. // bring in special classes and setup
    eventedThing.whenIOIsReady(function(results) {
    // do something with results
    otherEventedThing.whenMoreIOIsReady(function(moreResults) {
    // do something with more I/O
    });
    }).startEventLoop();
    Thursday, July 19, 12

    View full-size slide

  43. // bring in special classes and setup
    eventedThing.whenIOIsReady(function(results) {
    // do something with results
    otherEventedThing.whenMoreIOIsReady(function(moreResults) {
    // do something with more I/O
    evenMoreEventedThings.whenYetMorIOIsReady(function(moreResults) {
    // do even more stuff with I/O
    });
    });
    }).startEventLoop();
    Thursday, July 19, 12

    View full-size slide

  44. public class EchoServer {
    public static void main(String[] args) throws Exception {
    ServerBootstrap bootstrap = new ServerBootstrap(
    new NioServerSocketChannelFactory(
    Executors.newCachedThreadPool(),
    Executors.newCachedThreadPool()));
    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
    public ChannelPipeline getPipeline() throws Exception {
    return Channels.pipeline(new EchoServerHandler());
    }
    });
    bootstrap.bind(new InetSocketAddress(8080));
    }
    }
    Thursday, July 19, 12

    View full-size slide

  45. public class EchoServer {
    public static void main(String[] args) throws Exception {
    ServerBootstrap bootstrap = new ServerBootstrap(
    new NioServerSocketChannelFactory(
    Executors.newCachedThreadPool(),
    Executors.newCachedThreadPool()));
    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
    public ChannelPipeline getPipeline() throws Exception {
    return Channels.pipeline(new EchoServerHandler());
    }
    });
    bootstrap.bind(new InetSocketAddress(8080));
    }
    }
    Thursday, July 19, 12

    View full-size slide

  46. public class EchoServerHandler extends SimpleChannelUpstreamHandler {
    @Override
    public void messageReceived(ChannelHandlerContext ctx,
    MessageEvent e) {
    e.getChannel().write(e.getMessage());
    }
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx,
    ExceptionEvent e) {
    e.getChannel().close();
    }
    }
    Thursday, July 19, 12

    View full-size slide

  47. NOT easy to
    understand
    Thursday, July 19, 12

    View full-size slide

  48. kinda
    maximizes resources
    Thursday, July 19, 12

    View full-size slide

  49. Entire call chain
    must be evented
    Thursday, July 19, 12

    View full-size slide

  50. Not true
    parallelism
    Thursday, July 19, 12

    View full-size slide

  51. Not true
    parallelism
    unless you run lots of
    processes
    Thursday, July 19, 12

    View full-size slide

  52. NOT easy to
    understand
    kinda
    maximizes resources
    Thursday, July 19, 12

    View full-size slide

  53. Threads
    Thursday, July 19, 12

    View full-size slide

  54. Thursday, July 19, 12

    View full-size slide

  55. Thread.new {
    # your code
    }
    Thursday, July 19, 12

    View full-size slide

  56. fork {
    # your code
    }
    Thursday, July 19, 12

    View full-size slide

  57. Thread.new {
    # your code
    }
    Thursday, July 19, 12

    View full-size slide

  58. new Thread(new Runnable() {
    public void run() {
    // your code
    }
    }).start();
    Thursday, July 19, 12

    View full-size slide

  59. Need to do I/O?
    Thursday, July 19, 12

    View full-size slide

  60. Need to do I/O?
    Not a problem
    Thursday, July 19, 12

    View full-size slide

  61. Need to do I/O
    THREE TIMES?
    Thursday, July 19, 12

    View full-size slide

  62. Need to do I/O
    THREE TIMES?
    Not a problem
    Thursday, July 19, 12

    View full-size slide

  63. What if I need to
    calculate π to the
    2,345,123rd
    decimal place?
    Thursday, July 19, 12

    View full-size slide

  64. NOT A
    PROBLEM
    Thursday, July 19, 12

    View full-size slide

  65. Threads achieve
    true
    parallelism
    Thursday, July 19, 12

    View full-size slide

  66. easy to
    understand
    Thursday, July 19, 12

    View full-size slide

  67. maximizes
    resources
    Thursday, July 19, 12

    View full-size slide

  68. JRuby
    Thursday, July 19, 12

    View full-size slide

  69. Ruby runtime…
    Thursday, July 19, 12

    View full-size slide

  70. Ruby runtime…
    …running on the
    JVM
    Thursday, July 19, 12

    View full-size slide

  71. new Thread(new Runnable() {
    public void run() {
    List people = queryPeople();
    for (Person p: people) {
    if (p.getAge() < 18) {
    // do one thing for minors
    }
    else {
    // do another thing for adults
    }
    }
    }
    }).start();
    Thursday, July 19, 12

    View full-size slide

  72. Thread.new {
    query_people.each do |p|
    if p.get_age < 18
    # do one thing for minors
    else
    # do another thing for adults
    end
    end
    }
    Thursday, July 19, 12

    View full-size slide

  73. Thread.new {
    query_people.each do |p|
    if p.get_age < 18
    # do one thing for minors
    else
    # do another thing for adults
    end
    end
    }
    Thursday, July 19, 12

    View full-size slide

  74. Thread.new {
    query_people.each do |p|
    if p.get_age < 18
    # do one thing for minors
    else
    # do another thing for adults
    end
    end
    }
    Thursday, July 19, 12

    View full-size slide

  75. Fast, powerful
    dynamic
    language…
    Thursday, July 19, 12

    View full-size slide

  76. Fast, powerful
    dynamic
    language…
    …full power of the
    JVM
    Thursday, July 19, 12

    View full-size slide

  77. Other Rubies?
    MRI 1.8 Green Threads No Parallelism
    MRI 1.9 Real Threads No Parallelism
    Rubinius Real Threads, no GIL True Parallelism
    Thursday, July 19, 12

    View full-size slide

  78. Other Rubies?
    MRI 1.8 Green Threads No Parallelism
    MRI 1.9 Real Threads No Parallelism
    Rubinius Real Threads, no GIL True Parallelism
    Thursday, July 19, 12

    View full-size slide

  79. So, you’ve
    decided to use
    Threads…
    Thursday, July 19, 12

    View full-size slide

  80. You need to know
    four things
    Thursday, July 19, 12

    View full-size slide

  81. Thursday, July 19, 12

    View full-size slide

  82. Start & Manage
    Thursday, July 19, 12

    View full-size slide

  83. Start & Manage
    (Dealing with) Shared State
    Thursday, July 19, 12

    View full-size slide

  84. Start & Manage
    (Dealing with) Shared State
    Using Third Party Libraries
    Thursday, July 19, 12

    View full-size slide

  85. Start & Manage
    (Dealing with) Shared State
    Using Third Party Libraries
    Context Switching
    Thursday, July 19, 12

    View full-size slide

  86. Start & Manage
    Threads
    Thursday, July 19, 12

    View full-size slide

  87. Chaos
    Thursday, July 19, 12

    View full-size slide

  88. Thread.new {
    # your code
    }
    Thread.new {
    # moar code
    }
    # etc
    Thursday, July 19, 12

    View full-size slide

  89. Hand-Roll
    Thursday, July 19, 12

    View full-size slide

  90. threads = []
    threads << Thread.new {
    # your code
    }
    threads << Thread.new {
    # moar code
    }
    # etc
    threads.each(&:join)
    # All threads have completed
    exit 0
    Thursday, July 19, 12

    View full-size slide

  91. java.util.concurrent
    Thursday, July 19, 12

    View full-size slide

  92. java_import java.util.concurrent
    service = Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    service.shutdown
    service.await_termination(10,SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  93. java_import java.util.concurrent
    service = Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    service.shutdown
    service.await_termination(10,SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  94. java_import java.util.concurrent
    service = Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    service.shutdown
    service.await_termination(10,SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  95. java_import java.util.concurrent
    service = Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    service.shutdown
    service.await_termination(10,SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  96. java_import java.util.concurrent
    service = Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    service.shutdown
    service.await_termination(10,SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  97. java_import java.util.concurrent
    service = Executors.new_fixed_thread_pool(100)
    service.execute {
    # your code
    }
    service.execute {
    # some other code
    }
    service.shutdown
    service.await_termination(10,SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  98. service = Executors.new_fixed_thread_pool(10)
    tcp_server = TCPServer.new("127.0.0.1",8080)
    Signal.trap('SIGINT') {
    service.shutdown
    }
    loop do {
    s = tcp_server.accept
    service.execute {
    s.puts calculate_pi()
    s.close
    }
    break if service.is_shutdown
    }
    service.await_termination(10,TimeUnit.SECONDS)
    service.shutdown_now
    Thursday, July 19, 12

    View full-size slide

  99. So much more
    Thursday, July 19, 12

    View full-size slide

  100. ScheduledExecutorService
    So much more
    Thursday, July 19, 12

    View full-size slide

  101. ScheduledExecutorService
    So much more
    ThreadFactory
    Thursday, July 19, 12

    View full-size slide

  102. Read the javadocs
    ScheduledExecutorService
    So much more
    ThreadFactory
    Thursday, July 19, 12

    View full-size slide

  103. Shared State
    Thursday, July 19, 12

    View full-size slide

  104. results = []
    Thread.new {
    results << some_result()
    }
    Thread.new {
    results << other_result()
    }
    Thursday, July 19, 12

    View full-size slide

  105. mutex = Mutex.new
    results = []
    Thread.new {
    mutex.synchronize {
    results << some_result()
    }
    }
    Thread.new {
    mutex.synchronize {
    results << other_result()
    }
    }
    Thursday, July 19, 12

    View full-size slide

  106. Avoid explicit
    locking/sync
    Thursday, July 19, 12

    View full-size slide

  107. results = ConcurrentLinkedQueue.new
    Thread.new {
    results.offer(some_result())
    }
    Thread.new {
    results.offer(some_result())
    }
    Thursday, July 19, 12

    View full-size slide

  108. So much more
    Thursday, July 19, 12

    View full-size slide

  109. ConcurrentHashMap
    So much more
    Thursday, July 19, 12

    View full-size slide

  110. ConcurrentHashMap
    So much more
    AtomicReference
    Thursday, July 19, 12

    View full-size slide

  111. Read the javadocs
    ConcurrentHashMap
    So much more
    AtomicReference
    Thursday, July 19, 12

    View full-size slide

  112. Third Party
    Libraries
    Thursday, July 19, 12

    View full-size slide

  113. Are they thread
    safe?
    Thursday, July 19, 12

    View full-size slide

  114. Probably
    Thursday, July 19, 12

    View full-size slide

  115. Probably
    …but to be sure
    Thursday, July 19, 12

    View full-size slide

  116. variables
    Thursday, July 19, 12

    View full-size slide

  117. variables
    Global
    Thursday, July 19, 12

    View full-size slide

  118. variables
    Global
    Class
    Thursday, July 19, 12

    View full-size slide

  119. Most are fine
    Thursday, July 19, 12

    View full-size slide

  120. Context Switching
    Thursday, July 19, 12

    View full-size slide

  121. Amdahl’s Law
    Thursday, July 19, 12

    View full-size slide

  122. Performance Gains
    Increase in # of Threads
    Speedup
    Thursday, July 19, 12

    View full-size slide

  123. Performance Gains with Context Switching
    Increase in # of Threads
    Speedup
    Cost of Context Switching Takes
    Over
    Thursday, July 19, 12

    View full-size slide

  124. Thread Pools
    Thursday, July 19, 12

    View full-size slide

  125. Your Code
    Your Code
    Your Code
    Your Code
    Thread
    Thread
    Thread
    Thread Pool
    Your Code
    Your Code
    Your Code
    Your Code
    Thursday, July 19, 12

    View full-size slide

  126. Your Code
    Your Code
    Your Code
    Your Code
    Thread
    Thread
    Thread
    Thread Pool
    Your Code
    Your Code
    Your Code
    Your Code
    Thursday, July 19, 12

    View full-size slide

  127. Your Code
    Your Code
    Your Code
    Your Code
    Thread
    Thread
    Thread
    Thread Pool
    Your Code
    Your Code
    Your Code
    Thursday, July 19, 12

    View full-size slide

  128. Thread.new {
    # your code
    }
    Thursday, July 19, 12

    View full-size slide

  129. service.execute {
    # your code
    }
    Thursday, July 19, 12

    View full-size slide

  130. Not usually a
    concern
    Thursday, July 19, 12

    View full-size slide

  131. Always use Threads,
    right?
    Thursday, July 19, 12

    View full-size slide

  132. Thread UNsafe
    Libraries
    Thursday, July 19, 12

    View full-size slide

  133. Evented
    Thursday, July 19, 12

    View full-size slide

  134. Cannot use
    JRuby/JVM
    Thursday, July 19, 12

    View full-size slide

  135. Evented
    Thursday, July 19, 12

    View full-size slide

  136. Otherwise, Threads will
    simplify your code and
    maximize your
    resources
    Thursday, July 19, 12

    View full-size slide

  137. Ask “why
    shouldn’t I use
    threads?”
    Thursday, July 19, 12

    View full-size slide

  138. Exercises
    Thursday, July 19, 12

    View full-size slide

  139. Echo Server
    •Listen on a port
    •Respond to each request in a new Thread
    •Extra Credit: Record stats on requests in a
    shared data structure
    Thursday, July 19, 12

    View full-size slide

  140. Connection Pool
    •Allow N clients to access X shared instances of, say,
    Redis (where N > X)
    •Clients “check out” a connection and get exclusive
    access
    •Clients “check in” when done
    •Instances get re-used
    Thursday, July 19, 12

    View full-size slide

  141. THANKS!
    [email protected]
    @davetron5000
    www.naildrivin5.com
    www.awesomecommandlineapps.com
    Thursday, July 19, 12

    View full-size slide

  142. THANKS!
    [email protected]
    @davetron5000
    www.naildrivin5.com
    www.awesomecommandlineapps.com
    We’re Hiring!
    Thursday, July 19, 12

    View full-size slide