Asynchronous and Non-Blocking Scala For Fun & Profit

Transcript

1. Brendan McAdams
   10gen, Inc.
   brendan@10gen.com
   @rit
   Asynchronous + Non-Blocking Scala
   for Fun & Profit
   A look at Netty & NIO for Asynchronous networking
   via Scala
   Friday, March 9, 12
   

   View Slide

2. Goals
   Friday, March 9, 12
   

   View Slide

3. Goals
   • Some simple goals ...
   Friday, March 9, 12
   

   View Slide

4. Goals
   • Some simple goals ...
   • Stop wasting resources on blocking I/O
   Friday, March 9, 12
   

   View Slide

5. Goals
   • Some simple goals ...
   • Stop wasting resources on blocking I/O
   • Achieve C10K (10,000 simultaneous clients)
   Friday, March 9, 12
   

   View Slide

6. Goals
   • Some simple goals ...
   • Stop wasting resources on blocking I/O
   • Achieve C10K (10,000 simultaneous clients)
   • Profit
   Friday, March 9, 12
   

   View Slide

7. Explaining Non-Blocking & Asynchronous
   Friday, March 9, 12
   

   View Slide

8. Explaining Non-Blocking & Asynchronous
   • We are talking about I/O here
   Friday, March 9, 12
   

   View Slide

9. Explaining Non-Blocking & Asynchronous
   • We are talking about I/O here
   • By which, of course, we mean “Input / Output”
   Friday, March 9, 12
   

   View Slide

10. Explaining Non-Blocking & Asynchronous
    • We are talking about I/O here
    • By which, of course, we mean “Input / Output”
    • We’ll focus on networking I/O
    Friday, March 9, 12
    

    View Slide

11. Explaining Non-Blocking & Asynchronous
    Friday, March 9, 12
    

    View Slide

12. Explaining Non-Blocking & Asynchronous
    •Asynchronous describes a way to accomplish Non-Blocking I/O
    Friday, March 9, 12
    

    View Slide

13. Explaining Non-Blocking & Asynchronous
    •Asynchronous describes a way to accomplish Non-Blocking I/O
    • I like to think of these as a mini-stack, with Async on top
    of Non-blocking
    Friday, March 9, 12
    

    View Slide

14. Explaining Non-Blocking & Asynchronous
    Friday, March 9, 12
    

    View Slide

15. Explaining Non-Blocking & Asynchronous
    •Blocking presents a series of problems:
    Friday, March 9, 12
    

    View Slide

16. Explaining Non-Blocking & Asynchronous
    •Blocking presents a series of problems:
    • When a blocking I/O operation occurs, everything in
    that thread halts and waits; potentially idle system
    resources
    Friday, March 9, 12
    

    View Slide

17. Explaining Non-Blocking & Asynchronous
    •Blocking presents a series of problems:
    • When a blocking I/O operation occurs, everything in
    that thread halts and waits; potentially idle system
    resources
    • Internally, that “blocking” operation is a big loop per
    operation asking “Are we there yet?”
    Friday, March 9, 12
    

    View Slide

18. Explaining Non-Blocking & Asynchronous
    •Blocking presents a series of problems:
    • When a blocking I/O operation occurs, everything in
    that thread halts and waits; potentially idle system
    resources
    • Internally, that “blocking” operation is a big loop per
    operation asking “Are we there yet?”
    • “Idleness” occurs because the thread waiting for I/O to
    complete is doing nothing while it waits
    Friday, March 9, 12
    

    View Slide

19. Explaining Non-Blocking & Asynchronous
    •Blocking presents a series of problems:
    • When a blocking I/O operation occurs, everything in
    that thread halts and waits; potentially idle system
    resources
    • Internally, that “blocking” operation is a big loop per
    operation asking “Are we there yet?”
    • “Idleness” occurs because the thread waiting for I/O to
    complete is doing nothing while it waits
    • This can vastly limit our ability to scale
    Friday, March 9, 12
    

    View Slide

20. Explaining Non-Blocking & Asynchronous
    Friday, March 9, 12
    

    View Slide

21. Explaining Non-Blocking & Asynchronous
    • Non-Blocking I/O presents solutions:
    Friday, March 9, 12
    

    View Slide

22. Explaining Non-Blocking & Asynchronous
    • Non-Blocking I/O presents solutions:
    • Eschew blocking individually on each operation
    Friday, March 9, 12
    

    View Slide

23. Explaining Non-Blocking & Asynchronous
    • Non-Blocking I/O presents solutions:
    • Eschew blocking individually on each operation
    • Find ways to work with the kernel to more efficiently
    manage multiple blocking resources in groups
    Friday, March 9, 12
    

    View Slide

24. Explaining Non-Blocking & Asynchronous
    • Non-Blocking I/O presents solutions:
    • Eschew blocking individually on each operation
    • Find ways to work with the kernel to more efficiently
    manage multiple blocking resources in groups
    • Free up threads which are no longer blocked waiting
    on I/O to handle other requests while those requests
    waiting on I/O idle
    Friday, March 9, 12
    

    View Slide

25. Where does Asynchronous come in?
    Friday, March 9, 12
    

    View Slide

26. Where does Asynchronous come in?
    • If we are no longer blocking and instead reusing threads
    while I/O waits, we need a way to handle “completion” events
    Friday, March 9, 12
    

    View Slide

27. Where does Asynchronous come in?
    • If we are no longer blocking and instead reusing threads
    while I/O waits, we need a way to handle “completion” events
    • Asynchronous techniques (such as callbacks) allow us to
    achieve this
    Friday, March 9, 12
    

    View Slide

28. Where does Asynchronous come in?
    • If we are no longer blocking and instead reusing threads
    while I/O waits, we need a way to handle “completion” events
    • Asynchronous techniques (such as callbacks) allow us to
    achieve this
    • “Step to the side of the line, and we’ll call you when
    your order is ready”
    Friday, March 9, 12
    

    View Slide

29. Breaking it Down
    Friday, March 9, 12
    

    View Slide

30. Breaking it Down
    • The synchronous/blocking I/O model typically forces us into
    some paradigms
    Friday, March 9, 12
    

    View Slide

31. Breaking it Down
    • The synchronous/blocking I/O model typically forces us into
    some paradigms
    • Servers
    Friday, March 9, 12
    

    View Slide

32. Breaking it Down
    • The synchronous/blocking I/O model typically forces us into
    some paradigms
    • Servers
    • A 1:1 ratio of threads to client connections (scale
    limited)
    Friday, March 9, 12
    

    View Slide

33. Breaking it Down
    • The synchronous/blocking I/O model typically forces us into
    some paradigms
    • Servers
    • A 1:1 ratio of threads to client connections (scale
    limited)
    • Clients
    Friday, March 9, 12
    

    View Slide

34. Breaking it Down
    • The synchronous/blocking I/O model typically forces us into
    some paradigms
    • Servers
    • A 1:1 ratio of threads to client connections (scale
    limited)
    • Clients
    • Connection pools often larger than necessary to
    account for unavailable-while-blocking connection
    resources
    Friday, March 9, 12
    

    View Slide

35. Breaking it Down
    Friday, March 9, 12
    

    View Slide

36. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    Friday, March 9, 12
    

    View Slide

37. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    Friday, March 9, 12
    

    View Slide

38. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    Friday, March 9, 12
    

    View Slide

39. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    • A thread waiting for an I/O response no longer needs to block it,
    allowing other threads to reuse that connection simultaneously
    Friday, March 9, 12
    

    View Slide

40. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    • A thread waiting for an I/O response no longer needs to block it,
    allowing other threads to reuse that connection simultaneously
    • Operations such as a “write” can queue up until resource is ready,
    returning thread of execution immediately, callback results later.
    Friday, March 9, 12
    

    View Slide

41. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    • A thread waiting for an I/O response no longer needs to block it,
    allowing other threads to reuse that connection simultaneously
    • Operations such as a “write” can queue up until resource is ready,
    returning thread of execution immediately, callback results later.
    • Of course, this makes dispatch and good concurrency even
    tougher (Who said doing things right was ever easy?)
    Friday, March 9, 12
    

    View Slide

42. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    • A thread waiting for an I/O response no longer needs to block it,
    allowing other threads to reuse that connection simultaneously
    • Operations such as a “write” can queue up until resource is ready,
    returning thread of execution immediately, callback results later.
    • Of course, this makes dispatch and good concurrency even
    tougher (Who said doing things right was ever easy?)
    • Clients
    Friday, March 9, 12
    

    View Slide

43. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    • A thread waiting for an I/O response no longer needs to block it,
    allowing other threads to reuse that connection simultaneously
    • Operations such as a “write” can queue up until resource is ready,
    returning thread of execution immediately, callback results later.
    • Of course, this makes dispatch and good concurrency even
    tougher (Who said doing things right was ever easy?)
    • Clients
    • Significantly reduce pool sizes
    Friday, March 9, 12
    

    View Slide

44. Breaking it Down
    • If we go asynchronous/non-blocking we can change our destiny
    • Servers
    • : Ratio becomes
    possible (Scale - C10k and beyond)
    • A thread waiting for an I/O response no longer needs to block it,
    allowing other threads to reuse that connection simultaneously
    • Operations such as a “write” can queue up until resource is ready,
    returning thread of execution immediately, callback results later.
    • Of course, this makes dispatch and good concurrency even
    tougher (Who said doing things right was ever easy?)
    • Clients
    • Significantly reduce pool sizes
    • connection resources can be leveraged by many more
    simultaneous threads
    Friday, March 9, 12
    

    View Slide

45. Introducing NIO
    Friday, March 9, 12
    

    View Slide

46. Introducing NIO
    • History
    Friday, March 9, 12
    

    View Slide

47. Introducing NIO
    • History
    • “New I/O”, introduced in Java 1.4
    Friday, March 9, 12
    

    View Slide

48. Introducing NIO
    • History
    • “New I/O”, introduced in Java 1.4
    • Focus on low-level I/O as opposed to “Old” I/O’s high
    level-API
    Friday, March 9, 12
    

    View Slide

49. Introducing NIO
    • History
    • “New I/O”, introduced in Java 1.4
    • Focus on low-level I/O as opposed to “Old” I/O’s high
    level-API
    • ByteBuffers: http://www.kdgregory.com/index.php?page=java.byteBuffer
    Friday, March 9, 12
    

    View Slide

50. Introducing NIO
    Friday, March 9, 12
    

    View Slide

51. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    Friday, March 9, 12
    

    View Slide

52. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    • Callback when ready
    Friday, March 9, 12
    

    View Slide

53. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    • Callback when ready
    • Core units of work: Buffer, Channel and Selector
    Friday, March 9, 12
    

    View Slide

54. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    • Callback when ready
    • Core units of work: Buffer, Channel and Selector
    • Buffer are contiguous memory slots, offering data
    transfer operations
    Friday, March 9, 12
    

    View Slide

55. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    • Callback when ready
    • Core units of work: Buffer, Channel and Selector
    • Buffer are contiguous memory slots, offering data
    transfer operations
    • Channel instances are “bulk” data wrappers to Buffer
    Friday, March 9, 12
    

    View Slide

56. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    • Callback when ready
    • Core units of work: Buffer, Channel and Selector
    • Buffer are contiguous memory slots, offering data
    transfer operations
    • Channel instances are “bulk” data wrappers to Buffer
    • Selector is an event monitor which can watch
    multiple Channel instances in a single thread (the
    kernel coordinator)
    Friday, March 9, 12
    

    View Slide

57. Introducing NIO
    • For working with Networks, we must manually work with
    Selector, and request a window to read and write
    • Callback when ready
    • Core units of work: Buffer, Channel and Selector
    • Buffer are contiguous memory slots, offering data
    transfer operations
    • Channel instances are “bulk” data wrappers to Buffer
    • Selector is an event monitor which can watch
    multiple Channel instances in a single thread (the
    kernel coordinator)
    • Relocate the task of checking I/O status out of the
    “execution” threads
    Friday, March 9, 12
    

    View Slide

58. NIO in Practice
    Friday, March 9, 12
    

    View Slide

59. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    Friday, March 9, 12
    

    View Slide

60. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    Friday, March 9, 12
    

    View Slide

61. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    Friday, March 9, 12
    

    View Slide

62. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    Friday, March 9, 12
    

    View Slide

63. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    Friday, March 9, 12
    

    View Slide

64. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    Friday, March 9, 12
    

    View Slide

65. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    • Want to write? Tell the Selector you want to write
    Friday, March 9, 12
    

    View Slide

66. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    • Want to write? Tell the Selector you want to write
    • Eventually, when the Channel is available to write, an event
    will notify the Selector
    Friday, March 9, 12
    

    View Slide

67. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    • Want to write? Tell the Selector you want to write
    • Eventually, when the Channel is available to write, an event
    will notify the Selector
    • Remove “interested in writing” status
    Friday, March 9, 12
    

    View Slide

68. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    • Want to write? Tell the Selector you want to write
    • Eventually, when the Channel is available to write, an event
    will notify the Selector
    • Remove “interested in writing” status
    • Dispatch “time to write” to original caller
    Friday, March 9, 12
    

    View Slide

69. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    • Want to write? Tell the Selector you want to write
    • Eventually, when the Channel is available to write, an event
    will notify the Selector
    • Remove “interested in writing” status
    • Dispatch “time to write” to original caller
    • Write
    Friday, March 9, 12
    

    View Slide

70. NIO in Practice
    • I kicked back & forth with a few possible examples of NIO here
    • Conclusion: for time & sanity, omit a code sample
    • Here’s what you need to know
    • Register “Interests” with a Selector (Read, Write, etc)
    • Write a Selector loop which checks for notification events
    • Dispatch incoming events (such as “read”)
    • Want to write? Tell the Selector you want to write
    • Eventually, when the Channel is available to write, an event
    will notify the Selector
    • Remove “interested in writing” status
    • Dispatch “time to write” to original caller
    • Write
    • Rinse, repeat.
    Friday, March 9, 12
    

    View Slide

71. Introducing Netty
    Friday, March 9, 12
    

    View Slide

72. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    Friday, March 9, 12
    

    View Slide

73. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    Friday, March 9, 12
    

    View Slide

74. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    Friday, March 9, 12
    

    View Slide

75. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    Friday, March 9, 12
    

    View Slide

76. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    • ChannelBuffers
    Friday, March 9, 12
    

    View Slide

77. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    • ChannelBuffers
    • Can composite multiple ChannelBuffers
    Friday, March 9, 12
    

    View Slide

78. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    • ChannelBuffers
    • Can composite multiple ChannelBuffers
    • Organize individual pieces in one composite buffer
    Friday, March 9, 12
    

    View Slide

79. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    • ChannelBuffers
    • Can composite multiple ChannelBuffers
    • Organize individual pieces in one composite buffer
    • Supports ByteBuffer, Arrays, etc
    Friday, March 9, 12
    

    View Slide

80. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    • ChannelBuffers
    • Can composite multiple ChannelBuffers
    • Organize individual pieces in one composite buffer
    • Supports ByteBuffer, Arrays, etc
    • Avoid memory copy as much as possible
    Friday, March 9, 12
    

    View Slide

81. Introducing Netty
    • Simplify NIO (but still provides access to oio)
    • Really, just wrapping NIO to provide higher level
    abstraction
    • Hides the Selector nonsense away
    • Composable “Filter Pipeline” allows you to intercept multiple
    levels of input and output
    • ChannelBuffers
    • Can composite multiple ChannelBuffers
    • Organize individual pieces in one composite buffer
    • Supports ByteBuffer, Arrays, etc
    • Avoid memory copy as much as possible
    • Direct Memory allocated (rumors of memory leaks
    abound)
    Friday, March 9, 12
    

    View Slide

82. Pipelines == Sanity
    val channelFactory = new
    NioClientSocketChannelFactory(Executors.newCachedThreadPool(ThreadFactories("Hammersmith Netty
    Boss")),
    Executors.newCachedThreadPool(ThreadFactories("Hammersmith Netty Worker")))
    protected implicit val bootstrap = new ClientBootstrap(channelFactory)
    bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
    /* Big long, epic + awesomely insightful @havocp comment */
    private val appCallbackExecutor =
    new ThreadPoolExecutor(/** constructor snipped for slide sanity */)
    private val appCallbackExecutionHandler = new ExecutionHandler(appCallbackExecutor)
    def getPipeline = {
    val p = Channels.pipeline(new ReplyMessageDecoder(), appCallbackExecutionHandler, handler)
    p
    }
    })
    bootstrap.setOption("remoteAddress", addr)
    private val _f = bootstrap.connect()
    protected implicit val channel = _f.awaitUninterruptibly.getChannel
    Friday, March 9, 12
    

    View Slide

83. CAN HAS NETWORK WRITE?
    // It turns out writing can be easy...
    channel.write(outStream.buffer())
    Friday, March 9, 12
    

    View Slide

84. Stupid Pet Tricks
    Friday, March 9, 12
    

    View Slide

85. Every good story can use a MacGuffin...
    Friday, March 9, 12
    

    View Slide

86. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    Friday, March 9, 12
    

    View Slide

87. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    Friday, March 9, 12
    

    View Slide

88. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    Friday, March 9, 12
    

    View Slide

89. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    • This is a project I’ve already spent a good bit of time on –
    Hammersmith
    Friday, March 9, 12
    

    View Slide

90. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    • This is a project I’ve already spent a good bit of time on –
    Hammersmith
    • A few focal points to lead our discussion
    Friday, March 9, 12
    

    View Slide

91. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    • This is a project I’ve already spent a good bit of time on –
    Hammersmith
    • A few focal points to lead our discussion
    • Decoding & dispatching inbound messages
    Friday, March 9, 12
    

    View Slide

92. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    • This is a project I’ve already spent a good bit of time on –
    Hammersmith
    • A few focal points to lead our discussion
    • Decoding & dispatching inbound messages
    • Handling errors & exceptions across threads, time, and space
    Friday, March 9, 12
    

    View Slide

93. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    • This is a project I’ve already spent a good bit of time on –
    Hammersmith
    • A few focal points to lead our discussion
    • Decoding & dispatching inbound messages
    • Handling errors & exceptions across threads, time, and space
    • “Follow up” operations which rely on serverside “same connection”
    context
    Friday, March 9, 12
    

    View Slide

94. Every good story can use a MacGuffin...
    • MacGuffin (n) “A plot element that catches the viewers’ attention or
    drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well)
    • We aren’t writing fiction here, but we can use a MacGuffin to drive our
    story
    • For our MacGuffin, we’ll examine Asynchronous networking against a
    MongoDB Server
    • This is a project I’ve already spent a good bit of time on –
    Hammersmith
    • A few focal points to lead our discussion
    • Decoding & dispatching inbound messages
    • Handling errors & exceptions across threads, time, and space
    • “Follow up” operations which rely on serverside “same connection”
    context
    • Working with multi-state iterations which depend on IO for
    operations... a.k.a. “Database Cursors”
    Friday, March 9, 12
    

    View Slide

95. Problem #1: Decoding & Dispatching Reads
    Friday, March 9, 12
    

    View Slide

96. Problem #1: Decoding & Dispatching Reads
    • Packets aren’t bytes
    Friday, March 9, 12
    

    View Slide

97. Problem #1: Decoding & Dispatching Reads
    • Packets aren’t bytes
    • Network layers don’t know or care about your fancy
    application layer protocol
    Friday, March 9, 12
    

    View Slide

98. Problem #1: Decoding & Dispatching Reads
    • Packets aren’t bytes
    • Network layers don’t know or care about your fancy
    application layer protocol
    • The kernel reads things off the network into a big buffer of
    bytes
    Friday, March 9, 12
    

    View Slide

99. Problem #1: Decoding & Dispatching Reads
    • Packets aren’t bytes
    • Network layers don’t know or care about your fancy
    application layer protocol
    • The kernel reads things off the network into a big buffer of
    bytes
    • It’s up to us to figure out what parts of the bytes are relevant
    where
    Friday, March 9, 12
    

    View Slide

100. Problem #1: Decoding & Dispatching Reads
     • Packets aren’t bytes
     • Network layers don’t know or care about your fancy
     application layer protocol
     • The kernel reads things off the network into a big buffer of
     bytes
     • It’s up to us to figure out what parts of the bytes are relevant
     where
     • Challenge: Separate out individual writes and send them to the right
     place
     Friday, March 9, 12
     

     View Slide

101. Problem #1: Decoding & Dispatching Reads
     • Packets aren’t bytes
     • Network layers don’t know or care about your fancy
     application layer protocol
     • The kernel reads things off the network into a big buffer of
     bytes
     • It’s up to us to figure out what parts of the bytes are relevant
     where
     • Challenge: Separate out individual writes and send them to the right
     place
     • Conceptually, “Law of Demeter” (loose coupling) helps here. Doing
     it by hand you have to be careful not to eat somebody else’s lunch
     Friday, March 9, 12
     

     View Slide

102. Problem #1: Decoding & Dispatching Reads
     • Packets aren’t bytes
     • Network layers don’t know or care about your fancy
     application layer protocol
     • The kernel reads things off the network into a big buffer of
     bytes
     • It’s up to us to figure out what parts of the bytes are relevant
     where
     • Challenge: Separate out individual writes and send them to the right
     place
     • Conceptually, “Law of Demeter” (loose coupling) helps here. Doing
     it by hand you have to be careful not to eat somebody else’s lunch
     • NIO leaves you on your own
     Friday, March 9, 12
     

     View Slide

103. Problem #1: Decoding & Dispatching Reads
     • Packets aren’t bytes
     • Network layers don’t know or care about your fancy
     application layer protocol
     • The kernel reads things off the network into a big buffer of
     bytes
     • It’s up to us to figure out what parts of the bytes are relevant
     where
     • Challenge: Separate out individual writes and send them to the right
     place
     • Conceptually, “Law of Demeter” (loose coupling) helps here. Doing
     it by hand you have to be careful not to eat somebody else’s lunch
     • NIO leaves you on your own
     • Netty’s pipeline helps provide the “don’t eat my lunch” fix
     quite well IMHO
     Friday, March 9, 12
     

     View Slide

104. In Netty, add a decoder to the Pipeline
     protected[mongodb] class ReplyMessageDecoder extends LengthFieldBasedFrameDecoder(1024
     * 1024 * 4, 0, 4, -4, 0) with Logging {
     protected override def decode(ctx: ChannelHandlerContext, channel: Channel, buffer:
     ChannelBuffer): AnyRef = {
     val frame = super.decode(ctx, channel, buffer).asInstanceOf[ChannelBuffer]
     if (frame == null) {
     // don't have the whole message yet; netty will retry later
     null
     } else {
     // we have one message (and nothing else) in the "frame" buffer
     MongoMessage.unapply(new ChannelBufferInputStream(frame)) match {
     case reply: ReplyMessage 㱺
     reply
     case default 㱺
     // this should not happen;
     throw new Exception("Unknown message type '%s' incoming from MongoDB;
     ignoring.".format(default))
     }
     }
     }
     Friday, March 9, 12
     

     View Slide

105. In Netty, add a decoder to the Pipeline
     // Because we return a new object rather than a buffer from decode(),
     // we can use slice() here according to the docs (the slice won't escape
     // the decode() method so it will be valid while we're using it)
     protected override def extractFrame(buffer: ChannelBuffer, index: Int, length: Int):
     ChannelBuffer = {
     return buffer.slice(index, length);
     }
     }
     Friday, March 9, 12
     

     View Slide

106. Expect the pipeline to have a MongoMessage
     override def messageReceived(ctx: ChannelHandlerContext, e:
     MessageEvent) {
     val message = e.getMessage.asInstanceOf[MongoMessage]
     log.debug("Incoming Message received type %s",
     message.getClass.getName)
     message match {
     case reply: ReplyMessage 㱺 {
     Friday, March 9, 12
     

     View Slide

107. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     Friday, March 9, 12
     

     View Slide

108. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     Friday, March 9, 12
     

     View Slide

109. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     • Scala has a great construct to help with this: Either[L, R]
     Friday, March 9, 12
     

     View Slide

110. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     • Scala has a great construct to help with this: Either[L, R]
     • Pass a monad that can have one of two states: Failure or
     Success
     Friday, March 9, 12
     

     View Slide

111. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     • Scala has a great construct to help with this: Either[L, R]
     • Pass a monad that can have one of two states: Failure or
     Success
     • By convention, “Left” is an Error, “Right” is success
     Friday, March 9, 12
     

     View Slide

112. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     • Scala has a great construct to help with this: Either[L, R]
     • Pass a monad that can have one of two states: Failure or
     Success
     • By convention, “Left” is an Error, “Right” is success
     • Node does a similar passing of Success vs. Result
     Friday, March 9, 12
     

     View Slide

113. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     • Scala has a great construct to help with this: Either[L, R]
     • Pass a monad that can have one of two states: Failure or
     Success
     • By convention, “Left” is an Error, “Right” is success
     • Node does a similar passing of Success vs. Result
     • No special “different” handling in Netty vs. NIO
     Friday, March 9, 12
     

     View Slide

114. Problem #2: Handling Errors, Exceptions
     and Eldritch Horrors
     • Asynchronous and callback nature makes dispatching errors
     difficult: the “throw / catch” model becomes complicated
     • Scala has a great construct to help with this: Either[L, R]
     • Pass a monad that can have one of two states: Failure or
     Success
     • By convention, “Left” is an Error, “Right” is success
     • Node does a similar passing of Success vs. Result
     • No special “different” handling in Netty vs. NIO
     • Implicit tricks for the lazy who want to just write a “success”
     block
     Friday, March 9, 12
     

     View Slide

115. Yes, I wrote my own Futures. It was an accident...
     sealed trait RequestFuture {
     type T
     val body: Either[Throwable, T] 㱺 Unit
     def apply(error: Throwable) = body(Left(error))
     def apply[A <% T](result: A) = body(Right(result.asInstanceOf[T]))
     protected[futures] var completed = false
     }
     /**
     * Will pass any *generated* _id along with any relevant getLastError information
     * For an update, don't expect to get ObjectId
     */
     trait WriteRequestFuture extends RequestFuture {
     type T <: (Option[AnyRef] /* ID Type */ , WriteResult)
     }
     implicit def asWriteOp(f: Either[Throwable, (Option[AnyRef], WriteResult)] 㱺 Unit) =
     RequestFutures.write(f)
     Friday, March 9, 12
     

     View Slide

116. Did we succeed, or did we fail?
     val handler = RequestFutures.write((result: Either[Throwable, (Option[AnyRef],
     WriteResult)]) 㱺 {
     result match {
     case Right((oid, wr)) 㱺 {
     ok = Some(true)
     id = oid
     }
     case Left(t) 㱺 {
     ok = Some(false)
     log.error(t, "Command Failed.")
     }
     }
     })
     mongo.insert(Document("foo" -> "bar", "bar" -> "baz"))(handler)
     Friday, March 9, 12
     

     View Slide

117. Errors? We don’t need no stinkin’ errors...
     implicit def asSimpleWriteOp(f: (Option[AnyRef], WriteResult) 㱺 Unit):
     WriteRequestFuture = SimpleRequestFutures.write(f)
     def write(f: (Option[AnyRef], WriteResult) 㱺 Unit) =
     new WriteRequestFuture {
     val body = (result: Either[Throwable, (Option[AnyRef], WriteResult)]) 㱺
     result match {
     case Right((oid, wr)) 㱺 f(oid, wr)
     case Left(t) 㱺 log.error(t, "Command Failed.")
     }
     override def toString = "{SimpleWriteRequestFuture}"
     }
     Friday, March 9, 12
     

     View Slide

118. Problem #3: “Same Connection” Follow Up Operations
     Friday, March 9, 12
     

     View Slide

119. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     Friday, March 9, 12
     

     View Slide

120. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     Friday, March 9, 12
     

     View Slide

121. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     Friday, March 9, 12
     

     View Slide

122. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     • Somewhat easy in a synchronous framework
     Friday, March 9, 12
     

     View Slide

123. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     • Somewhat easy in a synchronous framework
     • Lock the connection out of the pool and keep it private
     Friday, March 9, 12
     

     View Slide

124. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     • Somewhat easy in a synchronous framework
     • Lock the connection out of the pool and keep it private
     • Don’t let anyone else touch it until you’re done
     Friday, March 9, 12
     

     View Slide

125. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     • Somewhat easy in a synchronous framework
     • Lock the connection out of the pool and keep it private
     • Don’t let anyone else touch it until you’re done
     • In Async, harder
     Friday, March 9, 12
     

     View Slide

126. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     • Somewhat easy in a synchronous framework
     • Lock the connection out of the pool and keep it private
     • Don’t let anyone else touch it until you’re done
     • In Async, harder
     • Only solution I’ve found is “ballot box stuffing”
     Friday, March 9, 12
     

     View Slide

127. Problem #3: “Same Connection” Follow Up Operations
     • Some databases, etc. have contextual operations as “follow ups”
     to a write, which can only be called on the same connection as the
     write
     • MySQL has last_insert_id() to fetch generated
     autoincrement
     • MongoDB has getLastError() to check success/failure of
     a write (and explicitly specify consistency requirements)
     • Somewhat easy in a synchronous framework
     • Lock the connection out of the pool and keep it private
     • Don’t let anyone else touch it until you’re done
     • In Async, harder
     • Only solution I’ve found is “ballot box stuffing”
     • Deliberate reversal of the “decoding problem”
     Friday, March 9, 12
     

     View Slide

128. Stuffing the Ballot Box
     // Quick callback when needed to be invoked immediately after write
     val writeCB: () 㱺 Unit = if (isWrite) {
     msg match {
     case wMsg: MongoClientWriteMessage 㱺 if (concern.safe_?) {
     val gle = createCommand(wMsg.namespace.split("\\.")(0), Document("getlasterror" -
     > 1))
     dispatcher.put(gle.requestID, CompletableRequest(msg, f))
     gle.write(outStream)
     () 㱺 {}
     } else () 㱺 { wMsg.ids.foreach(x 㱺 f((x, WriteResult(true)).asInstanceOf[f.T])) }
     case unknown 㱺 {
     val e = new IllegalArgumentException("Invalid type of message passed;
     WriteRequestFutures expect a MongoClientWriteMessage underneath them. Got " + unknown)
     () 㱺 { f(e) }
     }
     }
     } else () 㱺 {}
     Friday, March 9, 12
     

     View Slide

129. Problem #4: Multi-state resource iteration a.k.a. Cursors
     Friday, March 9, 12
     

     View Slide

130. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     Friday, March 9, 12
     

     View Slide

131. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     • Two primary calls on an Iterator[A]
     Friday, March 9, 12
     

     View Slide

132. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     • Two primary calls on an Iterator[A]
     • hasNext: Boolean
     Friday, March 9, 12
     

     View Slide

133. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     • Two primary calls on an Iterator[A]
     • hasNext: Boolean
     • next(): A
     Friday, March 9, 12
     

     View Slide

134. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     • Two primary calls on an Iterator[A]
     • hasNext: Boolean
     • next(): A
     • In a pure and simple form, the Iterator[A] is prepopulated
     with all of its elements.
     Friday, March 9, 12
     

     View Slide

135. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     • Two primary calls on an Iterator[A]
     • hasNext: Boolean
     • next(): A
     • In a pure and simple form, the Iterator[A] is prepopulated
     with all of its elements.
     • If the buffer is non-empty, hasNext == true and next() returns
     another element.
     Friday, March 9, 12
     

     View Slide

136. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In typical iteration, we are working with a dual-state monad
     • Two primary calls on an Iterator[A]
     • hasNext: Boolean
     • next(): A
     • In a pure and simple form, the Iterator[A] is prepopulated
     with all of its elements.
     • If the buffer is non-empty, hasNext == true and next() returns
     another element.
     • When the buffer is empty, iteration halts completely because
     there’s nothing left to iterate. hasNext == false, next() returns
     null, throws an exception or similar.
     Friday, March 9, 12
     

     View Slide

137. Problem #4: Multi-state resource iteration a.k.a. Cursors
     Friday, March 9, 12
     

     View Slide

138. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     Friday, March 9, 12
     

     View Slide

139. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     • This maps nice and simply to the Iterator[A] monad and everyone
     can chug along happily, not caring if I/O is synchronous or asynchronous
     Friday, March 9, 12
     

     View Slide

140. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     • This maps nice and simply to the Iterator[A] monad and everyone
     can chug along happily, not caring if I/O is synchronous or asynchronous
     • In reality though, forcing a client to buffer all of the results to a large
     query is tremendously inefficient
     Friday, March 9, 12
     

     View Slide

141. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     • This maps nice and simply to the Iterator[A] monad and everyone
     can chug along happily, not caring if I/O is synchronous or asynchronous
     • In reality though, forcing a client to buffer all of the results to a large
     query is tremendously inefficient
     • Do you have enough memory on the client side for the entire result
     set?
     Friday, March 9, 12
     

     View Slide

142. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     • This maps nice and simply to the Iterator[A] monad and everyone
     can chug along happily, not caring if I/O is synchronous or asynchronous
     • In reality though, forcing a client to buffer all of the results to a large
     query is tremendously inefficient
     • Do you have enough memory on the client side for the entire result
     set?
     • With async, we may have a lot of potentially large result sets buffered
     Friday, March 9, 12
     

     View Slide

143. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     • This maps nice and simply to the Iterator[A] monad and everyone
     can chug along happily, not caring if I/O is synchronous or asynchronous
     • In reality though, forcing a client to buffer all of the results to a large
     query is tremendously inefficient
     • Do you have enough memory on the client side for the entire result
     set?
     • With async, we may have a lot of potentially large result sets buffered
     • Many databases (MongoDB, MySQL, Oracle, etc) use a multi-state result
     known as a “cursor”
     Friday, March 9, 12
     

     View Slide

144. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • In a simple database, a query would return a batch of all of the query
     results, populating an Iterator[DBRow]
     • This maps nice and simply to the Iterator[A] monad and everyone
     can chug along happily, not caring if I/O is synchronous or asynchronous
     • In reality though, forcing a client to buffer all of the results to a large
     query is tremendously inefficient
     • Do you have enough memory on the client side for the entire result
     set?
     • With async, we may have a lot of potentially large result sets buffered
     • Many databases (MongoDB, MySQL, Oracle, etc) use a multi-state result
     known as a “cursor”
     • Let the server buffer memory and chunk up batches
     Friday, March 9, 12
     

     View Slide

145. Problem #4: Multi-state resource iteration a.k.a. Cursors
     Friday, March 9, 12
     

     View Slide

146. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Cursors will return an initial batch of results
     Friday, March 9, 12
     

     View Slide

147. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Cursors will return an initial batch of results
     • If there are more results available on the server a “Cursor ID”
     is returned w/ the batch
     Friday, March 9, 12
     

     View Slide

148. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Cursors will return an initial batch of results
     • If there are more results available on the server a “Cursor ID”
     is returned w/ the batch
     • Client can use getMore to fetch additional batches until
     server results are exhausted
     Friday, March 9, 12
     

     View Slide

149. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Cursors will return an initial batch of results
     • If there are more results available on the server a “Cursor ID”
     is returned w/ the batch
     • Client can use getMore to fetch additional batches until
     server results are exhausted
     • Once exhausted, getMore will return a batch and a Cursor ID
     of 0 (indicating “no more results”)
     Friday, March 9, 12
     

     View Slide

150. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Cursors will return an initial batch of results
     • If there are more results available on the server a “Cursor ID”
     is returned w/ the batch
     • Client can use getMore to fetch additional batches until
     server results are exhausted
     • Once exhausted, getMore will return a batch and a Cursor ID
     of 0 (indicating “no more results”)
     • Try doing this cleanly without blocking...
     Friday, March 9, 12
     

     View Slide

151. Problem #4: Multi-state resource iteration a.k.a. Cursors
     Friday, March 9, 12
     

     View Slide

152. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     Friday, March 9, 12
     

     View Slide

153. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     • The typical solution in a synchronous driver
     Friday, March 9, 12
     

     View Slide

154. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     • The typical solution in a synchronous driver
     • hasNext: Boolean
     Friday, March 9, 12
     

     View Slide

155. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     • The typical solution in a synchronous driver
     • hasNext: Boolean
     • “Is the local buffer non-empty?” || “are there more
     results on the server?”
     Friday, March 9, 12
     

     View Slide

156. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     • The typical solution in a synchronous driver
     • hasNext: Boolean
     • “Is the local buffer non-empty?” || “are there more
     results on the server?”
     • next: A
     Friday, March 9, 12
     

     View Slide

157. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     • The typical solution in a synchronous driver
     • hasNext: Boolean
     • “Is the local buffer non-empty?” || “are there more
     results on the server?”
     • next: A
     • If non-empty local buffer, return item
     Friday, March 9, 12
     

     View Slide

158. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Now we have 3 states with a Cursor
     • The typical solution in a synchronous driver
     • hasNext: Boolean
     • “Is the local buffer non-empty?” || “are there more
     results on the server?”
     • next: A
     • If non-empty local buffer, return item
     • If more on server, call getMore (Smarter code may be
     “predictive” about this and prefetch ahead of need)
     Friday, March 9, 12
     

     View Slide

159. Problem #4: Multi-state resource iteration a.k.a. Cursors
     Friday, March 9, 12
     

     View Slide

160. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     Friday, March 9, 12
     

     View Slide

161. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     Friday, March 9, 12
     

     View Slide

162. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     • blocking for getMore will block all of the interleaved ops
     Friday, March 9, 12
     

     View Slide

163. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     • blocking for getMore will block all of the interleaved ops
     • I hit this problem with Hammersmith
     Friday, March 9, 12
     

     View Slide

164. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     • blocking for getMore will block all of the interleaved ops
     • I hit this problem with Hammersmith
     • It initially led to heavy drinking
     Friday, March 9, 12
     

     View Slide

165. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     • blocking for getMore will block all of the interleaved ops
     • I hit this problem with Hammersmith
     • It initially led to heavy drinking
     • John de Goes (@jdegoes) and Josh Suereth (@jsuereth)
     suggested Iteratees as a solution
     Friday, March 9, 12
     

     View Slide

166. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     • blocking for getMore will block all of the interleaved ops
     • I hit this problem with Hammersmith
     • It initially led to heavy drinking
     • John de Goes (@jdegoes) and Josh Suereth (@jsuereth)
     suggested Iteratees as a solution
     • Reading Haskell white papers and Scalaz code made my
     brain hurt...
     Friday, March 9, 12
     

     View Slide

167. Problem #4: Multi-state resource iteration a.k.a. Cursors
     • Working asynchronously, that block on getMore will put you in the
     weeds
     • Typically a small pool of threads are being reused for multiple
     incoming reads (in Netty you must NEVER block on a receiver thread)
     • blocking for getMore will block all of the interleaved ops
     • I hit this problem with Hammersmith
     • It initially led to heavy drinking
     • John de Goes (@jdegoes) and Josh Suereth (@jsuereth)
     suggested Iteratees as a solution
     • Reading Haskell white papers and Scalaz code made my
     brain hurt...
     • ... As such, what follows is my interpretation and any mistakes
     & stupidity are entirely my own
     Friday, March 9, 12
     

     View Slide

168. Better Living Through Iteratees
     Friday, March 9, 12
     

     View Slide

169. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     Friday, March 9, 12
     

     View Slide

170. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     • Introduce a higher order function
     Friday, March 9, 12
     

     View Slide

171. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     • Introduce a higher order function
     • Pass a function which takes an argument of “Iteration
     State”
     Friday, March 9, 12
     

     View Slide

172. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     • Introduce a higher order function
     • Pass a function which takes an argument of “Iteration
     State”
     • Return “Iteration Commands” based on the state
     Friday, March 9, 12
     

     View Slide

173. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     • Introduce a higher order function
     • Pass a function which takes an argument of “Iteration
     State”
     • Return “Iteration Commands” based on the state
     • Code is now asynchronous
     Friday, March 9, 12
     

     View Slide

174. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     • Introduce a higher order function
     • Pass a function which takes an argument of “Iteration
     State”
     • Return “Iteration Commands” based on the state
     • Code is now asynchronous
     • If the response to “No more on client, server has some”
     is “go get more”, no blocking I/O is needed
     Friday, March 9, 12
     

     View Slide

175. Better Living Through Iteratees
     • Instead of a potentially blocking next: A, with iteratees we can
     handle any number of states cleanly and asynchronously
     • Introduce a higher order function
     • Pass a function which takes an argument of “Iteration
     State”
     • Return “Iteration Commands” based on the state
     • Code is now asynchronous
     • If the response to “No more on client, server has some”
     is “go get more”, no blocking I/O is needed
     • Pass a copy of the current method with the “get more”
     command, iteration continues after buffer replenishment
     Friday, March 9, 12
     

     View Slide

176. Iteration “State” and “Command”
     trait IterState
     // “Here’s a valid entry”, have fun!
     case class Entry[T: SerializableBSONObject](doc: T) extends IterState
     // Client buffer empty, but more on server
     case object Empty extends IterState
     // Both client buffer and server are exhausted
     case object EOF extends IterState
     trait IterCmd
     // I’m all done with this cursor - clean it up, shut it down, take out the trash
     case object Done extends IterCmd
     // Go get me an item to work on ... here’s a function to handle all states
     case class Next(op: (IterState) 㱺 IterCmd) extends IterCmd
     // Call getMore & retrieve another batch - here’s a function to handle all states
     case class NextBatch(op: (IterState) 㱺 IterCmd) extends IterCmd
     Friday, March 9, 12
     

     View Slide

177. The “next” method on the Cursor Class
     def next() = try {
     log.trace("NEXT: %s ", decoder.getClass)
     if (docs.length > 0) Cursor.Entry(docs.dequeue()) else if (hasMore)
     Cursor.Empty
     else
     Cursor.EOF
     } catch { // just in case
     case nse: java.util.NoSuchElementException 㱺 {
     log.debug("No Such Element Exception")
     if (hasMore) {
     log.debug("Has More.")
     Cursor.Empty
     } else {
     log.debug("Cursor Exhausted.")
     Cursor.EOF
     }
     }
     }
     def iterate = Cursor.iterate(this) _
     Friday, March 9, 12
     

     View Slide

178. Iteration “Helper” function
     def iterate[T: SerializableBSONObject](cursor: Cursor[T])(op: (IterState) 㱺 IterCmd)
     {
     log.trace("Iterating '%s' with op: '%s'", cursor, op)
     def next(f: (IterState) 㱺 IterCmd): Unit = op(cursor.next()) match {
     case Done 㱺 {
     log.trace("Closing Cursor.")
     cursor.close()
     }
     case Next(tOp) 㱺 {
     log.trace("Next!")
     next(tOp)
     }
     case NextBatch(tOp) 㱺 cursor.nextBatch(() 㱺 {
     log.debug("Next Batch Loaded.")
     next(tOp)
     })
     }
     next(op)
     }
     Friday, March 9, 12
     

     View Slide

179. A Simple Iteration of a Cursor
     def iterateComplexCursor(conn: MongoConnection) = {
     var x = 0
     conn(integrationTestDBName).find("books")(Document.empty, Document.empty)((cursor:
     Cursor[Document]) 㱺 {
     def next(op: Cursor.IterState): Cursor.IterCmd = op match {
     case Cursor.Entry(doc) 㱺 {
     x += 1
     if (x < 100) Cursor.Next(next) else Cursor.Done
     }
     case Cursor.Empty 㱺 {
     if (x < 100) Cursor.NextBatch(next) else Cursor.Done
     }
     case Cursor.EOF 㱺 {
     Cursor.Done
     }
     }
     Cursor.iterate(cursor)(next)
     })
     x must eventually(5, 5.seconds)(be_==(100))
     }
     Friday, March 9, 12
     

     View Slide

180. Briefly: NIO.2 / AIO
     • JDK 7 introduces a higher level async API to NIO without
     going as high level as Netty does
     • NIO.2 / AIO brings in “AsynchronousSocketChannels”
     • Removes need to select / poll by hand
     • Configurable timeouts
     • Two options for how to get responses; both sanely map to
     Scala
     • java.util.concurrent.Future
     • CompletionHandler
     • Easily logically mapped to Either[E, T] with
     implicits
     • Probably not ‘prime time’ usable for library authors *yet* ...
     due to dependency on JDK7
     Friday, March 9, 12
     

     View Slide

181. @mongodb
     conferences, appearances, and meetups
     http://www.10gen.com/events
     http://bit.ly/mongofb
     Facebook | Twitter | LinkedIn
     http://linkd.in/joinmongo
     download at mongodb.org
     github.com/mongodb/casbah
     github.com/bwmcadams/hammersmith
     These slides will be online later at:
     http://speakerdeck.com/u/bwmcadams/
     brendan@10gen.com
     (twitter: @rit)
     Friday, March 9, 12
     

     View Slide