Tweet
Tweet

Slide 1

Slide 1 text

Brendan McAdams 10gen, Inc. brendan@10gen.com @rit Asynchronous + Non-Blocking Scala for Fun & Profit A look at Netty & NIO for Asynchronous networking via Scala Friday, March 9, 12

Slide 2

Slide 2 text

Goals Friday, March 9, 12

Slide 3

Slide 3 text

Goals • Some simple goals ... Friday, March 9, 12

Slide 4

Slide 4 text

Goals • Some simple goals ... • Stop wasting resources on blocking I/O Friday, March 9, 12

Slide 5

Slide 5 text

Goals • Some simple goals ... • Stop wasting resources on blocking I/O • Achieve C10K (10,000 simultaneous clients) Friday, March 9, 12

Slide 6

Slide 6 text

Goals • Some simple goals ... • Stop wasting resources on blocking I/O • Achieve C10K (10,000 simultaneous clients) • Profit Friday, March 9, 12

Slide 7

Slide 7 text

Explaining Non-Blocking & Asynchronous Friday, March 9, 12

Slide 8

Slide 8 text

Explaining Non-Blocking & Asynchronous • We are talking about I/O here Friday, March 9, 12

Slide 9

Slide 9 text

Explaining Non-Blocking & Asynchronous • We are talking about I/O here • By which, of course, we mean “Input / Output” Friday, March 9, 12

Slide 10

Slide 10 text

Explaining Non-Blocking & Asynchronous • We are talking about I/O here • By which, of course, we mean “Input / Output” • We’ll focus on networking I/O Friday, March 9, 12

Slide 11

Slide 11 text

Explaining Non-Blocking & Asynchronous Friday, March 9, 12

Slide 12

Slide 12 text

Explaining Non-Blocking & Asynchronous •Asynchronous describes a way to accomplish Non-Blocking I/O Friday, March 9, 12

Slide 13

Slide 13 text

Explaining Non-Blocking & Asynchronous •Asynchronous describes a way to accomplish Non-Blocking I/O • I like to think of these as a mini-stack, with Async on top of Non-blocking Friday, March 9, 12

Slide 14

Slide 14 text

Explaining Non-Blocking & Asynchronous Friday, March 9, 12

Slide 15

Slide 15 text

Explaining Non-Blocking & Asynchronous •Blocking presents a series of problems: Friday, March 9, 12

Slide 16

Slide 16 text

Explaining Non-Blocking & Asynchronous •Blocking presents a series of problems: • When a blocking I/O operation occurs, everything in that thread halts and waits; potentially idle system resources Friday, March 9, 12

Slide 17

Slide 17 text

Explaining Non-Blocking & Asynchronous •Blocking presents a series of problems: • When a blocking I/O operation occurs, everything in that thread halts and waits; potentially idle system resources • Internally, that “blocking” operation is a big loop per operation asking “Are we there yet?” Friday, March 9, 12

Slide 18

Slide 18 text

Explaining Non-Blocking & Asynchronous •Blocking presents a series of problems: • When a blocking I/O operation occurs, everything in that thread halts and waits; potentially idle system resources • Internally, that “blocking” operation is a big loop per operation asking “Are we there yet?” • “Idleness” occurs because the thread waiting for I/O to complete is doing nothing while it waits Friday, March 9, 12

Slide 19

Slide 19 text

Explaining Non-Blocking & Asynchronous •Blocking presents a series of problems: • When a blocking I/O operation occurs, everything in that thread halts and waits; potentially idle system resources • Internally, that “blocking” operation is a big loop per operation asking “Are we there yet?” • “Idleness” occurs because the thread waiting for I/O to complete is doing nothing while it waits • This can vastly limit our ability to scale Friday, March 9, 12

Slide 20

Slide 20 text

Explaining Non-Blocking & Asynchronous Friday, March 9, 12

Slide 21

Slide 21 text

Explaining Non-Blocking & Asynchronous • Non-Blocking I/O presents solutions: Friday, March 9, 12

Slide 22

Slide 22 text

Explaining Non-Blocking & Asynchronous • Non-Blocking I/O presents solutions: • Eschew blocking individually on each operation Friday, March 9, 12

Slide 23

Slide 23 text

Explaining Non-Blocking & Asynchronous • Non-Blocking I/O presents solutions: • Eschew blocking individually on each operation • Find ways to work with the kernel to more efficiently manage multiple blocking resources in groups Friday, March 9, 12

Slide 24

Slide 24 text

Explaining Non-Blocking & Asynchronous • Non-Blocking I/O presents solutions: • Eschew blocking individually on each operation • Find ways to work with the kernel to more efficiently manage multiple blocking resources in groups • Free up threads which are no longer blocked waiting on I/O to handle other requests while those requests waiting on I/O idle Friday, March 9, 12

Slide 25

Slide 25 text

Where does Asynchronous come in? Friday, March 9, 12

Slide 26

Slide 26 text

Where does Asynchronous come in? • If we are no longer blocking and instead reusing threads while I/O waits, we need a way to handle “completion” events Friday, March 9, 12

Slide 27

Slide 27 text

Where does Asynchronous come in? • If we are no longer blocking and instead reusing threads while I/O waits, we need a way to handle “completion” events • Asynchronous techniques (such as callbacks) allow us to achieve this Friday, March 9, 12

Slide 28

Slide 28 text

Where does Asynchronous come in? • If we are no longer blocking and instead reusing threads while I/O waits, we need a way to handle “completion” events • Asynchronous techniques (such as callbacks) allow us to achieve this • “Step to the side of the line, and we’ll call you when your order is ready” Friday, March 9, 12

Slide 29

Slide 29 text

Breaking it Down Friday, March 9, 12

Slide 30

Slide 30 text

Breaking it Down • The synchronous/blocking I/O model typically forces us into some paradigms Friday, March 9, 12

Slide 31

Slide 31 text

Breaking it Down • The synchronous/blocking I/O model typically forces us into some paradigms • Servers Friday, March 9, 12

Slide 32

Slide 32 text

Breaking it Down • The synchronous/blocking I/O model typically forces us into some paradigms • Servers • A 1:1 ratio of threads to client connections (scale limited) Friday, March 9, 12

Slide 33

Slide 33 text

Breaking it Down • The synchronous/blocking I/O model typically forces us into some paradigms • Servers • A 1:1 ratio of threads to client connections (scale limited) • Clients Friday, March 9, 12

Slide 34

Slide 34 text

Breaking it Down • The synchronous/blocking I/O model typically forces us into some paradigms • Servers • A 1:1 ratio of threads to client connections (scale limited) • Clients • Connection pools often larger than necessary to account for unavailable-while-blocking connection resources Friday, March 9, 12

Slide 35

Slide 35 text

Breaking it Down Friday, March 9, 12

Slide 36

Slide 36 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny Friday, March 9, 12

Slide 37

Slide 37 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers Friday, March 9, 12

Slide 38

Slide 38 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) Friday, March 9, 12

Slide 39

Slide 39 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) • A thread waiting for an I/O response no longer needs to block it, allowing other threads to reuse that connection simultaneously Friday, March 9, 12

Slide 40

Slide 40 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) • A thread waiting for an I/O response no longer needs to block it, allowing other threads to reuse that connection simultaneously • Operations such as a “write” can queue up until resource is ready, returning thread of execution immediately, callback results later. Friday, March 9, 12

Slide 41

Slide 41 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) • A thread waiting for an I/O response no longer needs to block it, allowing other threads to reuse that connection simultaneously • Operations such as a “write” can queue up until resource is ready, returning thread of execution immediately, callback results later. • Of course, this makes dispatch and good concurrency even tougher (Who said doing things right was ever easy?) Friday, March 9, 12

Slide 42

Slide 42 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) • A thread waiting for an I/O response no longer needs to block it, allowing other threads to reuse that connection simultaneously • Operations such as a “write” can queue up until resource is ready, returning thread of execution immediately, callback results later. • Of course, this makes dispatch and good concurrency even tougher (Who said doing things right was ever easy?) • Clients Friday, March 9, 12

Slide 43

Slide 43 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) • A thread waiting for an I/O response no longer needs to block it, allowing other threads to reuse that connection simultaneously • Operations such as a “write” can queue up until resource is ready, returning thread of execution immediately, callback results later. • Of course, this makes dispatch and good concurrency even tougher (Who said doing things right was ever easy?) • Clients • Significantly reduce pool sizes Friday, March 9, 12

Slide 44

Slide 44 text

Breaking it Down • If we go asynchronous/non-blocking we can change our destiny • Servers • : Ratio becomes possible (Scale - C10k and beyond) • A thread waiting for an I/O response no longer needs to block it, allowing other threads to reuse that connection simultaneously • Operations such as a “write” can queue up until resource is ready, returning thread of execution immediately, callback results later. • Of course, this makes dispatch and good concurrency even tougher (Who said doing things right was ever easy?) • Clients • Significantly reduce pool sizes • connection resources can be leveraged by many more simultaneous threads Friday, March 9, 12

Slide 45

Slide 45 text

Introducing NIO Friday, March 9, 12

Slide 46

Slide 46 text

Introducing NIO • History Friday, March 9, 12

Slide 47

Slide 47 text

Introducing NIO • History • “New I/O”, introduced in Java 1.4 Friday, March 9, 12

Slide 48

Slide 48 text

Introducing NIO • History • “New I/O”, introduced in Java 1.4 • Focus on low-level I/O as opposed to “Old” I/O’s high level-API Friday, March 9, 12

Slide 49

Slide 49 text

Introducing NIO • History • “New I/O”, introduced in Java 1.4 • Focus on low-level I/O as opposed to “Old” I/O’s high level-API • ByteBuffers: http://www.kdgregory.com/index.php?page=java.byteBuffer Friday, March 9, 12

Slide 50

Slide 50 text

Introducing NIO Friday, March 9, 12

Slide 51

Slide 51 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write Friday, March 9, 12

Slide 52

Slide 52 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write • Callback when ready Friday, March 9, 12

Slide 53

Slide 53 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write • Callback when ready • Core units of work: Buffer, Channel and Selector Friday, March 9, 12

Slide 54

Slide 54 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write • Callback when ready • Core units of work: Buffer, Channel and Selector • Buffer are contiguous memory slots, offering data transfer operations Friday, March 9, 12

Slide 55

Slide 55 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write • Callback when ready • Core units of work: Buffer, Channel and Selector • Buffer are contiguous memory slots, offering data transfer operations • Channel instances are “bulk” data wrappers to Buffer Friday, March 9, 12

Slide 56

Slide 56 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write • Callback when ready • Core units of work: Buffer, Channel and Selector • Buffer are contiguous memory slots, offering data transfer operations • Channel instances are “bulk” data wrappers to Buffer • Selector is an event monitor which can watch multiple Channel instances in a single thread (the kernel coordinator) Friday, March 9, 12

Slide 57

Slide 57 text

Introducing NIO • For working with Networks, we must manually work with Selector, and request a window to read and write • Callback when ready • Core units of work: Buffer, Channel and Selector • Buffer are contiguous memory slots, offering data transfer operations • Channel instances are “bulk” data wrappers to Buffer • Selector is an event monitor which can watch multiple Channel instances in a single thread (the kernel coordinator) • Relocate the task of checking I/O status out of the “execution” threads Friday, March 9, 12

Slide 58

Slide 58 text

NIO in Practice Friday, March 9, 12

Slide 59

Slide 59 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here Friday, March 9, 12

Slide 60

Slide 60 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample Friday, March 9, 12

Slide 61

Slide 61 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know Friday, March 9, 12

Slide 62

Slide 62 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) Friday, March 9, 12

Slide 63

Slide 63 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events Friday, March 9, 12

Slide 64

Slide 64 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) Friday, March 9, 12

Slide 65

Slide 65 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) • Want to write? Tell the Selector you want to write Friday, March 9, 12

Slide 66

Slide 66 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) • Want to write? Tell the Selector you want to write • Eventually, when the Channel is available to write, an event will notify the Selector Friday, March 9, 12

Slide 67

Slide 67 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) • Want to write? Tell the Selector you want to write • Eventually, when the Channel is available to write, an event will notify the Selector • Remove “interested in writing” status Friday, March 9, 12

Slide 68

Slide 68 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) • Want to write? Tell the Selector you want to write • Eventually, when the Channel is available to write, an event will notify the Selector • Remove “interested in writing” status • Dispatch “time to write” to original caller Friday, March 9, 12

Slide 69

Slide 69 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) • Want to write? Tell the Selector you want to write • Eventually, when the Channel is available to write, an event will notify the Selector • Remove “interested in writing” status • Dispatch “time to write” to original caller • Write Friday, March 9, 12

Slide 70

Slide 70 text

NIO in Practice • I kicked back & forth with a few possible examples of NIO here • Conclusion: for time & sanity, omit a code sample • Here’s what you need to know • Register “Interests” with a Selector (Read, Write, etc) • Write a Selector loop which checks for notification events • Dispatch incoming events (such as “read”) • Want to write? Tell the Selector you want to write • Eventually, when the Channel is available to write, an event will notify the Selector • Remove “interested in writing” status • Dispatch “time to write” to original caller • Write • Rinse, repeat. Friday, March 9, 12

Slide 71

Slide 71 text

Introducing Netty Friday, March 9, 12

Slide 72

Slide 72 text

Introducing Netty • Simplify NIO (but still provides access to oio) Friday, March 9, 12

Slide 73

Slide 73 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction Friday, March 9, 12

Slide 74

Slide 74 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away Friday, March 9, 12

Slide 75

Slide 75 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output Friday, March 9, 12

Slide 76

Slide 76 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output • ChannelBuffers Friday, March 9, 12

Slide 77

Slide 77 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output • ChannelBuffers • Can composite multiple ChannelBuffers Friday, March 9, 12

Slide 78

Slide 78 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output • ChannelBuffers • Can composite multiple ChannelBuffers • Organize individual pieces in one composite buffer Friday, March 9, 12

Slide 79

Slide 79 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output • ChannelBuffers • Can composite multiple ChannelBuffers • Organize individual pieces in one composite buffer • Supports ByteBuffer, Arrays, etc Friday, March 9, 12

Slide 80

Slide 80 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output • ChannelBuffers • Can composite multiple ChannelBuffers • Organize individual pieces in one composite buffer • Supports ByteBuffer, Arrays, etc • Avoid memory copy as much as possible Friday, March 9, 12

Slide 81

Slide 81 text

Introducing Netty • Simplify NIO (but still provides access to oio) • Really, just wrapping NIO to provide higher level abstraction • Hides the Selector nonsense away • Composable “Filter Pipeline” allows you to intercept multiple levels of input and output • ChannelBuffers • Can composite multiple ChannelBuffers • Organize individual pieces in one composite buffer • Supports ByteBuffer, Arrays, etc • Avoid memory copy as much as possible • Direct Memory allocated (rumors of memory leaks abound) Friday, March 9, 12

Slide 82

Slide 82 text

Pipelines == Sanity val channelFactory = new NioClientSocketChannelFactory(Executors.newCachedThreadPool(ThreadFactories("Hammersmith Netty Boss")), Executors.newCachedThreadPool(ThreadFactories("Hammersmith Netty Worker"))) protected implicit val bootstrap = new ClientBootstrap(channelFactory) bootstrap.setPipelineFactory(new ChannelPipelineFactory() { /* Big long, epic + awesomely insightful @havocp comment */ private val appCallbackExecutor = new ThreadPoolExecutor(/** constructor snipped for slide sanity */) private val appCallbackExecutionHandler = new ExecutionHandler(appCallbackExecutor) def getPipeline = { val p = Channels.pipeline(new ReplyMessageDecoder(), appCallbackExecutionHandler, handler) p } }) bootstrap.setOption("remoteAddress", addr) private val _f = bootstrap.connect() protected implicit val channel = _f.awaitUninterruptibly.getChannel Friday, March 9, 12

Slide 83

Slide 83 text

CAN HAS NETWORK WRITE? // It turns out writing can be easy... channel.write(outStream.buffer()) Friday, March 9, 12

Slide 84

Slide 84 text

Stupid Pet Tricks Friday, March 9, 12

Slide 85

Slide 85 text

Every good story can use a MacGuffin... Friday, March 9, 12

Slide 86

Slide 86 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) Friday, March 9, 12

Slide 87

Slide 87 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story Friday, March 9, 12

Slide 88

Slide 88 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server Friday, March 9, 12

Slide 89

Slide 89 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server • This is a project I’ve already spent a good bit of time on – Hammersmith Friday, March 9, 12

Slide 90

Slide 90 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server • This is a project I’ve already spent a good bit of time on – Hammersmith • A few focal points to lead our discussion Friday, March 9, 12

Slide 91

Slide 91 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server • This is a project I’ve already spent a good bit of time on – Hammersmith • A few focal points to lead our discussion • Decoding & dispatching inbound messages Friday, March 9, 12

Slide 92

Slide 92 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server • This is a project I’ve already spent a good bit of time on – Hammersmith • A few focal points to lead our discussion • Decoding & dispatching inbound messages • Handling errors & exceptions across threads, time, and space Friday, March 9, 12

Slide 93

Slide 93 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server • This is a project I’ve already spent a good bit of time on – Hammersmith • A few focal points to lead our discussion • Decoding & dispatching inbound messages • Handling errors & exceptions across threads, time, and space • “Follow up” operations which rely on serverside “same connection” context Friday, March 9, 12

Slide 94

Slide 94 text

Every good story can use a MacGuffin... • MacGuffin (n) “A plot element that catches the viewers’ attention or drives the plot of a work of fiction” (Sometimes “maguffin” or “McGuffin” as well) • We aren’t writing fiction here, but we can use a MacGuffin to drive our story • For our MacGuffin, we’ll examine Asynchronous networking against a MongoDB Server • This is a project I’ve already spent a good bit of time on – Hammersmith • A few focal points to lead our discussion • Decoding & dispatching inbound messages • Handling errors & exceptions across threads, time, and space • “Follow up” operations which rely on serverside “same connection” context • Working with multi-state iterations which depend on IO for operations... a.k.a. “Database Cursors” Friday, March 9, 12

Slide 95

Slide 95 text

Problem #1: Decoding & Dispatching Reads Friday, March 9, 12

Slide 96

Slide 96 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes Friday, March 9, 12

Slide 97

Slide 97 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol Friday, March 9, 12

Slide 98

Slide 98 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol • The kernel reads things off the network into a big buffer of bytes Friday, March 9, 12

Slide 99

Slide 99 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol • The kernel reads things off the network into a big buffer of bytes • It’s up to us to figure out what parts of the bytes are relevant where Friday, March 9, 12

Slide 100

Slide 100 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol • The kernel reads things off the network into a big buffer of bytes • It’s up to us to figure out what parts of the bytes are relevant where • Challenge: Separate out individual writes and send them to the right place Friday, March 9, 12

Slide 101

Slide 101 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol • The kernel reads things off the network into a big buffer of bytes • It’s up to us to figure out what parts of the bytes are relevant where • Challenge: Separate out individual writes and send them to the right place • Conceptually, “Law of Demeter” (loose coupling) helps here. Doing it by hand you have to be careful not to eat somebody else’s lunch Friday, March 9, 12

Slide 102

Slide 102 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol • The kernel reads things off the network into a big buffer of bytes • It’s up to us to figure out what parts of the bytes are relevant where • Challenge: Separate out individual writes and send them to the right place • Conceptually, “Law of Demeter” (loose coupling) helps here. Doing it by hand you have to be careful not to eat somebody else’s lunch • NIO leaves you on your own Friday, March 9, 12

Slide 103

Slide 103 text

Problem #1: Decoding & Dispatching Reads • Packets aren’t bytes • Network layers don’t know or care about your fancy application layer protocol • The kernel reads things off the network into a big buffer of bytes • It’s up to us to figure out what parts of the bytes are relevant where • Challenge: Separate out individual writes and send them to the right place • Conceptually, “Law of Demeter” (loose coupling) helps here. Doing it by hand you have to be careful not to eat somebody else’s lunch • NIO leaves you on your own • Netty’s pipeline helps provide the “don’t eat my lunch” fix quite well IMHO Friday, March 9, 12

Slide 104

Slide 104 text

In Netty, add a decoder to the Pipeline protected[mongodb] class ReplyMessageDecoder extends LengthFieldBasedFrameDecoder(1024 * 1024 * 4, 0, 4, -4, 0) with Logging { protected override def decode(ctx: ChannelHandlerContext, channel: Channel, buffer: ChannelBuffer): AnyRef = { val frame = super.decode(ctx, channel, buffer).asInstanceOf[ChannelBuffer] if (frame == null) { // don't have the whole message yet; netty will retry later null } else { // we have one message (and nothing else) in the "frame" buffer MongoMessage.unapply(new ChannelBufferInputStream(frame)) match { case reply: ReplyMessage 㱺 reply case default 㱺 // this should not happen; throw new Exception("Unknown message type '%s' incoming from MongoDB; ignoring.".format(default)) } } } Friday, March 9, 12

Slide 105

Slide 105 text

In Netty, add a decoder to the Pipeline // Because we return a new object rather than a buffer from decode(), // we can use slice() here according to the docs (the slice won't escape // the decode() method so it will be valid while we're using it) protected override def extractFrame(buffer: ChannelBuffer, index: Int, length: Int): ChannelBuffer = { return buffer.slice(index, length); } } Friday, March 9, 12

Slide 106

Slide 106 text

Expect the pipeline to have a MongoMessage override def messageReceived(ctx: ChannelHandlerContext, e: MessageEvent) { val message = e.getMessage.asInstanceOf[MongoMessage] log.debug("Incoming Message received type %s", message.getClass.getName) message match { case reply: ReplyMessage 㱺 { Friday, March 9, 12

Slide 107

Slide 107 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors Friday, March 9, 12

Slide 108

Slide 108 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated Friday, March 9, 12

Slide 109

Slide 109 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated • Scala has a great construct to help with this: Either[L, R] Friday, March 9, 12

Slide 110

Slide 110 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated • Scala has a great construct to help with this: Either[L, R] • Pass a monad that can have one of two states: Failure or Success Friday, March 9, 12

Slide 111

Slide 111 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated • Scala has a great construct to help with this: Either[L, R] • Pass a monad that can have one of two states: Failure or Success • By convention, “Left” is an Error, “Right” is success Friday, March 9, 12

Slide 112

Slide 112 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated • Scala has a great construct to help with this: Either[L, R] • Pass a monad that can have one of two states: Failure or Success • By convention, “Left” is an Error, “Right” is success • Node does a similar passing of Success vs. Result Friday, March 9, 12

Slide 113

Slide 113 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated • Scala has a great construct to help with this: Either[L, R] • Pass a monad that can have one of two states: Failure or Success • By convention, “Left” is an Error, “Right” is success • Node does a similar passing of Success vs. Result • No special “different” handling in Netty vs. NIO Friday, March 9, 12

Slide 114

Slide 114 text

Problem #2: Handling Errors, Exceptions and Eldritch Horrors • Asynchronous and callback nature makes dispatching errors difficult: the “throw / catch” model becomes complicated • Scala has a great construct to help with this: Either[L, R] • Pass a monad that can have one of two states: Failure or Success • By convention, “Left” is an Error, “Right” is success • Node does a similar passing of Success vs. Result • No special “different” handling in Netty vs. NIO • Implicit tricks for the lazy who want to just write a “success” block Friday, March 9, 12

Slide 115

Slide 115 text

Yes, I wrote my own Futures. It was an accident... sealed trait RequestFuture { type T val body: Either[Throwable, T] 㱺 Unit def apply(error: Throwable) = body(Left(error)) def apply[A <% T](result: A) = body(Right(result.asInstanceOf[T])) protected[futures] var completed = false } /** * Will pass any *generated* _id along with any relevant getLastError information * For an update, don't expect to get ObjectId */ trait WriteRequestFuture extends RequestFuture { type T <: (Option[AnyRef] /* ID Type */ , WriteResult) } implicit def asWriteOp(f: Either[Throwable, (Option[AnyRef], WriteResult)] 㱺 Unit) = RequestFutures.write(f) Friday, March 9, 12

Slide 116

Slide 116 text

Did we succeed, or did we fail? val handler = RequestFutures.write((result: Either[Throwable, (Option[AnyRef], WriteResult)]) 㱺 { result match { case Right((oid, wr)) 㱺 { ok = Some(true) id = oid } case Left(t) 㱺 { ok = Some(false) log.error(t, "Command Failed.") } } }) mongo.insert(Document("foo" -> "bar", "bar" -> "baz"))(handler) Friday, March 9, 12

Slide 117

Slide 117 text

Errors? We don’t need no stinkin’ errors... implicit def asSimpleWriteOp(f: (Option[AnyRef], WriteResult) 㱺 Unit): WriteRequestFuture = SimpleRequestFutures.write(f) def write(f: (Option[AnyRef], WriteResult) 㱺 Unit) = new WriteRequestFuture { val body = (result: Either[Throwable, (Option[AnyRef], WriteResult)]) 㱺 result match { case Right((oid, wr)) 㱺 f(oid, wr) case Left(t) 㱺 log.error(t, "Command Failed.") } override def toString = "{SimpleWriteRequestFuture}" } Friday, March 9, 12

Slide 118

Slide 118 text

Problem #3: “Same Connection” Follow Up Operations Friday, March 9, 12

Slide 119

Slide 119 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write Friday, March 9, 12

Slide 120

Slide 120 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement Friday, March 9, 12

Slide 121

Slide 121 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) Friday, March 9, 12

Slide 122

Slide 122 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) • Somewhat easy in a synchronous framework Friday, March 9, 12

Slide 123

Slide 123 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) • Somewhat easy in a synchronous framework • Lock the connection out of the pool and keep it private Friday, March 9, 12

Slide 124

Slide 124 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) • Somewhat easy in a synchronous framework • Lock the connection out of the pool and keep it private • Don’t let anyone else touch it until you’re done Friday, March 9, 12

Slide 125

Slide 125 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) • Somewhat easy in a synchronous framework • Lock the connection out of the pool and keep it private • Don’t let anyone else touch it until you’re done • In Async, harder Friday, March 9, 12

Slide 126

Slide 126 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) • Somewhat easy in a synchronous framework • Lock the connection out of the pool and keep it private • Don’t let anyone else touch it until you’re done • In Async, harder • Only solution I’ve found is “ballot box stuffing” Friday, March 9, 12

Slide 127

Slide 127 text

Problem #3: “Same Connection” Follow Up Operations • Some databases, etc. have contextual operations as “follow ups” to a write, which can only be called on the same connection as the write • MySQL has last_insert_id() to fetch generated autoincrement • MongoDB has getLastError() to check success/failure of a write (and explicitly specify consistency requirements) • Somewhat easy in a synchronous framework • Lock the connection out of the pool and keep it private • Don’t let anyone else touch it until you’re done • In Async, harder • Only solution I’ve found is “ballot box stuffing” • Deliberate reversal of the “decoding problem” Friday, March 9, 12

Slide 128

Slide 128 text

Stuffing the Ballot Box // Quick callback when needed to be invoked immediately after write val writeCB: () 㱺 Unit = if (isWrite) { msg match { case wMsg: MongoClientWriteMessage 㱺 if (concern.safe_?) { val gle = createCommand(wMsg.namespace.split("\\.")(0), Document("getlasterror" - > 1)) dispatcher.put(gle.requestID, CompletableRequest(msg, f)) gle.write(outStream) () 㱺 {} } else () 㱺 { wMsg.ids.foreach(x 㱺 f((x, WriteResult(true)).asInstanceOf[f.T])) } case unknown 㱺 { val e = new IllegalArgumentException("Invalid type of message passed; WriteRequestFutures expect a MongoClientWriteMessage underneath them. Got " + unknown) () 㱺 { f(e) } } } } else () 㱺 {} Friday, March 9, 12

Slide 129

Slide 129 text

Problem #4: Multi-state resource iteration a.k.a. Cursors Friday, March 9, 12

Slide 130

Slide 130 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad Friday, March 9, 12

Slide 131

Slide 131 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad • Two primary calls on an Iterator[A] Friday, March 9, 12

Slide 132

Slide 132 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad • Two primary calls on an Iterator[A] • hasNext: Boolean Friday, March 9, 12

Slide 133

Slide 133 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad • Two primary calls on an Iterator[A] • hasNext: Boolean • next(): A Friday, March 9, 12

Slide 134

Slide 134 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad • Two primary calls on an Iterator[A] • hasNext: Boolean • next(): A • In a pure and simple form, the Iterator[A] is prepopulated with all of its elements. Friday, March 9, 12

Slide 135

Slide 135 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad • Two primary calls on an Iterator[A] • hasNext: Boolean • next(): A • In a pure and simple form, the Iterator[A] is prepopulated with all of its elements. • If the buffer is non-empty, hasNext == true and next() returns another element. Friday, March 9, 12

Slide 136

Slide 136 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In typical iteration, we are working with a dual-state monad • Two primary calls on an Iterator[A] • hasNext: Boolean • next(): A • In a pure and simple form, the Iterator[A] is prepopulated with all of its elements. • If the buffer is non-empty, hasNext == true and next() returns another element. • When the buffer is empty, iteration halts completely because there’s nothing left to iterate. hasNext == false, next() returns null, throws an exception or similar. Friday, March 9, 12

Slide 137

Slide 137 text

Problem #4: Multi-state resource iteration a.k.a. Cursors Friday, March 9, 12

Slide 138

Slide 138 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] Friday, March 9, 12

Slide 139

Slide 139 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] • This maps nice and simply to the Iterator[A] monad and everyone can chug along happily, not caring if I/O is synchronous or asynchronous Friday, March 9, 12

Slide 140

Slide 140 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] • This maps nice and simply to the Iterator[A] monad and everyone can chug along happily, not caring if I/O is synchronous or asynchronous • In reality though, forcing a client to buffer all of the results to a large query is tremendously inefficient Friday, March 9, 12

Slide 141

Slide 141 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] • This maps nice and simply to the Iterator[A] monad and everyone can chug along happily, not caring if I/O is synchronous or asynchronous • In reality though, forcing a client to buffer all of the results to a large query is tremendously inefficient • Do you have enough memory on the client side for the entire result set? Friday, March 9, 12

Slide 142

Slide 142 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] • This maps nice and simply to the Iterator[A] monad and everyone can chug along happily, not caring if I/O is synchronous or asynchronous • In reality though, forcing a client to buffer all of the results to a large query is tremendously inefficient • Do you have enough memory on the client side for the entire result set? • With async, we may have a lot of potentially large result sets buffered Friday, March 9, 12

Slide 143

Slide 143 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] • This maps nice and simply to the Iterator[A] monad and everyone can chug along happily, not caring if I/O is synchronous or asynchronous • In reality though, forcing a client to buffer all of the results to a large query is tremendously inefficient • Do you have enough memory on the client side for the entire result set? • With async, we may have a lot of potentially large result sets buffered • Many databases (MongoDB, MySQL, Oracle, etc) use a multi-state result known as a “cursor” Friday, March 9, 12

Slide 144

Slide 144 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • In a simple database, a query would return a batch of all of the query results, populating an Iterator[DBRow] • This maps nice and simply to the Iterator[A] monad and everyone can chug along happily, not caring if I/O is synchronous or asynchronous • In reality though, forcing a client to buffer all of the results to a large query is tremendously inefficient • Do you have enough memory on the client side for the entire result set? • With async, we may have a lot of potentially large result sets buffered • Many databases (MongoDB, MySQL, Oracle, etc) use a multi-state result known as a “cursor” • Let the server buffer memory and chunk up batches Friday, March 9, 12

Slide 145

Slide 145 text

Problem #4: Multi-state resource iteration a.k.a. Cursors Friday, March 9, 12

Slide 146

Slide 146 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Cursors will return an initial batch of results Friday, March 9, 12

Slide 147

Slide 147 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Cursors will return an initial batch of results • If there are more results available on the server a “Cursor ID” is returned w/ the batch Friday, March 9, 12

Slide 148

Slide 148 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Cursors will return an initial batch of results • If there are more results available on the server a “Cursor ID” is returned w/ the batch • Client can use getMore to fetch additional batches until server results are exhausted Friday, March 9, 12

Slide 149

Slide 149 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Cursors will return an initial batch of results • If there are more results available on the server a “Cursor ID” is returned w/ the batch • Client can use getMore to fetch additional batches until server results are exhausted • Once exhausted, getMore will return a batch and a Cursor ID of 0 (indicating “no more results”) Friday, March 9, 12

Slide 150

Slide 150 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Cursors will return an initial batch of results • If there are more results available on the server a “Cursor ID” is returned w/ the batch • Client can use getMore to fetch additional batches until server results are exhausted • Once exhausted, getMore will return a batch and a Cursor ID of 0 (indicating “no more results”) • Try doing this cleanly without blocking... Friday, March 9, 12

Slide 151

Slide 151 text

Problem #4: Multi-state resource iteration a.k.a. Cursors Friday, March 9, 12

Slide 152

Slide 152 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor Friday, March 9, 12

Slide 153

Slide 153 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor • The typical solution in a synchronous driver Friday, March 9, 12

Slide 154

Slide 154 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor • The typical solution in a synchronous driver • hasNext: Boolean Friday, March 9, 12

Slide 155

Slide 155 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor • The typical solution in a synchronous driver • hasNext: Boolean • “Is the local buffer non-empty?” || “are there more results on the server?” Friday, March 9, 12

Slide 156

Slide 156 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor • The typical solution in a synchronous driver • hasNext: Boolean • “Is the local buffer non-empty?” || “are there more results on the server?” • next: A Friday, March 9, 12

Slide 157

Slide 157 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor • The typical solution in a synchronous driver • hasNext: Boolean • “Is the local buffer non-empty?” || “are there more results on the server?” • next: A • If non-empty local buffer, return item Friday, March 9, 12

Slide 158

Slide 158 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Now we have 3 states with a Cursor • The typical solution in a synchronous driver • hasNext: Boolean • “Is the local buffer non-empty?” || “are there more results on the server?” • next: A • If non-empty local buffer, return item • If more on server, call getMore (Smarter code may be “predictive” about this and prefetch ahead of need) Friday, March 9, 12

Slide 159

Slide 159 text

Problem #4: Multi-state resource iteration a.k.a. Cursors Friday, March 9, 12

Slide 160

Slide 160 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds Friday, March 9, 12

Slide 161

Slide 161 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) Friday, March 9, 12

Slide 162

Slide 162 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) • blocking for getMore will block all of the interleaved ops Friday, March 9, 12

Slide 163

Slide 163 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) • blocking for getMore will block all of the interleaved ops • I hit this problem with Hammersmith Friday, March 9, 12

Slide 164

Slide 164 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) • blocking for getMore will block all of the interleaved ops • I hit this problem with Hammersmith • It initially led to heavy drinking Friday, March 9, 12

Slide 165

Slide 165 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) • blocking for getMore will block all of the interleaved ops • I hit this problem with Hammersmith • It initially led to heavy drinking • John de Goes (@jdegoes) and Josh Suereth (@jsuereth) suggested Iteratees as a solution Friday, March 9, 12

Slide 166

Slide 166 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) • blocking for getMore will block all of the interleaved ops • I hit this problem with Hammersmith • It initially led to heavy drinking • John de Goes (@jdegoes) and Josh Suereth (@jsuereth) suggested Iteratees as a solution • Reading Haskell white papers and Scalaz code made my brain hurt... Friday, March 9, 12

Slide 167

Slide 167 text

Problem #4: Multi-state resource iteration a.k.a. Cursors • Working asynchronously, that block on getMore will put you in the weeds • Typically a small pool of threads are being reused for multiple incoming reads (in Netty you must NEVER block on a receiver thread) • blocking for getMore will block all of the interleaved ops • I hit this problem with Hammersmith • It initially led to heavy drinking • John de Goes (@jdegoes) and Josh Suereth (@jsuereth) suggested Iteratees as a solution • Reading Haskell white papers and Scalaz code made my brain hurt... • ... As such, what follows is my interpretation and any mistakes & stupidity are entirely my own Friday, March 9, 12

Slide 168

Slide 168 text

Better Living Through Iteratees Friday, March 9, 12

Slide 169

Slide 169 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously Friday, March 9, 12

Slide 170

Slide 170 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously • Introduce a higher order function Friday, March 9, 12

Slide 171

Slide 171 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously • Introduce a higher order function • Pass a function which takes an argument of “Iteration State” Friday, March 9, 12

Slide 172

Slide 172 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously • Introduce a higher order function • Pass a function which takes an argument of “Iteration State” • Return “Iteration Commands” based on the state Friday, March 9, 12

Slide 173

Slide 173 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously • Introduce a higher order function • Pass a function which takes an argument of “Iteration State” • Return “Iteration Commands” based on the state • Code is now asynchronous Friday, March 9, 12

Slide 174

Slide 174 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously • Introduce a higher order function • Pass a function which takes an argument of “Iteration State” • Return “Iteration Commands” based on the state • Code is now asynchronous • If the response to “No more on client, server has some” is “go get more”, no blocking I/O is needed Friday, March 9, 12

Slide 175

Slide 175 text

Better Living Through Iteratees • Instead of a potentially blocking next: A, with iteratees we can handle any number of states cleanly and asynchronously • Introduce a higher order function • Pass a function which takes an argument of “Iteration State” • Return “Iteration Commands” based on the state • Code is now asynchronous • If the response to “No more on client, server has some” is “go get more”, no blocking I/O is needed • Pass a copy of the current method with the “get more” command, iteration continues after buffer replenishment Friday, March 9, 12

Slide 176

Slide 176 text

Iteration “State” and “Command” trait IterState // “Here’s a valid entry”, have fun! case class Entry[T: SerializableBSONObject](doc: T) extends IterState // Client buffer empty, but more on server case object Empty extends IterState // Both client buffer and server are exhausted case object EOF extends IterState trait IterCmd // I’m all done with this cursor - clean it up, shut it down, take out the trash case object Done extends IterCmd // Go get me an item to work on ... here’s a function to handle all states case class Next(op: (IterState) 㱺 IterCmd) extends IterCmd // Call getMore & retrieve another batch - here’s a function to handle all states case class NextBatch(op: (IterState) 㱺 IterCmd) extends IterCmd Friday, March 9, 12

Slide 177

Slide 177 text

The “next” method on the Cursor Class def next() = try { log.trace("NEXT: %s ", decoder.getClass) if (docs.length > 0) Cursor.Entry(docs.dequeue()) else if (hasMore) Cursor.Empty else Cursor.EOF } catch { // just in case case nse: java.util.NoSuchElementException 㱺 { log.debug("No Such Element Exception") if (hasMore) { log.debug("Has More.") Cursor.Empty } else { log.debug("Cursor Exhausted.") Cursor.EOF } } } def iterate = Cursor.iterate(this) _ Friday, March 9, 12

Slide 178

Slide 178 text

Iteration “Helper” function def iterate[T: SerializableBSONObject](cursor: Cursor[T])(op: (IterState) 㱺 IterCmd) { log.trace("Iterating '%s' with op: '%s'", cursor, op) def next(f: (IterState) 㱺 IterCmd): Unit = op(cursor.next()) match { case Done 㱺 { log.trace("Closing Cursor.") cursor.close() } case Next(tOp) 㱺 { log.trace("Next!") next(tOp) } case NextBatch(tOp) 㱺 cursor.nextBatch(() 㱺 { log.debug("Next Batch Loaded.") next(tOp) }) } next(op) } Friday, March 9, 12

Slide 179

Slide 179 text

A Simple Iteration of a Cursor def iterateComplexCursor(conn: MongoConnection) = { var x = 0 conn(integrationTestDBName).find("books")(Document.empty, Document.empty)((cursor: Cursor[Document]) 㱺 { def next(op: Cursor.IterState): Cursor.IterCmd = op match { case Cursor.Entry(doc) 㱺 { x += 1 if (x < 100) Cursor.Next(next) else Cursor.Done } case Cursor.Empty 㱺 { if (x < 100) Cursor.NextBatch(next) else Cursor.Done } case Cursor.EOF 㱺 { Cursor.Done } } Cursor.iterate(cursor)(next) }) x must eventually(5, 5.seconds)(be_==(100)) } Friday, March 9, 12

Slide 180

Slide 180 text

Briefly: NIO.2 / AIO • JDK 7 introduces a higher level async API to NIO without going as high level as Netty does • NIO.2 / AIO brings in “AsynchronousSocketChannels” • Removes need to select / poll by hand • Configurable timeouts • Two options for how to get responses; both sanely map to Scala • java.util.concurrent.Future • CompletionHandler • Easily logically mapped to Either[E, T] with implicits • Probably not ‘prime time’ usable for library authors *yet* ... due to dependency on JDK7 Friday, March 9, 12

Slide 181

Slide 181 text

@mongodb conferences, appearances, and meetups http://www.10gen.com/events http://bit.ly/mongofb Facebook | Twitter | LinkedIn http://linkd.in/joinmongo download at mongodb.org github.com/mongodb/casbah github.com/bwmcadams/hammersmith These slides will be online later at: http://speakerdeck.com/u/bwmcadams/ brendan@10gen.com (twitter: @rit) Friday, March 9, 12