Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ZeroMQ: Scriptable Sockets

ZeroMQ: Scriptable Sockets

ZeroMQ is a socket abstraction and concurrency framework that's changing the way we think and reason about distributed systems. Mailboxes, atomic message delivery and swappable transports allow for fast, flexible and resilient network topologies. Its I/O model also sits very well with all Ruby implementations. In this talk we'll discuss:

* What's wrong with socket I/O ?
* Supported messaging patterns
* Transport agnostic messaging
* Resiliency (operations and upgrades)
* Building out topologies just in time (interjection principle)
* Performance and throughput
* The Mongrel2 Ruby adapter
* How to use it from Ruby
* Small case study

28314d64ae18c43f6b5428dc00a48e6e?s=128

Lourens Naudé

August 18, 2012
Tweet

Transcript

  1. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ZeroMQ: scriptable sockets Lourens Naudé European Ruby Camp 2012 Berlin Sábado, 18 de Agosto de 12
  2. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 In 2012 it’s still too “expensive” to create and maintain integration points. Why this talk ? Sábado, 18 de Agosto de 12
  3. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Agenda 35k feet view Messages and patterns Transports and devices Resiliency Rewby Sábado, 18 de Agosto de 12
  4. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 What’s wrong with socket I/O ? Sábado, 18 de Agosto de 12
  5. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Skills Strong dev profile required Multiplexed I/O knowledge Understand protocols Buffer management Sábado, 18 de Agosto de 12
  6. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Not portable Standards compliance, meh Socket options Error codes Ruby abstracts most of these Sábado, 18 de Agosto de 12
  7. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Not flexible 2 patterns Bi-directional or multicast Lock-step connection Fixed network topology Sábado, 18 de Agosto de 12
  8. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Zero broker, admin, cost, waste, price tag and (almost) zero latency What is ZeroMQ ? Sábado, 18 de Agosto de 12
  9. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 IM for apps Not a server or message queue Communication / concurrency library Messaging semantics for sockets BSD sockets API Async by design Sábado, 18 de Agosto de 12
  10. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 How did we get here ? Sábado, 18 de Agosto de 12
  11. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 No really ... Martin Sustrik + Pieter Hintjens Everything AMQP isn’t Messages, not byte streams Financial services - optimization iMatix && community aka Crossroads I/O (fork) Sábado, 18 de Agosto de 12
  12. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Rich bindings ecosystem 30+ language bindings ... Sábado, 18 de Agosto de 12
  13. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 In the wild ... and financial industry + startups Sábado, 18 de Agosto de 12
  14. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Super (scriptable) sockets What is a ZeroMQ socket ? Sábado, 18 de Agosto de 12
  15. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 How ? An API model Embeddable servers Multiple transports Socket metaphor Familiar BSD socket API Multiplexing, framing etc. Sábado, 18 de Agosto de 12
  16. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 bind() or connect() to multiple endpoints, simultaneously Multiple endpoints Sábado, 18 de Agosto de 12
  17. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 bind() or connect() in any order Async connect Sábado, 18 de Agosto de 12
  18. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Socket types PUB SUB REQ REP PUSH PULL Sábado, 18 de Agosto de 12
  19. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Touching the water PUSH PULL PUSH PULL PULL PUSH PULL PUSH VENTILATOR SINK Sábado, 18 de Agosto de 12
  20. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Opaque binary data (blobs) Messages Sábado, 18 de Agosto de 12
  21. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Message format Framed wire protocol No contracts Serialize as you wish No auth / security Sábado, 18 de Agosto de 12
  22. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Message delivery Multipart messages supported Send 5GB if you like Atomic delivery NO PERSISTENCE Sábado, 18 de Agosto de 12
  23. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ENOTAMQ Sábado, 18 de Agosto de 12
  24. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Request / Reply Supported patterns Sábado, 18 de Agosto de 12
  25. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Request / reply REQ REP REQ REP REP REP REQ REQ Sábado, 18 de Agosto de 12
  26. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Request / Reply usage REQ socket load balances Lock step Bi-directional Custom routing Downstream topology flow control Sábado, 18 de Agosto de 12
  27. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Publish / subscribe Supported patterns Sábado, 18 de Agosto de 12
  28. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Publish / subscribe PUB SUB SUB SUB Sábado, 18 de Agosto de 12
  29. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Logging SUB PUB PUB PUB Sábado, 18 de Agosto de 12
  30. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Publish / subscribe usage Market data, game state etc. PUB socket distributes SUB socket filters Subscription forwarding (3.x) Sábado, 18 de Agosto de 12
  31. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Pipeline Supported patterns Sábado, 18 de Agosto de 12
  32. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Pipeline PUSH PULL PULL PULL Sábado, 18 de Agosto de 12
  33. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Pipeline usage Like REQ/REP, downstream reply Extendable - UNIX pipes Round-robin distribution Tasks with variable processing Sábado, 18 de Agosto de 12
  34. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Parallel pipeline PUSH PULL PUSH PULL PULL PUSH PULL PUSH VENTILATOR SINK Sábado, 18 de Agosto de 12
  35. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Transport agnostic messaging Transports Sábado, 18 de Agosto de 12
  36. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Supported transports inproc:// - threads in a process ipc:// - processes on a box tcp:// - nodes in a network Uniform API - it just works Sábado, 18 de Agosto de 12
  37. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Lightweight switches Devices Sábado, 18 de Agosto de 12
  38. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 eg. pub/sub persistence SUB PUB PUB SUB SUB Sábado, 18 de Agosto de 12
  39. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Device main loop f = ctx.connect(:SUB, "tcp:// 0.0.0.0:5110") b = ctx.bind(:PUB, “tcp:// 0.0.0.0:5111”) loop do m = f.recv process(m) b.send(m) end Sábado, 18 de Agosto de 12
  40. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Uses Proxy Routing (LRU) Filtering / de-dup Translation Persistence Sábado, 18 de Agosto de 12
  41. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Interjection Principle Scalable topologies Sábado, 18 de Agosto de 12
  42. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 eg. saving bandwidth SUB PUB PUB SUB SUB SUB SUB Sábado, 18 de Agosto de 12
  43. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Asynchronous external dependencies. Resiliency Sábado, 18 de Agosto de 12
  44. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ZeroMQ resiliency Atomic message delivery Delayed retry on failure Multi bind() and connect() Unordered bind() or connect() SOA - easy upgrades Sábado, 18 de Agosto de 12
  45. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Water marks Throttling / flow control High Water Mark (HWM) Socket option DROPS - REP, PUB, SUB BLOCKS - REQ, PUSH Sábado, 18 de Agosto de 12
  46. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 99% of deployments don’t need enterprise reliability Of reliability Sábado, 18 de Agosto de 12
  47. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Then ... Costly transactions Small topology Large ops team Little churn Sábado, 18 de Agosto de 12
  48. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ... now Negligible transaction cost Large topology Small ops team Rapid churn rate Sábado, 18 de Agosto de 12
  49. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Reliability as patterns Lazy Pirate (REQ / REP) Majordomo (queue) Titanic (persistence) Binary Star (High Availability) See http://zguide.zeromq.org Sábado, 18 de Agosto de 12
  50. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 http://zguide.zeromq.org Best documentation. Ever. Sábado, 18 de Agosto de 12
  51. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Low latency, high throughput. Performance Sábado, 18 de Agosto de 12
  52. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Lab: 6 000 000 msgs/s Ruby: 200 000 msgs/s (16 byte payload) Numbers Sábado, 18 de Agosto de 12
  53. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Measured per message. Latency Sábado, 18 de Agosto de 12
  54. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Measured at specific points Throughput Sábado, 18 de Agosto de 12
  55. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ZeroMQ optimizations Dedicated I/O thread I/O model suits scripting languages Syscalls and allocations etc. Sábado, 18 de Agosto de 12
  56. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Opportunistic batching Latency > TCP Throughput > TCP Avoid the network stack Scales to any pipe size Sábado, 18 de Agosto de 12
  57. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 gem install rbczmq Rewby. Sábado, 18 de Agosto de 12
  58. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ctx = ZMQ::Context.new(1) rep = ctx.bind(:REP, “inproc://p” req = ctx.connect(:REQ, “inproc://p” req.send(“message”) rep.receive(“message”) ctx.destroy Context - sockets state Sábado, 18 de Agosto de 12
  59. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ctx = ZMQ::Context.new(1) sub = ctx.socket(:SUB) sub.subscribe(“APPLE”) sub.connect(“tcp://0.0.0.0:5000”) sub.connect(“ipc://other.pub”) loop do process(sub.recv) end Initialize a socket Sábado, 18 de Agosto de 12
  60. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ZeroMQ concurrency PUSH PULL PULL PULL MAIN THREAD THREAD 1 THREAD 2 THREAD 3 I/O THREAD Sábado, 18 de Agosto de 12
  61. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 ctx = ZMQ::Context.new s = ctx.bind(:PUSH, "inproc://work") 3.times{ ws << spawn_worker(ctx) } loop do msg = work.shift break if msg == “exit” s.send(msg) end Master (work distributor) Sábado, 18 de Agosto de 12
  62. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Thread.new do s = ctx.connect(:PULL, “inproc:// work”) loop do msg = s.recv break if msg == "quit" process(msg) end end Workers (consumer threads) Sábado, 18 de Agosto de 12
  63. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Mongrel2 Case study Sábado, 18 de Agosto de 12
  64. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 How it works Multi-protocol ZeroMQ for backends Async backend requests Language agnostic (bindings) Sábado, 18 de Agosto de 12
  65. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Mongrel2 topology PULL PUB PULL PUB PUSH SUB BACKEND A BACKEND B MONGREL2 CLIENT Sábado, 18 de Agosto de 12
  66. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Best practices Patterns - learn them Stable bind() Unstable connect() Interjection principle Set water marks Read http://zguide.zeromq.org Sábado, 18 de Agosto de 12
  67. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 The Future UDP, DNS transports etc. Scalability Protocol (Linux Kernel) Mobile / embedded Monitoring support (beta 3.2) Sábado, 18 de Agosto de 12
  68. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 Questions ? (or sync up after ?) O fim Sábado, 18 de Agosto de 12
  69. Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood

    City, CA 94063 | (888) 274-0929 fork @methodmissing follow @methodmissing Thanks! Sábado, 18 de Agosto de 12