Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ZeroMQ: scriptable sockets
Lourens Naudé
European Ruby Camp 2012
Berlin
Sábado, 18 de Agosto de 12
Slide 2
Slide 2 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
In 2012 it’s still too
“expensive” to create and
maintain integration points.
Why this talk ?
Sábado, 18 de Agosto de 12
Slide 3
Slide 3 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Agenda
35k feet view
Messages and patterns
Transports and devices
Resiliency
Rewby
Sábado, 18 de Agosto de 12
Slide 4
Slide 4 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
What’s wrong with socket I/O ?
Sábado, 18 de Agosto de 12
Slide 5
Slide 5 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Skills
Strong dev profile required
Multiplexed I/O knowledge
Understand protocols
Buffer management
Sábado, 18 de Agosto de 12
Slide 6
Slide 6 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Not portable
Standards compliance, meh
Socket options
Error codes
Ruby abstracts most of these
Sábado, 18 de Agosto de 12
Slide 7
Slide 7 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Not flexible
2 patterns
Bi-directional or multicast
Lock-step connection
Fixed network topology
Sábado, 18 de Agosto de 12
Slide 8
Slide 8 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Zero broker, admin, cost,
waste, price tag and
(almost) zero latency
What is ZeroMQ ?
Sábado, 18 de Agosto de 12
Slide 9
Slide 9 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
IM for apps
Not a server or message queue
Communication / concurrency library
Messaging semantics for sockets
BSD sockets API
Async by design
Sábado, 18 de Agosto de 12
Slide 10
Slide 10 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
How did we get here ?
Sábado, 18 de Agosto de 12
Slide 11
Slide 11 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
No really ...
Martin Sustrik + Pieter Hintjens
Everything AMQP isn’t
Messages, not byte streams
Financial services - optimization
iMatix && community
aka Crossroads I/O (fork)
Sábado, 18 de Agosto de 12
Slide 12
Slide 12 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Rich bindings ecosystem
30+ language bindings ...
Sábado, 18 de Agosto de 12
Slide 13
Slide 13 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
In the wild
... and financial industry + startups
Sábado, 18 de Agosto de 12
Slide 14
Slide 14 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Super (scriptable) sockets
What is a ZeroMQ socket ?
Sábado, 18 de Agosto de 12
Slide 15
Slide 15 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
How ?
An API model
Embeddable servers
Multiple transports
Socket metaphor
Familiar BSD socket API
Multiplexing, framing etc.
Sábado, 18 de Agosto de 12
Slide 16
Slide 16 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
bind() or connect() to
multiple endpoints,
simultaneously
Multiple endpoints
Sábado, 18 de Agosto de 12
Slide 17
Slide 17 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
bind() or connect() in any
order
Async connect
Sábado, 18 de Agosto de 12
Slide 18
Slide 18 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Socket types
PUB SUB
REQ REP
PUSH PULL
Sábado, 18 de Agosto de 12
Slide 19
Slide 19 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Touching the water
PUSH
PULL
PUSH
PULL
PULL
PUSH
PULL
PUSH
VENTILATOR
SINK
Sábado, 18 de Agosto de 12
Slide 20
Slide 20 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Opaque binary data (blobs)
Messages
Sábado, 18 de Agosto de 12
Slide 21
Slide 21 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Message format
Framed wire protocol
No contracts
Serialize as you wish
No auth / security
Sábado, 18 de Agosto de 12
Slide 22
Slide 22 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Message delivery
Multipart messages supported
Send 5GB if you like
Atomic delivery
NO PERSISTENCE
Sábado, 18 de Agosto de 12
Slide 23
Slide 23 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ENOTAMQ
Sábado, 18 de Agosto de 12
Slide 24
Slide 24 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Request / Reply
Supported patterns
Sábado, 18 de Agosto de 12
Slide 25
Slide 25 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Request / reply
REQ
REP
REQ
REP REP
REP
REQ REQ
Sábado, 18 de Agosto de 12
Slide 26
Slide 26 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Request / Reply usage
REQ socket load balances
Lock step
Bi-directional
Custom routing
Downstream topology flow control
Sábado, 18 de Agosto de 12
Slide 27
Slide 27 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Publish / subscribe
Supported patterns
Sábado, 18 de Agosto de 12
Slide 28
Slide 28 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Publish / subscribe
PUB
SUB SUB SUB
Sábado, 18 de Agosto de 12
Slide 29
Slide 29 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Logging
SUB
PUB PUB PUB
Sábado, 18 de Agosto de 12
Slide 30
Slide 30 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Publish / subscribe usage
Market data, game state etc.
PUB socket distributes
SUB socket filters
Subscription forwarding (3.x)
Sábado, 18 de Agosto de 12
Slide 31
Slide 31 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Pipeline
Supported patterns
Sábado, 18 de Agosto de 12
Slide 32
Slide 32 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Pipeline
PUSH
PULL PULL PULL
Sábado, 18 de Agosto de 12
Slide 33
Slide 33 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Pipeline usage
Like REQ/REP, downstream reply
Extendable - UNIX pipes
Round-robin distribution
Tasks with variable processing
Sábado, 18 de Agosto de 12
Slide 34
Slide 34 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Parallel pipeline
PUSH
PULL
PUSH
PULL
PULL
PUSH
PULL
PUSH
VENTILATOR
SINK
Sábado, 18 de Agosto de 12
Slide 35
Slide 35 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Transport agnostic
messaging
Transports
Sábado, 18 de Agosto de 12
Slide 36
Slide 36 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Supported transports
inproc:// - threads in a process
ipc:// - processes on a box
tcp:// - nodes in a network
Uniform API - it just works
Sábado, 18 de Agosto de 12
Slide 37
Slide 37 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Lightweight switches
Devices
Sábado, 18 de Agosto de 12
Slide 38
Slide 38 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
eg. pub/sub persistence
SUB
PUB
PUB
SUB SUB
Sábado, 18 de Agosto de 12
Slide 39
Slide 39 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Device main loop
f = ctx.connect(:SUB, "tcp://
0.0.0.0:5110")
b = ctx.bind(:PUB, “tcp://
0.0.0.0:5111”)
loop do
m = f.recv
process(m)
b.send(m)
end
Sábado, 18 de Agosto de 12
Slide 40
Slide 40 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Uses
Proxy
Routing (LRU)
Filtering / de-dup
Translation
Persistence
Sábado, 18 de Agosto de 12
Slide 41
Slide 41 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Interjection Principle
Scalable topologies
Sábado, 18 de Agosto de 12
Slide 42
Slide 42 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
eg. saving bandwidth
SUB
PUB
PUB
SUB SUB
SUB SUB
Sábado, 18 de Agosto de 12
Slide 43
Slide 43 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Asynchronous external
dependencies.
Resiliency
Sábado, 18 de Agosto de 12
Slide 44
Slide 44 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ZeroMQ resiliency
Atomic message delivery
Delayed retry on failure
Multi bind() and connect()
Unordered bind() or connect()
SOA - easy upgrades
Sábado, 18 de Agosto de 12
Slide 45
Slide 45 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Water marks
Throttling / flow control
High Water Mark (HWM)
Socket option
DROPS - REP, PUB, SUB
BLOCKS - REQ, PUSH
Sábado, 18 de Agosto de 12
Slide 46
Slide 46 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
99% of deployments don’t need
enterprise reliability
Of reliability
Sábado, 18 de Agosto de 12
Slide 47
Slide 47 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Then ...
Costly transactions
Small topology
Large ops team
Little churn
Sábado, 18 de Agosto de 12
Slide 48
Slide 48 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
... now
Negligible transaction cost
Large topology
Small ops team
Rapid churn rate
Sábado, 18 de Agosto de 12
Slide 49
Slide 49 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Reliability as patterns
Lazy Pirate (REQ / REP)
Majordomo (queue)
Titanic (persistence)
Binary Star (High Availability)
See http://zguide.zeromq.org
Sábado, 18 de Agosto de 12
Slide 50
Slide 50 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
http://zguide.zeromq.org
Best documentation. Ever.
Sábado, 18 de Agosto de 12
Slide 51
Slide 51 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Low latency, high throughput.
Performance
Sábado, 18 de Agosto de 12
Slide 52
Slide 52 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Lab: 6 000 000 msgs/s
Ruby: 200 000 msgs/s
(16 byte payload)
Numbers
Sábado, 18 de Agosto de 12
Slide 53
Slide 53 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Measured per message.
Latency
Sábado, 18 de Agosto de 12
Slide 54
Slide 54 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Measured at specific points
Throughput
Sábado, 18 de Agosto de 12
Slide 55
Slide 55 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ZeroMQ optimizations
Dedicated I/O thread
I/O model suits scripting languages
Syscalls and allocations
etc.
Sábado, 18 de Agosto de 12
Slide 56
Slide 56 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Opportunistic batching
Latency > TCP
Throughput > TCP
Avoid the network stack
Scales to any pipe size
Sábado, 18 de Agosto de 12
Slide 57
Slide 57 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
gem install rbczmq
Rewby.
Sábado, 18 de Agosto de 12
Slide 58
Slide 58 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ctx = ZMQ::Context.new(1)
rep = ctx.bind(:REP, “inproc://p”
req = ctx.connect(:REQ, “inproc://p”
req.send(“message”)
rep.receive(“message”)
ctx.destroy
Context - sockets state
Sábado, 18 de Agosto de 12
Slide 59
Slide 59 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ctx = ZMQ::Context.new(1)
sub = ctx.socket(:SUB)
sub.subscribe(“APPLE”)
sub.connect(“tcp://0.0.0.0:5000”)
sub.connect(“ipc://other.pub”)
loop do
process(sub.recv)
end
Initialize a socket
Sábado, 18 de Agosto de 12
Slide 60
Slide 60 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ZeroMQ concurrency
PUSH
PULL PULL PULL
MAIN THREAD
THREAD 1 THREAD 2 THREAD 3
I/O
THREAD
Sábado, 18 de Agosto de 12
Slide 61
Slide 61 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
ctx = ZMQ::Context.new
s = ctx.bind(:PUSH, "inproc://work")
3.times{ ws << spawn_worker(ctx) }
loop do
msg = work.shift
break if msg == “exit”
s.send(msg)
end
Master (work distributor)
Sábado, 18 de Agosto de 12
Slide 62
Slide 62 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Thread.new do
s = ctx.connect(:PULL, “inproc://
work”)
loop do
msg = s.recv
break if msg == "quit"
process(msg)
end
end
Workers (consumer threads)
Sábado, 18 de Agosto de 12
Slide 63
Slide 63 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Mongrel2
Case study
Sábado, 18 de Agosto de 12
Slide 64
Slide 64 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
How it works
Multi-protocol
ZeroMQ for backends
Async backend requests
Language agnostic (bindings)
Sábado, 18 de Agosto de 12
Slide 65
Slide 65 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Mongrel2 topology
PULL
PUB
PULL
PUB
PUSH
SUB
BACKEND A BACKEND B
MONGREL2
CLIENT
Sábado, 18 de Agosto de 12
Slide 66
Slide 66 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Best practices
Patterns - learn them
Stable bind()
Unstable connect()
Interjection principle
Set water marks
Read http://zguide.zeromq.org
Sábado, 18 de Agosto de 12
Slide 67
Slide 67 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
The Future
UDP, DNS transports etc.
Scalability Protocol (Linux Kernel)
Mobile / embedded
Monitoring support (beta 3.2)
Sábado, 18 de Agosto de 12
Slide 68
Slide 68 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
Questions ?
(or sync up after ?)
O fim
Sábado, 18 de Agosto de 12
Slide 69
Slide 69 text
Wildfire Interactive, Inc. | 1600 Seaport Boulevard, Suite 500, Redwood City, CA 94063 | (888) 274-0929
fork @methodmissing
follow @methodmissing
Thanks!
Sábado, 18 de Agosto de 12