ZeroMQ: Scale Up !

Talks presented at OSDC.fr 2012 (OpenWorld Forum 2012).
It's kinda a ZMQ 101, and how to use ZMQ for concurrent application design

Transcript

1. ZeroMQ: Scale Up ! whoami
2. Why ZeroMQ ? we live in a connected world ! we have to develop faster
3. Simple server (Python) import zmq context = zmq.Context(1) socket = context.socket(zmq.REP) socket.bind("tcp://*:5555") while True: # wait for request from client message = socket.recv() print ">: {}".format(message) answer = message[::-1] socket.send(answer) print "<: {}".format(answer) Client (Ruby) require 'zmq' context = ZMQ::Context.new(1) socket = context.socket(ZMQ::REQ) socket.connect("tcp://localhost:5555") message = "Have you met Ted ?" puts "<: #{message}" socket.send(message) answer = socket.recv() puts ">: #{answer}"
4. Basics BSD socket API message oriented various connection patterns small (20k SLOC of C++) fast LGPLv2+
5. various transports: tcp, ipc, inproc, pgm multithreaded friendly zero-copy for messages auto-reconnect Native Patterns
6. Request-Reply Publish-Subscribe
7. Pipeline various high-level patterns Lazy/Simple/Paranoid pirate majordomo all described in the zguide !
8. Various routing patterns round-robin (REQ, PUSH, DEALER) multicast (PUB) fair queueing (REP, SUB, PULL, DEALER) explicit routing (ROUTER) point to point (PAIR) Concurrency ?
9. Today ? data-centric model one datum, many executors you must control data accesses Consequences complexity race conditions heisenbugs
10. How ? adopt message-driven design ! design independant tasks (threads, processes) use inproc:// (threads), ipc:// (processes), tcp:// (network) In practice ... use actor model formalized by Carl Hewitt in 1973 Popularized by Erlang
11. Benefits lock-free no active wait ! rapid prototyping e = mc²
12. Conclusion