ZeroMQ - Toolkit for high-performance distributed applications

Transcript

1. ZEROMQ TOOLKIT FOR HIGH-PERFORMANCE DISTRIBUTED APPLICATIONS zdeslav vojković @zdeslav_v zdeslav@plural.hr / www.plural.hr
2. DISTRIBUTED APPLICATIONS
3. DISTRIBUTED APPLICATIONS
4. WHAT IS ZEROMQ? not an MQ! networking and concurrency library asynchronous messaging routing/topology aware brokerless high performance multiple transports cross-platform numerous bindings (client libs) LGPL licensed
5. DESIGN AND IMPLEMENTATION
6. NO BROKER
7. CONVENTIONAL SOCKETS (TCP/UDP)
8. ZEROMQ SOCKETS
9. MESSAGES Asynchronous Atomic Multipart
10. PERFORMANCE Low latency, high throughput Zero-copy, reference counted Batching Configurable through socket options RabbitMQ: 4500 msg/s -> ZeroMQ: 250,000 msg/s apples to oranges (but similar use case with transient messages)
11. ZEROMQ API Objects: context, socket, message Flow: create context > bind/connect > send/receive # s e r v e r p r o c e s s c o n t e x t = z m q . C o n t e x t ( 1 ) s o c k e t = c o n t e x t . s o c k e t ( z m q . P U B ) s o c k e t . b i n d ( " t c p : / / * : 5 5 5 6 " ) s o c k e t . s e n d ( " s o m e e v e n t " ) # c l i e n t p r o c e s s c o n t e x t = z m q . C o n t e x t ( ) s o c k e t = c o n t e x t . s o c k e t ( z m q . S U B ) s o c k e t . c o n n e c t ( " t c p : / / s e r v e r : 5 5 5 6 " ) e v e n t _ d a t a = s o c k e t . r e c v ( )
12. NOT IN THE BOX Serialization Compression Security ...
13. TRANSPORTS
14. TRANSPORTS # s e r v e r p r o c e s s c o n t e x t = z m q . C o n t e x t ( 1 ) s o c k e t = c o n t e x t . s o c k e t ( z m q . P U B ) # s o c k e t . b i n d ( " t c p : / / * : 5 5 5 6 " ) s o c k e t . b i n d ( " i p c : / / / t m p / f e e d s / 0 " ) s o c k e t . s e n d ( " s o m e e v e n t " ) # c l i e n t p r o c e s s c o n t e x t = z m q . C o n t e x t ( ) s o c k e t = c o n t e x t . s o c k e t ( z m q . S U B ) # s o c k e t . c o n n e c t ( " t c p : / / l o c a l h o s t : 5 5 5 6 " ) s o c k e t . c o n n e c t ( " i p c : / / / t m p / f e e d s / 0 " ) e v e n t _ d a t a = s o c k e t . r e c v ( )
15. COMMUNICATION PATTERNS
16. PATTERN: PUBLISH-SUBSCRIBE
17. PATTERN: PUBLISH-SUBSCRIBE Sockets: PUB/SUB SUB can subscribe to specific topic or all messages
18. PATTERN: PUBLISH-SUBSCRIBE Sockets: PUB/SUB, XPUB/XSUB
19. PATTERN: CLIENT-SERVER
20. PATTERN: CLIENT-SERVER Sockets: REQ/REP c o n t e x t = z m q . C o n t e x t ( 1 ) s o c k e t = c o n t e x t . s o c k e t ( z m q . R E Q ) s o c k e t . c o n n e c t ( " t c p : / / l o c a l h o s t : 5 5 5 6 " ) s o c k e t . s e n d ( " s o m e e v e n t " ) r e p l y = s o c k e t . r e c v ( ) # b l o c k s
21. PATTERN: CLIENT-SERVER Sockets: REQ/REP, ROUTER/DEALER
22. PATTERN: PIPELINE
23. PATTERN: PIPELINE Sockets: PUSH/PULL
24. PATTERN: THREAD SIGNALING
25. PATTERN: THREAD SIGNALING Socket: PAIR
26. TOPOLOGIES AND DESIGNS
27. DISTRIBUTED LOGGING: PUB-SUB
28. MONGREL2: WEB SERVER
29. CHAT SERVER: OPPOSITE OF MONGREL
30. CONSUMER CONTROLLED PIPELINE
31. THANK YOU! http://zguide.zeromq.org