ØMQ and Perl - Speaker Deck

Slide 1

Slide 1 text

ØMQ and Perl Anthony Johnson @agjhnsn

Slide 2

Slide 2 text

What is ØMQ? And what isn’t it?

Slide 3

Slide 3 text

A better socket • Abstracts connection boilerplate • Gracefully handles connections • Messages don’t change during send

Slide 4

Slide 4 text

A better message queue • Queues packets being sent or received • One-way and two-way message patterns for concurrency • Decentralized

Slide 5

Slide 5 text

It’s not: • A BSD/POSIX socket • Low latency • A job queue

Slide 6

Slide 6 text

When to use ØMQ? • Replacing an RPC or RESTful API • Asynchronous tasks model with workers • Cross language support is needed • Just need to write a socket

Slide 7

Slide 7 text

ØMQ isn’t a replacement for • A client-facing HTTP API • Low latency sockets • Job queues

Slide 8

Slide 8 text

ØMQ Socket Foundations

Slide 9

Slide 9 text

Paired socket types • REQ/REP - send packet, expect response • PUSH/PULL - send/receive packet, no response • PUB/SUB - similar to pull, packet duplicated to all clients • ROUTER/DEALER - accumulate and distribute packets • REQ/ROUTER and DEALER/REP - load balancing

Slide 10

Slide 10 text

Rouer/dealer model

Slide 11

Slide 11 text

Multipart messages • Message can be grouped for throughput • Queues in memory • Won’t send automatically • Delivers all or none

Slide 12

Slide 12 text

Messages are strings • ØMQ knows nothing about your data • Data is a string and string length • Optionally use a serializer

Slide 13

Slide 13 text

Queue HWM (High Water Mark) • Each socket has a HWM • Sockets have send and/or receive buffers • When the HWM is reached, sockets may block or drop • ØMQ v2 HWM is inﬁnite • ØMQ v3 HWM is 1000 messages

Slide 14

Slide 14 text

ØMQ and Perl

Slide 15

Slide 15 text

Installing • ZMQ • ZMQ::LibZMQ2 or ZMQ::LibZMQ3 • ZeroMQ is deprecated • Development library

Slide 16

Slide 16 text

Forcing ZMQ::LibZMQ2

Slide 17

Slide 17 text

What does it look like?

Slide 18

Slide 18 text

The server

Slide 19

Slide 19 text

The client

Slide 20

Slide 20 text

Common Patterns

Slide 21

Slide 21 text

Serialization Using JSON for serialization

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Socket reliability • Protect against socket failure • Poll receiving socket instead of blocking • Resend request after timeout, abandon if still no replies

Slide 24

Slide 24 text

Polling sockets • Not blocking on receive • Can poll on multiple sockets • On socket receive, execute callback

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Polling multiple sockets

Slide 27

Slide 27 text

“Classy programmers are like classy hit men Always clean-up when you ﬁnish the job.”

Slide 28

Slide 28 text

Cleaning up • Set the linger option on sockets as they are closed • Sockets that aren’t ﬂushed cause the context to hang

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

Dynamic Discovery How do sockets know of each other?

Slide 31

Slide 31 text

Static discovery

Slide 32

Slide 32 text

Dynamic discovery through routing

Slide 33

Slide 33 text

Broker

Slide 34

Slide 34 text

Preemptively use a broker Don’t wait to scale out

Slide 35

Slide 35 text

Or, decentralize EVERYTHING.

Slide 36

Slide 36 text

More Advice • Mind your context scope when forking • Don’t be afraid to over engineer

Slide 37

Slide 37 text

Resources • ØMQ Guide http://zguide.zeromq.org/ • ØMQ zguide on Github https://github.com/imatix/zguide