Ømq & Services @ Chartboost

This is an internal talk I gave at Chartboost on ZeroMq and how it can power distributed systems.

Transcript

1. ØMQ & SERVICES
2. ØMQ & SERVICES part i: zeromq
3. wtf is zmq?
4. Zeromq is what bsd sockets may have looked like, if they were designed today.” “
5. Zeromq is a communication library.
6. # C & C++ void *context = zmq_init(1); # ruby context = ZMQ::Context.new(1) # php $context = new ZMQContext(1); # etc. polyglot
7. atomic finite &
8. messaging patterns
9. request/reply client server blah? blah!
10. request/reply server client server client client client
11. request/reply server client server client client client
12. request/reply server client server client client client
13. push/pull pusher puller blah!
14. push/pull STEP 2 node node node node STEP 1 STEP 3 node node node
15. node push/pull STEP 2 node node node node STEP 1 STEP 3 node node
16. node push/pull STEP 2 node node node node STEP 1 STEP 3 node node
17. pub/sub subscriber subscriber subscriber subscriber publisher
18. pub/sub subscriber subscriber subscriber subscriber publisher
19. pub/sub subscriber subscriber subscriber subscriber publisher
20. irl client server subscriber worker worker subscriber REQ PUSH PUSH PUSH PUB PUB node node node node node
21. examples
22. a chat service chat server sean kenneth mark
23. a chat service chat server sean kenneth mark pub pub pub push
24. a chat service chat server sean kenneth mark hi! hi! hi!
25. a chat service chat server sean kenneth mark hi! hi! hi!
26. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need pub = context.socket(ZMQ::PUB) pull = context.socket(ZMQ::PULL) # bind the sockets pub.bind('tcp://*:1338') pull.bind('tcp://*:1337') # wait for input, and forward to all subscribers while body = pull.recv payload = JSON.parse(body) pub.send "#{payload['user']}> #{payload['message'].cyan}" end server.rb
27. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need pub = context.socket(ZMQ::PUB) pull = context.socket(ZMQ::PULL) # bind the sockets pub.bind('tcp://*:1338') pull.bind('tcp://*:1337') # wait for input, and forward to all subscribers while body = pull.recv payload = JSON.parse(body) pub.send "#{payload['user']}> #{payload['message'].cyan}" end server.rb
28. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need pub = context.socket(ZMQ::PUB) pull = context.socket(ZMQ::PULL) # bind the sockets pub.bind('tcp://*:1338') pull.bind('tcp://*:1337') # wait for input, and forward to all subscribers while body = pull.recv payload = JSON.parse(body) pub.send "#{payload['user']}> #{payload['message'].cyan}" end server.rb
29. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need pub = context.socket(ZMQ::PUB) pull = context.socket(ZMQ::PULL) # bind the sockets pub.bind('tcp://*:1338') pull.bind('tcp://*:1337') # wait for input, and forward to all subscribers while body = pull.recv payload = JSON.parse(body) pub.send "#{payload['user']}> #{payload['message'].cyan}" end server.rb
30. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need pub = context.socket(ZMQ::PUB) pull = context.socket(ZMQ::PULL) # bind the sockets pub.bind('tcp://*:1338') pull.bind('tcp://*:1337') # wait for input, and forward to all subscribers while body = pull.recv payload = JSON.parse(body) pub.send "#{payload['user']}> #{payload['message'].cyan}" end server.rb
31. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
32. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
33. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
34. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
35. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
36. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
37. a chat service # create the context context = ZMQ::Context.new(1) # create the two sockets we need sub = context.socket(ZMQ::SUB) sub.setsockopt(ZMQ::SUBSCRIBE, '') push = context.socket(ZMQ::PUSH) # bind the sockets sub.connect("tcp://#{server}:1338") push.connect("tcp://#{server}:1337") # wait for some input while line = gets.chomp push.send(line) unless line == '' # dump buffered messages puts buffered_msg while (buffered_msg = sub.recv(ZMQ::NOBLOCK)) end client.rb
38. demo $ git clone https://github.com/ \ ChartBoost/zmq-examples.git $ bundle install $ ruby chat/client.rb
39. demo $ git clone https://github.com/ \ ChartBoost/zmq-examples.git $ bundle install $ ruby chat/client.rb
40. thanks @KOB — KENNETH@CHARTBOOST.COM