Message Queues for Everyone

An intro to AMQP for Node.js developers

Transcript

1. Message queues

   for everyone
   An intro workshop to AMQP
   For Node.js Developers
   

   View Slide

2. AMQP
   

   View Slide

3. Advanced Message

   Queuing Protocol
   

   View Slide

4. 0.9.1
   

   View Slide

5. View Slide

6. Why?
   

   View Slide

7. Job Queue

   Event Stream

   Buffered RPC
   

   View Slide

8. Job Queue
   

   View Slide

9. Producer Consumer
   

   View Slide

10. Producer
    Consumer
    Producer
    

    View Slide

11. Producer Consumer
    Producer Consumer
    

    View Slide

12. Producer Consumer
    Producer Consumer
    

    View Slide

13. Exchange In

    

    Queue Out
    

    View Slide

14. Slow Operations

    Expensive Operations

    Brittle Operations
    

    View Slide

15. Producer
    c = create_channel
    q = c.queue("task")
    x = c.default_exchange
    x.publish(
    "job1",
    "payload"
    )
    Consumer
    c = create_channel
    q = c.queue("task")
    q.subscribe(worker)
    worker = fn(*message)
    {
    // do stuff
    }
    

    View Slide

16. Acknowledging Messages
    q = c.queue("task")
    q.subscribe(ack = TRUE, worker)
    worker = fn(body, headers, info, m) {
    // do work
    m.acknowledge()
    }
    

    View Slide

17. Publishing Options
    Mandatory
    Will error if not queued
    Confirm
    Server acknowledges receipt
    

    View Slide

18. Queue Declare Options
    Passive
    Don't create, get reference to
    existing queue
    Durable
    Queue still exists after a broker
    restart
    

    View Slide

19. Event Stream
    

    View Slide

20. Producer
    Consumer
    Producer
    Consumer
    Consumer
    Consumer
    Producer
    

    View Slide

21. Messages
    Routing Key
    

    View Slide

22. Exchanges
    Fanout: all queues

    Direct: matching key

    Topic: key patterns
    

    View Slide

23. Exchange
    |

    Binding

    |
    Queue
    

    View Slide

24. Topic Exchange
    routing.key
    Exact match
    routing.*.key
    routing.#.key
    Wildcard
    matching
    routing.key
    routing.a.key
    routing.b.key
    routing.a.b.key
    

    View Slide

25. (soft) Real-time updates

    User notifications

    Work distribution
    

    View Slide

26. Binding a Queue
    x = c.exchange("activity”)
    q = c.queue("alert")
    q.bind(x, "balance.*")
    q.bind(x, "transfer.out")
    x.publish("balance.low")
    x.publish("transfer.in")
    x.publish("transfer.out")
    

    View Slide

27. Queue Declare Options
    Exclusive
    Only one consumer on this queue
    at any time
    Auto Delete
    Queue is deleted after consumer
    disconnects
    

    View Slide

28. Simulator Demo
    

    View Slide

29. Buffered RPC
    

    View Slide

30. Producer
    Consumer
    

    View Slide

31. Smooth out Spikes

    Transparently scale

    Deferred Response
    

    View Slide

32. Message Header
    Reply To

    Correlation ID
    

    View Slide

33. Reply To
    reply = c.queue("abcdef")
    x.publish("request", headers: {
    replyTo: "abcdef"
    }})
    reply.subscribe(handleResponse)
    

    View Slide

34. Correlation ID
    x.publish("request1", headers: {
    replyTo: "abcdef",
    correlationId: "request1"
    })
    x.publish("request2", headers: {
    replyTo: "abcdef",
    correlationId: "request2"
    })
    

    View Slide

35. Admin Interface Demo
    

    View Slide

36. Why not just
    use … ?
    

    View Slide

37. Redis pub/sub

    (Resque etc)
    

    View Slide

38. Beanstalkd
    

    View Slide

39. JMS

    Stomp
    

    View Slide

40. MQTT
    

    View Slide

41. ØMQ
    

    View Slide

42. Node.js Quickstart
    

    View Slide

43. npm install amqp
    

    View Slide

44. var amqp = require('amqp')

    

    var conn = amqp.createConnection(

    { url: 'amqp://un:pw@host:5672/vhost' },

    { reconnect: false }

    )

    

    conn.on('ready', function() {

    // Do stuff

    })
    

    View Slide

45. var opts = { noDeclare: true }

    conn.queue(name, opts, function(q) {

    q.subscribe(

    { ack: true, prefetch: 1 },

    onMessage

    )

    })

    function onMessage(body, meta, info, msg) {

    // do some stuff

    msg.acknowledge()

    }
    

    View Slide

46. var opts = {

    type: 'topic',

    noDeclare: true

    }

    conn.exchange(name, opts, function(x) {

    x.publish('key', 'body', {

    contentType: 'text/json'

    })

    })
    

    View Slide

47. Questions?
    

    View Slide

48. Lets go!
    http://amqp-nodejs.herokuapp.com
    https://addons.heroku.com/marketplace/rabbitmq-bigwig
    

    View Slide

49. @glenathan
    

    View Slide