The Wonderful World of Webhooks

1. The Wonderful World of Webhooks Lorna Mitchell, IBM August 2016
2. Webhooks in the Wild Webhooks power: • slack notifications • continuous integration services • other integrations @lornajane
3. How APIs Work @lornajane
4. How APIs Work @lornajane
5. How APIs Work @lornajane
6. How Webhooks Work @lornajane
7. How Webhooks Work @lornajane
8. How Webhooks Work @lornajane
9. What About Time? @lornajane
10. APIs Over Time @lornajane
11. Webhooks Over Time @lornajane
12. Webhooks Require PreArrangement @lornajane
13. Designing Webhooks @lornajane
14. Webhooks in the Wild: GitHub "ref": "refs/heads/master", "before": "1ae6a404351cead52df24893621d82ba6ec84a1c", "after": "e8474d83985330fa36f8862b37ca84ada4313392", "created": false, "deleted": false, "forced": false, "compare": "https://github.com/lornajane/demo/compare/1ae6a404351c...e847 "commits": [ ... ], "repository": { ... }, "pusher": { ... }, "sender": { ... } @lornajane
15. Designing Webhooks Consider the use cases: • try to include all information for common outcomes • consider impact of payload size vs potentially many followup API calls @lornajane
16. Webhooks in your Applications @lornajane
17. Receiving Webhooks It's just a POST request! Advice: • DO: accept, store and acknowledge quickly • DON'T: validate or process before acknowledging @lornajane
18. Storing Data Simplest queue can go in your database! • an ID • the payload that came in (extract fields to search on) • a status field (new/processed/failed) A cron job to work through what needs doing and update the status field. @lornajane
19. Storing Data @lornajane
20. Storing Data @lornajane
21. Storing Data If you outgrow the database method, use a queue. • e.g. RabbitMQ, Beanstalkd, Gearman • easy way to link many workers with work to do @lornajane
22. Publishing Webhooks Offering webhook integrations is ideal if: • you have clients polling your API a lot • it's common for another system to react to changes in your system • you want to offer notifications for specific events • any of the above apply either internally or externally @lornajane
23. Example App: Retro Guestbook In the olden days, we had guestbooks on our websites. • Allow user to leave their name and a comment • Show the comments left so far • Include webhook management for notifications of new comments @lornajane
24. Example App: Retro Guestbook @lornajane
25. Saving Data: Basic Process @lornajane
26. Saving Data: Handling Webhooks @lornajane
27. Saving Data: Handling Webhooks @lornajane
28. Demo Setup • An endpoint that we can send webhooks to for testing purposes: http://requestb.in/ • Queue is RabbitMQ (https://www.rabbitmq.com/) • RabbitMQ's management plugin has a web interface: http://192.168.121.8:15672 • (comments also go into the database that powers the site) • Worker scripts to send any/all webhooks @lornajane
29. Example App: Use a Queue $connection = new \PhpAmqpLib\Connection\AMQPConnection( $host, $port, $user, $pass); $channel = $connection->channel(); $channel->queue_declare('comments', false, true, false, false); $msg = new \PhpAmqpLib\Message\AMQPMessage( json_encode($comment), ["delivery_mode" => 2] ); $channel->basic_publish($msg, '', 'comments'); @lornajane
30. Example App: Workers Use The Queue amqp.connect('amqp://localhost', function(err, conn) { conn.createChannel(function(err, ch) { var q = 'comments'; ch.assertQueue(q, {durable: true}); ch.consume(q, function(msg) { // make the POST request for each webhook }); }); }); @lornajane
31. Example App: Workers Send Webhooks // hooks populated from couchdb hooks.forEach(function (url) { request({ url: url, method: "post", json: msg.content.toString() }, function (error, response, body) { // log hook outcome }); }); ch.ack(msg); @lornajane
32. Saving Data: Handling Webhooks @lornajane
33. Saving Data: Handling Webhooks @lornajane
34. Webhooks in Your Applications • Use them WHEN you want to notify other systems • Examples of HOW to use webhooks hopefully gave you some ideas • Webhooks are HTTP: we already understand this @lornajane
35. Webhooks ... are awesome :) @lornajane
36. Thanks! Contact: • lorna.mitchell@uk.ibm.com • @lornajane • http://lornajane.net @lornajane