Working with Webhooks

Transcript

1. Working with
   Webhooks
   Lorna Mitchell, IBM
   May 2017
   

   View Slide

2. Webhooks in the Wild
   Webhooks power:
   • slack notifications
   • continuous integration services
   • other integrations
   @lornajane
   

   View Slide

3. How APIs Work
   @lornajane
   

   View Slide

4. How APIs Work
   @lornajane
   

   View Slide

5. How APIs Work
   @lornajane
   

   View Slide

6. How Webhooks Work
   @lornajane
   

   View Slide

7. How Webhooks Work
   @lornajane
   

   View Slide

8. How Webhooks Work
   @lornajane
   

   View Slide

9. What About Time?
   @lornajane
   

   View Slide

10. APIs Over Time
    @lornajane
    

    View Slide

11. Webhooks Over Time
    @lornajane
    

    View Slide

12. Webhooks Require PreArrangement
    @lornajane
    

    View Slide

13. Designing Webhooks
    @lornajane
    

    View Slide

14. Webhooks in the Wild: GitHub
    "ref": "refs/heads/master",
    "before": "1ae6a404351cead52df24893621d82ba6ec84a1c",
    "after": "e8474d83985330fa36f8862b37ca84ada4313392",
    "created": false,
    "deleted": false,
    "forced": false,
    "compare": "https://github.com/lornajane/demo/compare/1ae6a404351c...e847
    "commits": [ ... ],
    "repository": { ... },
    "pusher": { ... },
    "sender": { ... }
    @lornajane
    

    View Slide

15. Designing Webhooks
    Consider the use cases:
    • try to include all information for common outcomes
    • consider impact of payload size vs potentially many followup API calls
    @lornajane
    

    View Slide

16. Receiving Webhooks
    @lornajane
    

    View Slide

17. Receiving Webhooks
    It's just a POST request! Advice:
    • DO: accept, store and acknowledge quickly
    • DON'T: validate or process before acknowledging
    @lornajane
    

    View Slide

18. Ngrok for Testing Webhooks
    https://ngrok.com/ - secure tunnel to your dev platform
    Use this tool to:
    • webhook into code running locally
    • inspect the request and response of the webhook
    • replay requests and see the responses
    @lornajane
    

    View Slide

19. Storing Data
    Simplest queue can go in your database!
    • an ID
    • the payload that came in (extract fields to search on)
    • a status field (new/processed/failed)
    A cron job to work through what needs doing and update the status field.
    @lornajane
    

    View Slide

20. Storing Data
    @lornajane
    

    View Slide

21. Storing Data
    @lornajane
    

    View Slide

22. Storing Data
    If you outgrow the database method, use a queue.
    • e.g. RabbitMQ, Beanstalkd, Gearman
    • easy way to link many workers with work to do
    @lornajane
    

    View Slide

23. Publishing Webhooks
    @lornajane
    

    View Slide

24. Publishing Webhooks
    Offering webhook integrations is ideal if:
    • you have clients polling your API a lot
    • it's common for another system to react to changes in your system
    • you want to offer notifications for specific events
    • any of the above apply either internally or externally
    @lornajane
    

    View Slide

25. Example App: Retro Guestbook
    In the olden days, we had guestbooks on our websites.
    • Allow user to leave their name and a comment
    • Show the comments left so far
    • Include webhook management for notifications of new comments
    @lornajane
    

    View Slide

26. Example App: Retro Guestbook
    @lornajane
    

    View Slide

27. Saving Data: Basic Process
    @lornajane
    

    View Slide

28. Saving Data: Handling Webhooks
    @lornajane
    

    View Slide

29. Saving Data: Handling Webhooks
    @lornajane
    

    View Slide

30. Data Hygiene
    Workers should be independent
    • pass in data up front if you can
    • ideally one component talks to each data store
    • keep metadata and data separate
    @lornajane
    

    View Slide

31. Saving Data: Handling Webhooks
    @lornajane
    

    View Slide

32. Saving Data: Handling Webhooks
    @lornajane
    

    View Slide

33. Saving Data: Handling Webhooks
    @lornajane
    

    View Slide

34. About Workers
    Workers are:
    • long running processes
    • may be able to process multiple message types
    • may have many of the same worker (horizontal scaling)
    • can be any tech (for bursty workloads, try serverless workers)
    Many microservices patterns also work well here
    @lornajane
    

    View Slide

35. Webhooks in Your Applications
    • Use them WHEN you want to notify other systems
    • Examples of HOW to use webhooks hopefully gave you some ideas
    • Webhooks are HTTP: we already understand this
    @lornajane
    

    View Slide

36. Webhooks
    ... are awesome :)
    @lornajane
    

    View Slide

37. Thanks!
    • Slides: http://lornajane.net/resources
    • Example app: https://github.com/ibm-cds-labs/guestbook
    • RabbitMQ: https://www.rabbitmq.com/
    • Bluemix: http://www.ibm.com/bluemix
    • Requestbin: http://requestb.in
    • Ngrok: https://ngrok.com/
    • Website: http://lornajane.net
    @lornajane
    

    View Slide