Implementing Webhooks, not as trivial as it may seem

Implementing Webhooks, not as trivial as it may seem

Webhooks, we know what they are, right? You’ve probably already used some to be notified of Github commits, to react to text messages received via Twilio, or created a fulfillment for a Dialogflow chatbot to answer users. From the Webhook consumer side, it doesn’t seem complicated to code… On the server-side, is it really just a web API to implement? Hmm… maybe not! Actually, we’ll see that it might be not as simple as it first sounds ! After an introduction on the concept of Webhooks, we’ll create our own callbacks to be notified of events. Then, we’ll go on the other side of the mirror, by creating our own Webhook backend. We’ll study how to deal with client subscription queues, manage all kinds of errors, debug the webhook, handle retries to avoid flooding subscribers, or how to secure those hooks. There’s lots to cover!

137d3908243acfc30e126615d59d4e6d?s=128

Guillaume Laforge

November 15, 2019
Tweet

Transcript

  1. 1.

    @glaforge WEB HOOKS WEB HOOKS Not as trivial as it

    may seem Not as trivial as it may seem @glaforge
  2. 3.

    @glaforge Ask a service to notify you via an HTTP

    callback to a URL you specify when an event occur Ask a service to notify you via an HTTP callback to a URL of your choice when an event occurs @glaforge
  3. 6.

    @glaforge Emails • SendGrid • MailChimp Chat messages • Dialogflow

    • Intercom Payments • Stripe • BrainTree Build results • TravisCI • CircleCI Who else is using webhooks? @glaforge
  4. 13.

    @glaforge Check runs Check suites Commit comments Branch / tag

    creation Branch / tag deletion Deploy keys Deployments Deployment statuses Forks Wiki Issue comments Issues Labels Collaborator Milestones Page builds Projects Project cards Project columns Visibility changes Pull requests PR reviews PR review comments Pushes Registry packages Releases Repositories Repository imports Repository vuln. alerts Stars Statuses Team adds Watches @glaforge
  5. 16.

    @glaforge Develop & deploy a handler to receive POST requests

    Register the handler’s URL to the service provider Service provider sends a requests to your handler when an event occurs 1 2 3 Add a webhook mechanism to a service provider 0 @glaforge
  6. 45.

    @glaforge Implementing Webhooks CLIENT • Reply with 200 • Reply

    fast • Ack reception and defer work to a worker queue • Calls should be idempotent • IP whitelisting • Check request signature • Take advantage of serverless solutions SERVER • Send small data payloads (re-fetch) • Timeout if client too slow • Retry with exponential backoff • Keep track of non-responding handlers and delivery with a worker queue • Batch events when too frequent • Use a dead letter queue for auditing • Use HTTPS for secured connections • Sign requests with a secret • Use proper authentication / authorization solutions @glaforge
  7. 48.

    @glaforge Resources Crafting a great webhooks experience (John Sheehan) https://speakerdeck.com/apistrat/crafting-a-great-webhooks-experience-by-john-sheehan

    WebHooks: the definitive guide https://requestbin.com/blog/working-with-webhooks/ WebHooks: The API Strikes Back (InfoQ) https://www.infoq.com/presentations/webhooks-api/ Webhooks vs APIs https://hackernoon.com/webhook-vs-api-whats-the-difference-8d41e6661652 What is a Webhooks push-style API & how does it work (ProgrammableWeb) https://www.programmableweb.com/news/what-webhooks-push-styled-api-and-how-does-it-work/analysis/The2017/03/28 Webhooks do’s & dont’s: what we learned after integration 100+ APIs https://restful.io/webhooks-dos-and-dont-s-what-we-learned-after-integrating-100-apis-d567405a3671#.s0qgt1i9p @glaforge
  8. 49.

    @glaforge Resources How & why Pusher adopted Webhooks https://www.programmableweb.com/news/what-are-webhooks-and-how-do-they-enable-real-time-web/2012/01/30 Webhooks

    vs WebSub: Which Is Better For Real-Time Event Streaming? https://nordicapis.com/webhooks-vs-websub-which-one-is-better-to-stream-your-events-in-real-time/ Webhooks, the devil is in the details https://techblog.commercetools.com/webhooks-the-devil-in-the-details-ca7f7982c24f#.u49yswnm7 How to design a webhook for my API https://phalt.github.io/webhooks-in-apis/ Serverless webhooks to revolutionize the SaaS https://tomasz.janczuk.org/2018/03/serverless-webhooks-to-revolutionize-the-saas.html @glaforge