Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Crafting a Great Webhooks Experience

Crafting a Great Webhooks Experience

Presented at API Craft SF on 8/21/14

John Sheehan

August 21, 2014
Tweet

More Decks by John Sheehan

Other Decks in Technology

Transcript

  1. Crafting a Great
    Webhooks Experience
    John Sheehan
    CEO, @Runscope
    Tuesday, October 7, 14

    View Slide

  2. Tuesday, October 7, 14

    View Slide

  3. Tuesday, October 7, 14

    View Slide

  4. Tuesday, October 7, 14

    View Slide

  5. Tuesday, October 7, 14

    View Slide

  6. Tuesday, October 7, 14

    View Slide

  7. "user defined
    callbacks made
    with HTTP POST"
    Tuesday, October 7, 14

    View Slide

  8. "Webhooks are the
    easiest way to remotely
    execute code."
    -- Jeff Lindsay once
    when we were talking
    Tuesday, October 7, 14

    View Slide

  9. HTTP Push
    Notifications
    Tuesday, October 7, 14

    View Slide

  10. A Reverse API
    Tuesday, October 7, 14

    View Slide

  11. Provider makes request to
    URL when an event happens.
    Consumer sets up a server to
    listen for callbacks.
    Consumer registers callback
    URL with provider.
    Tuesday, October 7, 14

    View Slide

  12. Provider makes request to
    URL when an event happens.
    Consumer sets up a server to
    listen for callbacks.
    Consumer registers callback
    URL with provider.
    Tuesday, October 7, 14

    View Slide

  13. Provider makes request to
    URL when an event happens.
    Consumer sets up a server to
    listen for callbacks.
    Consumer registers callback
    URL with provider.
    Tuesday, October 7, 14

    View Slide

  14. Tuesday, October 7, 14

    View Slide

  15. Implementing
    Webhooks
    Tuesday, October 7, 14

    View Slide

  16. url = get_callback_url()
    data = get_webhook_payload_json()
    try:
    resp = requests.post(url, data=data)
    if not resp.ok:
    _logger.error(resp.content)
    except Exception as e:
    _logger.error(e)
    Tuesday, October 7, 14

    View Slide

  17. Problem #1:
    Error Handling
    Tuesday, October 7, 14

    View Slide

  18. > POST /callback
    < 400 Bad Request
    Tuesday, October 7, 14

    View Slide

  19. > POST /callback
    < 302 Found
    < Location: http://
    Tuesday, October 7, 14

    View Slide

  20. > POST /callback
    < 200 OK
    < Content-Type: text/plain
    <
    Tuesday, October 7, 14

    View Slide

  21. Error Handling
    Suggestions
    Tuesday, October 7, 14

    View Slide

  22. Be lenient in what you accept
    back if you can reasonably guess.
    Retry failed callbacks with
    exponential back off.
    Decide if redirects are to be
    followed or not.
    Tuesday, October 7, 14

    View Slide

  23. Be lenient in what you accept
    back if you can reasonably guess.
    Retry failed callbacks with
    exponential back off.
    Decide if redirects are to be
    followed or not.
    Tuesday, October 7, 14

    View Slide

  24. Be lenient in what you accept
    back if you can reasonably guess.
    Retry failed callbacks with
    exponential back off.
    Decide if redirects are to be
    followed or not.
    Tuesday, October 7, 14

    View Slide

  25. Problem #2:
    Flooding
    Tuesday, October 7, 14

    View Slide

  26. Tuesday, October 7, 14

    View Slide

  27. Active
    Queues


    Tuesday, October 7, 14

    View Slide

  28. Problem #3:
    Security
    Tuesday, October 7, 14

    View Slide

  29. > POST http://localhost:3000
    Tuesday, October 7, 14

    View Slide

  30. > POST http://foo.lvh.me
    Tuesday, October 7, 14

    View Slide

  31. DoS Attack Vector
    Tuesday, October 7, 14

    View Slide

  32. Proving the Source
    Tuesday, October 7, 14

    View Slide

  33. Validation
    Techniques
    Tuesday, October 7, 14

    View Slide

  34. Key Sharing
    Tuesday, October 7, 14

    View Slide

  35. Request
    Signing
    Tuesday, October 7, 14

    View Slide

  36. Re-fetch
    > POST /callback
    > { id: 123 }
    > GET /users/123
    < { id: 123 }
    Webhook Callback
    App Code
    Tuesday, October 7, 14

    View Slide

  37. Security
    Suggestions
    Tuesday, October 7, 14

    View Slide

  38. Validate your requests.
    Document it well!
    Resolve IPs before making
    request. Consider proxying.
    Consider subscription validation
    for high-volume cases.
    Tuesday, October 7, 14

    View Slide

  39. Validate your requests.
    Document it well!
    Resolve IPs before making
    request. Consider proxying.
    Consider subscription validation
    for high-volume cases.
    Tuesday, October 7, 14

    View Slide

  40. Validate your requests.
    Document it well!
    Resolve IPs before making
    request. Consider proxying.
    Consider subscription validation
    for high-volume cases.
    Tuesday, October 7, 14

    View Slide

  41. Developer
    Experience
    Tuesday, October 7, 14

    View Slide

  42. Payload Design
    Tuesday, October 7, 14

    View Slide

  43. Fat vs.Thin
    Tuesday, October 7, 14

    View Slide

  44. - or -
    { }
    payload=
    Tuesday, October 7, 14

    View Slide

  45. - or -
    data = JSON.loads(request.body)
    name = data["name"]
    name = request.form.get("name")
    Tuesday, October 7, 14

    View Slide

  46. payload = request.form.get("payload")
    data = JSON.loads(payload)
    name = data["name"]
    Tuesday, October 7, 14

    View Slide

  47. Mirror API
    Resources
    Tuesday, October 7, 14

    View Slide

  48. Complete
    Documentation!
    Tuesday, October 7, 14

    View Slide

  49. Tooling
    Tuesday, October 7, 14

    View Slide

  50. Accept Multiple
    Callback URLs
    Tuesday, October 7, 14

    View Slide

  51. Hooks API
    Tuesday, October 7, 14

    View Slide

  52. Debugger &
    Logs
    Tuesday, October 7, 14

    View Slide

  53. Manual
    Retries
    Tuesday, October 7, 14

    View Slide

  54. Generate
    Test Callbacks
    Tuesday, October 7, 14

    View Slide

  55. Tunneling
    Tuesday, October 7, 14

    View Slide

  56. Thank you!
    Questions?
    Try Runscope free:
    runscope.com
    Tuesday, October 7, 14

    View Slide