Crafting a Great Webhooks Experience

Presented at API Craft SF on 8/21/14

Transcript

1. Crafting a Great
   Webhooks Experience
   John Sheehan
   CEO, @Runscope
   Tuesday, October 7, 14
   

   View Slide

2. Tuesday, October 7, 14
   

   View Slide

3. Tuesday, October 7, 14
   

   View Slide

4. Tuesday, October 7, 14
   

   View Slide

5. Tuesday, October 7, 14
   

   View Slide

6. Tuesday, October 7, 14
   

   View Slide

7. "user defined
   callbacks made
   with HTTP POST"
   Tuesday, October 7, 14
   

   View Slide

8. "Webhooks are the
   easiest way to remotely
   execute code."
   -- Jeff Lindsay once
   when we were talking
   Tuesday, October 7, 14
   

   View Slide

9. HTTP Push
   Notifications
   Tuesday, October 7, 14
   

   View Slide

10. A Reverse API
    Tuesday, October 7, 14
    

    View Slide

11. Provider makes request to
    URL when an event happens.
    Consumer sets up a server to
    listen for callbacks.
    Consumer registers callback
    URL with provider.
    Tuesday, October 7, 14
    

    View Slide

12. Provider makes request to
    URL when an event happens.
    Consumer sets up a server to
    listen for callbacks.
    Consumer registers callback
    URL with provider.
    Tuesday, October 7, 14
    

    View Slide

13. Provider makes request to
    URL when an event happens.
    Consumer sets up a server to
    listen for callbacks.
    Consumer registers callback
    URL with provider.
    Tuesday, October 7, 14
    

    View Slide

14. Tuesday, October 7, 14
    

    View Slide

15. Implementing
    Webhooks
    Tuesday, October 7, 14
    

    View Slide

16. url = get_callback_url()
    data = get_webhook_payload_json()
    try:
    resp = requests.post(url, data=data)
    if not resp.ok:
    _logger.error(resp.content)
    except Exception as e:
    _logger.error(e)
    Tuesday, October 7, 14
    

    View Slide

17. Problem #1:
    Error Handling
    Tuesday, October 7, 14
    

    View Slide

18. > POST /callback
    < 400 Bad Request
    Tuesday, October 7, 14
    

    View Slide

19. > POST /callback
    < 302 Found
    < Location: http://
    Tuesday, October 7, 14
    

    View Slide

20. > POST /callback
    < 200 OK
    < Content-Type: text/plain
    <
    Tuesday, October 7, 14
    

    View Slide

21. Error Handling
    Suggestions
    Tuesday, October 7, 14
    

    View Slide

22. Be lenient in what you accept
    back if you can reasonably guess.
    Retry failed callbacks with
    exponential back off.
    Decide if redirects are to be
    followed or not.
    Tuesday, October 7, 14
    

    View Slide

23. Be lenient in what you accept
    back if you can reasonably guess.
    Retry failed callbacks with
    exponential back off.
    Decide if redirects are to be
    followed or not.
    Tuesday, October 7, 14
    

    View Slide

24. Be lenient in what you accept
    back if you can reasonably guess.
    Retry failed callbacks with
    exponential back off.
    Decide if redirects are to be
    followed or not.
    Tuesday, October 7, 14
    

    View Slide

25. Problem #2:
    Flooding
    Tuesday, October 7, 14
    

    View Slide

26. Tuesday, October 7, 14
    

    View Slide

27. Active
    Queues
    ↪
    ↪
    Tuesday, October 7, 14
    

    View Slide

28. Problem #3:
    Security
    Tuesday, October 7, 14
    

    View Slide

29. > POST http://localhost:3000
    Tuesday, October 7, 14
    

    View Slide

30. > POST http://foo.lvh.me
    Tuesday, October 7, 14
    

    View Slide

31. DoS Attack Vector
    Tuesday, October 7, 14
    

    View Slide

32. Proving the Source
    Tuesday, October 7, 14
    

    View Slide

33. Validation
    Techniques
    Tuesday, October 7, 14
    

    View Slide

34. Key Sharing
    Tuesday, October 7, 14
    

    View Slide

35. Request
    Signing
    Tuesday, October 7, 14
    

    View Slide

36. Re-fetch
    > POST /callback
    > { id: 123 }
    > GET /users/123
    < { id: 123 }
    Webhook Callback
    App Code
    Tuesday, October 7, 14
    

    View Slide

37. Security
    Suggestions
    Tuesday, October 7, 14
    

    View Slide

38. Validate your requests.
    Document it well!
    Resolve IPs before making
    request. Consider proxying.
    Consider subscription validation
    for high-volume cases.
    Tuesday, October 7, 14
    

    View Slide

39. Validate your requests.
    Document it well!
    Resolve IPs before making
    request. Consider proxying.
    Consider subscription validation
    for high-volume cases.
    Tuesday, October 7, 14
    

    View Slide

40. Validate your requests.
    Document it well!
    Resolve IPs before making
    request. Consider proxying.
    Consider subscription validation
    for high-volume cases.
    Tuesday, October 7, 14
    

    View Slide

41. Developer
    Experience
    Tuesday, October 7, 14
    

    View Slide

42. Payload Design
    Tuesday, October 7, 14
    

    View Slide

43. Fat vs.Thin
    Tuesday, October 7, 14
    

    View Slide

44. - or -
    { }
    payload=
    Tuesday, October 7, 14
    

    View Slide

45. - or -
    data = JSON.loads(request.body)
    name = data["name"]
    name = request.form.get("name")
    Tuesday, October 7, 14
    

    View Slide

46. payload = request.form.get("payload")
    data = JSON.loads(payload)
    name = data["name"]
    Tuesday, October 7, 14
    

    View Slide

47. Mirror API
    Resources
    Tuesday, October 7, 14
    

    View Slide

48. Complete
    Documentation!
    Tuesday, October 7, 14
    

    View Slide

49. Tooling
    Tuesday, October 7, 14
    

    View Slide

50. Accept Multiple
    Callback URLs
    Tuesday, October 7, 14
    

    View Slide

51. Hooks API
    Tuesday, October 7, 14
    

    View Slide

52. Debugger &
    Logs
    Tuesday, October 7, 14
    

    View Slide

53. Manual
    Retries
    Tuesday, October 7, 14
    

    View Slide

54. Generate
    Test Callbacks
    Tuesday, October 7, 14
    

    View Slide

55. Tunneling
    Tuesday, October 7, 14
    

    View Slide

56. Thank you!
    Questions?
    Try Runscope free:
    runscope.com
    Tuesday, October 7, 14
    

    View Slide