Slide 1

Slide 1 text

Webhooks Lessons (Un)Learned Danger Casey API Problem Solver, GTM Guy, General Nuisance [email protected] July 2023

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Get out your phone ● Open Postman, Insomnia, or curl (for bonus points) ● Tweet at @ngrokHQ if I say something useful Requests for you

Slide 4

Slide 4 text

© ngrok. All rights reserved. Confidential Information of ngrok Agenda 01 Intro 02 Webhooks 101 03 The Fatal Flaw 04 Our Risks 05 What should Providers do? 06 What should Consumers do? 07 Closing / Q&A

Slide 5

Slide 5 text

01 Who am I?

Slide 6

Slide 6 text

© ngrok. All rights reserved. Confidential Information of ngrok Who am I?

Slide 7

Slide 7 text

© ngrok. All rights reserved. Confidential Information of ngrok Who am I? https://www.youtube.com/@geekamongthetrees

Slide 8

Slide 8 text

© ngrok. All rights reserved. Confidential Information of ngrok Why are you talking about this? Ref: https://webhooks.fyi/

Slide 9

Slide 9 text

02 Webhooks 101

Slide 10

Slide 10 text

© ngrok. All rights reserved. Confidential Information of ngrok Webhooks: In the Beginning Ref: https://progrium.github.io/blog/2007/05/03/web-hooks-to-revolutionize-the-web/

Slide 11

Slide 11 text

© ngrok. All rights reserved. Confidential Information of ngrok Sidebar on Jeff Lindsay ● Coined the concept of “web hooks” ● Early employee at Twilio ● Created localtunnel (inspiration for ngrok) ● Architect at dotCloud (which created Docker)

Slide 12

Slide 12 text

© ngrok. All rights reserved. Confidential Information of ngrok Back to webhooks..

Slide 13

Slide 13 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Simple protocol: HTTP ● Simple payload: JSON or XML ● Tech stack agnostic ● Share state between systems ● Building everything yourself sucks ● Super easy to spoof & compromise 😈 ● Er.. I mean super easy to test and mock But why webhooks?

Slide 14

Slide 14 text

03 The Fatal Flaw

Slide 15

Slide 15 text

© ngrok. All rights reserved. Confidential Information of ngrok How do we secure this?

Slide 16

Slide 16 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok Let’s play with it

Slide 17

Slide 17 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok (go to demo)

Slide 18

Slide 18 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok (606) 531-1770

Slide 19

Slide 19 text

04 Our Risks

Slide 20

Slide 20 text

© ngrok. All rights reserved. Confidential Information of ngrok What we’re doing

Slide 21

Slide 21 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Interception ● Impersonation ● Modification/Manipulation ● Replay attacks ● Downtime (missing requests) ● Forward Compatibility ● Multicast Destinations Risks & Considerations

Slide 22

Slide 22 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Interception ● Impersonation ● Modification/Manipulation ● Replay attacks ● Downtime (missing requests) ● Forward Compatibility ● Multicast Destinations Risks & Considerations

Slide 23

Slide 23 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok Let’s play with it abuse it

Slide 24

Slide 24 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok (back to demo)

Slide 25

Slide 25 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok (606) 531-1770

Slide 26

Slide 26 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok Use Postman to POST to https://thatrocks.ngrok.io

Slide 27

Slide 27 text

05 Best Practices for Providers

Slide 28

Slide 28 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok https

Slide 29

Slide 29 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Show the end to end process ○ Including parameters, urls, and options ● Show a payload ○ Yes, all of them ● Demonstrate verification ● Embed verification in your libraries Docs

Slide 30

Slide 30 text

© ngrok. All rights reserved. Confidential Information of ngrok Secure your payload

Slide 31

Slide 31 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Mitigates replay ○ Requires a (somewhat) sync’d clock ○ Choose one format (unix timestamp ftw) ○ Include it in the signature Use the timestamp (or request id) Ref: https://webhooks.fyi/security/replay-prevention

Slide 32

Slide 32 text

© ngrok. All rights reserved. Confidential Information of ngrok Multi-version support Ref: https://webhooks.fyi/ops-experience/versioning ● X-PagerDuty-Signature: v1=f03de6f61df6e454f3620c4d6aca17ad072d3f8bbb2760eac3b2ad 391b5e8073, v2=130dcacb53a94d983a37cf2acba98e805a1c37185309ba56fdcccb cf00d6dd8b ● X-HubSpot-Signature-v2: X-HubSpot-Signature-v3:

Slide 33

Slide 33 text

© ngrok. All rights reserved. Confidential Information of ngrok ● X-PagerDuty-Signature: v1=f03de6f61df6e454f3620c4d6aca17ad072d3f8bbb2760eac3b2ad 391b5e8073, v1=130dcacb53a94d983a37cf2acba98e805a1c37185309ba56fdcccb cf00d6dd8b Zero Downtime (key rotation) Ref: https://webhooks.fyi/ops-experience/versioning

Slide 34

Slide 34 text

© ngrok. All rights reserved. Confidential Information of ngrok State of webhooks

Slide 35

Slide 35 text

© ngrok. All rights reserved. Confidential Information of ngrok State of webhooks (cont)

Slide 36

Slide 36 text

06 Best Practices for Consumers

Slide 37

Slide 37 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok https

Slide 38

Slide 38 text

© ngrok. All rights reserved. Confidential Information of ngrok What we’re doing

Slide 39

Slide 39 text

© ngrok. All rights reserved. Confidential Information of ngrok What we should actually do

Slide 40

Slide 40 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok (back to demo) * with verification

Slide 41

Slide 41 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok (606) 531-1770

Slide 42

Slide 42 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok Use Postman to POST to https://tekrocks.ngrok.io

Slide 43

Slide 43 text

Closing Thoughts 07

Slide 44

Slide 44 text

© ngrok. All rights reserved. Confidential Information of ngrok //todo for providers ● https or death ● Doc all of it - patterns, payloads, & code ● Strong authN methods ● Mitigate replay attacks (timestamps) ● Bonus points: Testing mechanism ● Bonus points: Multi-version support Ref: https://webhooks.fyi/best-practices/webhook-providers

Slide 45

Slide 45 text

© ngrok. All rights reserved. Confidential Information of ngrok ● Transact over https ● Verify senders ● Validate payloads ● Check (on critical events) ● Bonus points: IP restrictions //todo for consumers Ref: https://webhooks.fyi/best-practices/webhook-consumers

Slide 46

Slide 46 text

© ngrok. All rights reserved. Confidential Information of ngrok

Slide 47

Slide 47 text

Thank you

Slide 48

Slide 48 text

© ngrok. All rights reserved. Confidential Information of ngrok © ngrok. All rights reserved. Confidential Information of ngrok About those bonus points for curl If you show me that you used curl, let me know today and I’ll give you a free year of ngrok Pro.

Slide 49

Slide 49 text

https://that.land/43z1eGt

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Webhooks Lessons (Un)Learned Danger Casey API Problem Solver, GTM Guy, General Nuisance [email protected] July 2023