[Confoo 2026] Avoiding Déjà Vu - Building Resilient APIs with Idempotency

Repeated API requests happen more often than we'd like - double clicks, flaky networks, retries, and proxies can all replay calls unexpectedly. Without idempotency, this can lead to duplicate charges, broken state, and confused users.

This talk explores what really goes wrong when idempotency is missing, how companies like Stripe and Shopify design for retries in practice, and why implementing idempotency is often far simpler than we expect!

Transcript

1. Avoiding Déjà Vu: Building Resilient APIs with Idempotency Paul Conroy / @conroyp
2. From Dublin, Ireland Started playing with the web 30+ years ago (Notepad, Frontpage & Geocities!) CTO at Square1 conroyp.com / @conroyp Paul Conroy 👴 🌍 🇮🇪
3. https://en.wikipedia.org/wiki/Déjà_vu
4. https://en.wikipedia.org/wiki/Déjà_vu
5. How does Déjà Vu affect our API servers?
6. 💸 Double charges How does Déjà Vu affect our API servers?
7. 💸 Double charges 📦 Duplicate orders How does Déjà Vu affect our API servers?
8. 💸 Double charges 📦 Duplicate orders 🐛 Data integrity issues How does Déjà Vu affect our API servers?
9. 💸 Double charges 📦 Duplicate orders 🐛 Data integrity issues How does Déjà Vu affect our API servers? 🤬 Angry customers
10. 💸 Double charges 📦 Duplicate orders 🐛 Data integrity issues How does Déjà Vu affect our API servers? 🤬 Angry customers 🗣 Higher support costs
12. What is idempotency?
13. What is idempotency? How do I pronounce
14. eye-dum-po-ten-see
15. https://en.wikipedia.org/wiki/Idempotence “Idem” - same “Potence” - power
16. An operation is idempotent if applying it multiple times has the same effect as applying it once.
20. Idempotency in our APIs
21. 🧑💻
22. 🧑💻
23. 🧑💻
24. 🧑💻
25. 🧑💻
26. 🧑💻 🍔🍟
27. 🧑💻 🍔🍟 💶
28. 🧑💻
29. 🧑💻
30. 🧑💻
31. 🧑💻
32. 🤷
33. 🧑💻
34. 🧑💻
35. 🤷
36. 🤷
37. Reasons for retries • Impatient users • Mobile apps retrying failed connections • Load balancer failovers • CDN fallbacks
38. 💸 Double charges 📦 Duplicate orders 🐛 Data integrity issues 🤬 Angry customers 🗣 Higher support costs Consequences of missing idempotency
39. Who uses it?
40. Who uses it?
41. Who uses it?
42. Who uses it?
43. How can we make sure the same request gets processed only once?
44. Naturally idempotent: • GET • HEAD • OPTIONS HTTP Verbs • PUT • DELETE
45. Naturally idempotent: • GET • HEAD • OPTIONS Non-idempotent verbs: • POST • PATCH HTTP Verbs • PUT • DELETE
46. Naturally idempotent: • GET • HEAD • OPTIONS Non-idempotent verbs: • POST • PATCH HTTP Verbs • PUT • DELETE
47. Naturally idempotent: • GET • HEAD • OPTIONS Non-idempotent verbs: • POST • PATCH HTTP Verbs • PUT • DELETE
48. Naturally idempotent: • GET • HEAD • OPTIONS Non-idempotent verbs: • POST • PATCH HTTP Verbs • PUT • DELETE Observable side-effects
49. Naturally idempotent: • GET • HEAD • OPTIONS Non-idempotent verbs: • POST • PATCH HTTP Verbs • PUT • DELETE Observable side-effects
50. 🧑💻1
51. 🧑💻1 2 🧑💻
52. 🧑💻1 2 🧑💻
53. 🧑💻1 2 🧑💻 🍔🍟
54. 🧑💻1 2 🧑💻 🍔🍟
55. 🧑💻1 2 🧑💻 🍔🍟 💸💸
56. 🧑💻1 2 🧑💻 🍔🍟 💸💸 🍔🍟
57. 🧑💻1 2 🧑💻 🍔🍟 💸💸 🍔🍟
58. 🧑💻1
59. 🧑💻1 1 🧑💻
60. 🧑💻1 1 🧑💻
61. 🧑💻1 1 🧑💻
62. 🧑💻1 1 🧑💻
63. 🧑💻1 1 🧑💻
64. 🧑💻1 1 🧑💻 🍔🍟
65. 🧑💻1 1 🧑💻 🍔🍟
66. 🧑💻1 1 🧑💻 🍔🍟 💶
67. How does it work?
68. 🧑💻 Seen the request before?
69. 🧑💻 Seen the request before? Generate response No
70. 🧑💻 Seen the request before? Generate response Save response to cache No
71. 🧑💻 Seen the request before? Generate response Save response to cache Return response No
72. 🧑💻 Seen the request before? Retrieve response from cache Generate response Save response to cache Return response Yes No
73. Have we seen the request before?
74. Use a unique hash per request, storing it as a cache key. But where do we get the hash from? How do we know if we’ve seen a request before?
75. Request Body Build up hash using all parameters contained in the request order
76. Request Body Build up hash using all parameters contained in the request order 🧑💻 🍔🍟
77. Request Body Build up hash using all parameters contained in the request order 🧑💻 🧑💻 🍔🍟 🍔🍟
78. Request Body Build up hash using all parameters contained in the request order 🧑💻 🧑💻 🍔🍟 🍔🍟 🏢
79. IP Address Include user’s public IP address in hash 🧑💻 🍔🍟 104.17.3.109
80. IP Address Include user’s public IP address in hash 🧑💻 🧑💻 🍔🍟 🍔🍟 🏢 104.17.3.109 104.17.3.109
81. user_id: 3792 User ID Logged-in user’s ID 🧑💻 🧑💻 🍔🍟 🍔🍟 🏢 user_id: 2552
82. user_id: 3792 User ID Logged-in user’s ID 🧑💻 🧑💻 🍔🍟 🍔🍟 🏢 user_id: 2552 user_id: null user_id: null
83. How do we know if we’ve seen a request before? Use a unique hash per request, storing it as a cache key. But where do we get the hash from?
84. How do we know if we’ve seen a request before? • Request body - but duplicate orders! Use a unique hash per request, storing it as a cache key. But where do we get the hash from?
85. How do we know if we’ve seen a request before? • Request body - but duplicate orders! • IP address - but shared public IPs! Use a unique hash per request, storing it as a cache key. But where do we get the hash from?
86. How do we know if we’ve seen a request before? • Request body - but duplicate orders! • IP address - but shared public IPs! • User ID - but guest checkouts! Use a unique hash per request, storing it as a cache key. But where do we get the hash from?
87. How do we know if we’ve seen a request before? • Request body - but duplicate orders! • IP address - but shared public IPs! • User ID - but guest checkouts! Make the client do some work! Use a unique hash per request, storing it as a cache key. But where do we get the hash from?
88. 🧑💻1 1 🧑💻 🍔🍟 💶
89. 🧑💻1 1 🧑💻 🍔🍟 💶 Idempotency-Key: 1 Idempotency-Key: 1
90. 🧑💻Idempotency-Key: 1 • Make the client pass a key with each idempotent request • Use this as the basis for our server-side cache • Typically use UUIDs to avoid collisions • Client SDKs help with key generation
91. https://github.com/stripe/stripe-php/blob/master/lib/HttpClient/CurlClient.php#L271
92. https://github.com/stripe/stripe-php/blob/master/lib/HttpClient/CurlClient.php#L271
93. 🧑💻 Seen the request before? Idempotency-Key: 1
94. 🧑💻 Seen the request before? Generate response No Idempotency-Key: 1
95. 🧑💻 Seen the request before? Generate response Save response to cache No Idempotency-Key: 1
96. 🧑💻 Seen the request before? Generate response Save response to cache Return response No Idempotency-Key: 1
97. 🧑💻 Seen the request before? Retrieve response from cache Generate response Save response to cache Return response Yes No Idempotency-Key: 1
98. How does it work?
99. How does it really work?
100. Application 🧑💻
101. Application Middleware 🧑💻
102. Application Middleware 🧑💻
109. Make app generate response
110. Make app generate response Put whole structure into cache
111. Make app generate response Put whole structure into cache Cache duration!
113. Pull response from cache
114. Pull response from cache Has the key been used elsewhere?
115. Pull response from cache Has the key been used elsewhere? Show it’s a replayed request
118. Internal use?
119. Make app generate response Put whole structure into cache Cache duration!
120. Choose a TTL based on retry behaviour, business impact, and storage constraints How long should we cache for? ⏳ Short TTL (Mins - Hours) ⏱ Medium TTL (Hours to Days) 📅 Long TTL (Days to Weeks) 🔒 Infinite TTL (Persistent Storage)
121. Scenario: Mobile app order processing (Food delivery application) User Behaviour: Users place orders on their phones while outside. Retry Pattern: The mobile app automatically retries failed requests up to 3 times within a 5-minute window. Key Selection Strategy: UUIDs generally sufficient. ⏳ Short TTL (Mins - Hours) • Most connectivity issues resolve within minutes • Retries typically happen almost immediately or within a few minutes • After this window, a new order attempt is likely genuinely new
122. Scenario: Batch payment processing (B2B SaaS platform) User Behaviour: Businesses run scheduled payment batch jobs that process hundreds of transactions. Retry Pattern: Failed batches are retried same day or the next morning. Key Selection Strategy: UUIDs still generally sufficient - user/session identifiers may be helpful for multi-day caches. ⏱ Medium TTL (Hours to Days) • Business hours and operational patterns dictate retry windows • System failures may take several hours to resolve • Next-day retries are common in business workflows
123. Scenario: Subscription Management System (B2B SaaS platform handling subscription payments) User Behaviour: Users change subscription tiers, add features, etc. Retry Pattern: Support teams need to reprocess failed changes days later. Key Selection Strategy: Consider additional business context added to the key. 📅 Long TTL (Days to Weeks) • Customer service tickets often take days to resolve • Subscription changes have billing cycle implications • Retry attempts may happen after delays with customer communication
124. Scenario: Regulatory Compliance Reporting User Behaviour: System submits mandatory reports to govt agencies. Retry Pattern: Failed submissions retried indefinitely until successful, but must never be duplicated. Key Selection Strategy: Additional metadata about requester. 🔒 Infinite TTL (Persistent Storage) • Regulatory requirements prohibit both missed and duplicate reports • Legal penalties for non-compliance are severe • The reporting requirement never expires
125. Should errors be cached? Should we allow retries? What about errors? https://docs.stripe.com/api/idempotent_requests
126. Should errors be cached? Should we allow retries? What about errors? https://docs.stripe.com/api/idempotent_requests
127. Should errors be cached? Should we allow retries? What about errors? https://docs.stripe.com/api/idempotent_requests
128. Race conditions 🧑💻 ⏳
129. Race conditions 🧑💻 ⏳
130. Race conditions 🧑💻 ⏳
131. Cache locking Using a lock for the process allows us to ensure only one process handles the request.
 • Try to get a cache lock • If we can, process the request then release the lock • If we can’t, the same request is being processed by another process. Wait until it’s done
132. Cache locking
133. Cache locking Wait for other process to finish
134. Cache locking Wait for other process to finish Release the lock afterwards!
136. Another process finished it?
137. Another process finished it? Sleep for a moment, check again
138. Another process finished it? Sleep for a moment, check again Something has gone wrong!
139. https://github.com/square1-io/laravel-idempotency
140. Laravel Implementation
142. Race conditions
143. Race conditions User ID resolver
144. Race conditions User ID resolver Replay or throw exception?
145. https://apisyouwonthate.com/blog/idemptoency-keys/ Error on idempotent retry after success
146. Implementing Idempotency • Decide on the endpoints which need it • Select appropriate key cache TTLs • Document idempotent operations in your API • Allow your users to trust an operation is idempotent
147. Implementing Idempotency • Decide on the endpoints which need it • Select appropriate key cache TTLs • Document idempotent operations in your API • Allow your users to trust an operation is idempotent • Decide on the endpoints which need it • Select appropriate key cache TTLs • Document idempotent operations in your API
149. Registered As Canonical Username User ID Bigbird bigbird 123
150. Registered As Canonical Username User ID Bigbird bigbird 123
151. Registered As Canonical Username User ID Bigbird bigbird 123 BIGBIRD bigbird
152. Registered As Canonical Username User ID Bigbird bigbird 123 BIGBIRD bigbird
153. BiGbIrD Bigbird BIGBIRD BigBird getCanonicalUsername(user) bigbird
154. BiGbIrD Bigbird BIGBIRD BigBird getCanonicalUsername(user) bigbird
155. Registered As Canonical Username User ID Bigbird bigbird 123
156. Registered As Canonical Username User ID Bigbird bigbird 123 ᴮᴵᴳᴮᴵᴿᴰ
157. Registered As Canonical Username User ID Bigbird bigbird 123 ᴮᴵᴳᴮᴵᴿᴰ u'\u1d2e\u1d35\u1d33\u1d2e\u1d35\u1d3f\u1d30'
158. Registered As Canonical Username User ID Bigbird bigbird 123 ᴮᴵᴳᴮᴵᴿᴰ BIGBIRD u'\u1d2e\u1d35\u1d33\u1d2e\u1d35\u1d3f\u1d30'
159. Registered As Canonical Username User ID Bigbird bigbird 123 ᴮᴵᴳᴮᴵᴿᴰ BIGBIRD u'\u1d2e\u1d35\u1d33\u1d2e\u1d35\u1d3f\u1d30' 456
160. 1. Sign up with unicode username
161. 2. Trigger “forgot password” flow
162. 3. Submit “Forgot password” form
163. 3. Submit “Forgot password” form 🚨🚨🚨
164. 4. Take over account!
165. • getCanonicalUsername relied on nodeprep.prepare • Input string not being valid Unicode 3.2 meant nodeprep.prepare was no longer idempotent!
 • Fixed by double-checking username, and subsequently a library update What went wrong?
166. An operation is idempotent if applying it multiple times has the same effect as applying it once.
167. • Relying on server side hashes alone to identify repeat requests is risky • Make the client do the work! • Cache TTL appropriate to your use case • Don’t worry about verbs that are already idempotent • But don’t forget about DELETE! • Replay behaviour - document your choice! Takeaways
168. Idempotency saves your API from having to re-live the same request, over and over again
169. Idempotency saves your API from having to re-live the same request, over and over again
170. • Making retries safe with idempotent APIs (AWS)
 https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/ • Designing robust and predictable APIs with idempotency (Stripe) https://stripe.com/blog/idempotency • Creative usernames and Spotify account hijacking (Spotify) https://engineering.atspotify.com/2013/06/creative-usernames/ • Idempotency - what is it, and how can it help our Laravel APIs? https://www.conroyp.com/articles/what-is-idempotency-add-to-laravel-apis • Laravel Idempotency Package https://github.com/square1-io/laravel-idempotency Further reading
171. Thank you! 🌍 🌐 📧 Conroyp.com @conroyp paul@conroyp.com