Build Backwards-Compatible REST APIs with Rolling Versions

This is the slide to the talk I gave at SysConf 2023 by SysDsgn.

Transcript

1. Build Backwards-Compatible REST APIs with Rolling Versions Subomi Oluwalana
2. Hi, 󰗜 My name is Subomi Oluwalana, I’m a Software Engineer and the Co-Founder & CEO at Convoy.
3. Agenda 1. Introduction 2. Background 3. What are Rolling Versions? 4. The Golang Implementation 5. References
4. Background
5. - Improve the REST API without breaking old integrations. - Common wisdom is semantic versioning - /api/v1, /api/v2 - Incentivises big-bang changes. - Splits user base between versions. - We believe infrastructure should be boring. - Enter Rolling Versions.
6. What are Rolling Versions? Rolling Versions is an API versioning technique that ensures backward-compatibility by applying request and response transformation to each user request using change objects.
7. Image Credits: Keygen.sh
8. Image Credits: Keygen.sh v1.1 request v1.1 response
9. A quick example In this example, we describe an API used to retrieve a list of user objects, with the date versioning scheme. Versions - Initial Version - 2023-04-01 - 2023-05-01
10. A quick example
11. Demo time!
12. The Golang Implementation github.com/subomi/requestmigrations
13. The Migration Map It maps all the transformations to their respective versions.
14. Registering Migrations Here’s the code from the previous example.
15. The Migration Interface Ensure to set the body back with a io.NopCloser
16. Configuring the Library Retrieve version from request header. Retrieve version from the database.
17. Usage Wrap specific request handlers only. Use as a middleware, wrap the entire router.
18. Instrumentation Outputs a prometheus histogram metric to debug transformation latencies and stale versions.
19. Instrumentation Todo: Add graphs.
20. Instrumentation Todo: Add graphs.
21. Limitations httptest.ResponseRecorder - Designed for the test environment. - Enabled Quick Prototyping. - Backwards Compatible with Handlers. - Works with early versions of Go. http.ResponseController - Designed for regular production code. - Might not be Backwards Compatible with your Handlers. - Works with only Go 1.20+
22. Parting thoughts - Some changes to the API are much more complex than response transformations. See Changes with side effects in the Stripe article. - Handling arbitrary data migrations, e.g. Webhook payloads.
23. 1. Avoid N+1 Queries in your transformations. 2. Monitor versioned request latencies. 3. Perform end-to-end tests to verify correctness.
24. References - Rails Implementation - github.com/keygen-sh/request_migrations - Laravel Implementation - github.com/tomschlick/request-migrations - Backwards–compatibility in Cloudflare Workers - blog.cloudflare.com/backwards-compatibility-in-cloudflare-workers - Infrastructure should be boring - dragonflydb.io/blog/infrastructure-should-be-boring - APIs as Infrastructure: future-proofing Stripe with versioning - stripe.com/blog/api-versioning
25. @subomioluwalana @subomiOluwalana Subomi Oluwalana me@subomioluwalana.com