The Story of why we migrate to gRPC and how we go about it

At Spotify, we have historically built services based on our own proprietary messaging protocol and framework. Last year we finally kicked off the daunting multi-year task of migrating everything to gRPC.

Migrating over 1000 services to gRPC poses interesting challenges, many of them are not only technical in nature: they boil down to questions of engineering culture and leadership at scale: How do you get 200 autonomous engineering teams to align on something as fundamental and cross-cutting as a new RPC framework? How do you roll-out gRPC at scale whit minimal disruption to both the organization and to the end-user?

This presentation will address these questions alongside more technical discussions of advanced gRPC concepts such as interceptors, deadlines, and effective schema management and how they are essential in large distributed systems.

Transcript

1. The Story of why we migrate to gRPC and how we go about it @mattgruter 2019-05-22
2. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter “Developers don’t care about new RPC technologies” -- someone at KubeCon 2018
3. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter
4. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Why? Why are we doing this? How? How do we get there?
5. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Why?
6. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Spotify’s Infrastructure 200M active monthly users 8M QPS at ingress 1k developers 2.5k services Java, Python, ...
7. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Hermes Not this Hermes!
8. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Hermes But this
9. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Hermes - Written in 2012 - Based on ZeroMQ - JSON or Protobuf payload - Not a RPC framework
10. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Hermes works!
11. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Why Change?
12. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Hermes Ecosystem
13. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 404 Not Found
14. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter From DIY to FOSS
15. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Why gRPC?
16. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter The Proto
17. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Code Generation Java, Golang, Python, Ruby, Dart, PHP, Node.js, Objective-C, C#, C++
18. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Schema Management 1. Embrace the proto
19. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Schema Management 2. Shared repo for all protos
20. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Schema Management 3. Version on the proto package package spotify.playlist.v1; package spotify.playlist.v2beta1;
21. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Resiliency
22. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Resiliency: Retries
23. Retry Hedging Proposal
24. Thundering Herd
25. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Resiliency: Retry Throttling & Pushback Proposal
26. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Resiliency: Deadlines
27. Deadlines
28. Deadlines
29. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Load Balancing & Routing
30. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Load Balancing Server-side proxy dumb client extra hop Client-side no proxy complex client low latency
31. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Load Balancing Server-side proxy dumb client extra hop Client-side no proxy complex client low latency Lookaside no proxy thinner client low latency
32. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Lookaside Load Balancing
33. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Proxyless RPC mesh
34. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Why gRPC?
35. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter
36. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter
37. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter
38. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter How?
39. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes gRPC
40. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes gRPC 10% Distance Travelled 90% Remaining Distance to Destination
41. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes Not Hermes 10% Distance Travelled 90% Remaining Distance to Destination
42. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes gRPC 10% Distance Travelled 90% Remaining Distance to Destination
43. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Main Challenge Change is hard
44. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter “This is not the protocol you’re looking for”
45. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Main Challenge Change is hard
46. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Autonomous Teams
47. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Trust
48. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Autonomous Teams Autonomy != Freedom
49. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Henrik Kniberg / Spotify — Aligned Autonomy
50. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Aligned Autonomy
51. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Spotify’s Golden Path
52. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Align by making the right choice. the easy choice.
53. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 2. Don’t force it!
54. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 2. Don’t force it! Code generation
55. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 2. Don’t force it! Code generation Resiliency patterns
56. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 2. Don’t force it! Code generation Resiliency patterns Tracing
57. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 2. Don’t force it! Code generation Resiliency patterns Tracing <Fancy Tool X>
58. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter aka. never-ending migration aka. yet another protocol Hermes gRPC In case of emergency
59. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter 3. Force it!
60. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter $ sudo migrate
61. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes gRPC 10% Distance Travelled 90% Remaining Distance to Destination
62. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes gRPC
63. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Our Journey Hermes gRPC
64. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter “Developers don’t care about new RPC technologies” -- overheard at KubeCon 2018 Seattle
65. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter “Developers don’t have to care about new RPC technologies” -- “overheard” at KubeCon 2019 Barcelona
66. gRPC at Spotify- KubeCon/CloudNativeCon Europe 2019 - @mattgruter Thank You Matthias Grüter @mattgruter