Evolving big microservice architectures

Microservices are a well-established architecture applied by many organizations around the world to build scalable and fault-tolerant backend systems. But as these systems grow so does the number of services in them. And this brings many challenges when we want to introduce new functionality. For a simple feature, engineers may need to spend a lot of time designing the end to end flow, changing code in multiple services and setting up complex test scenarios. During this talk, we’ll explore how to evolve a microservice architecture to be easily extensible based on some lessons learned from running 5000 microservices in production. We’ll go over different architectural patterns and open source tools that we can use to make it easy for all engineers to understand, extend and be more and more productive in such big complex systems.

Transcript

1. Evolving big microservice architectures Nikolay Stoitsev, Engineering Manager @ Uber Engineering
4. Dispatch API
5. A monolith? Code was consistently good Easy to reuse code Well tested Hard to deploy
6. Pull Request Pull Request Pull Request Pull Request Pull Request ? ? ? ? ?
7. Pull Request Pull Request Pull Request Pull Request Pull Request ? ? ? ? ? Unknown risk Unknown priority
8. Low Risk High Risk High Risk Med Risk High Risk Build
9. Low Risk High Risk High Risk Med Risk High Risk Build ?
10. Low Risk High Risk High Risk Med Risk High Risk Build
11. Low Risk / P0 High Risk / P1 High Risk / P0 Med Risk / P3 High Risk / P0 Build
12. Low Risk / P0 High Risk / P1 High Risk / P0 Med Risk / P3 High Risk / P0 Build CI / CD
13. Gradual rollout is important
14. Lesson learned: A monolith can scale a lot, but...
15. Dispatch API
16. Dispatch API Users Trips Receipts Payments
17. Dispatch API Users Trips Receipts Payments
18. Dispatch API Users Trips Receipts Payments
19. Dispatch API Users Trips Receipts Payments
21. 4000+ microservices 1K+ avg deploys a day
22. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka Sample architecture
23. We want to introduce Uber Elevate
24. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka
25. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 Apache Kafka Apache Kafka Apache Kafka
26. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Apache Kafka Apache Kafka Apache Kafka
27. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Apache Kafka Apache Kafka Apache Kafka
28. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Task #3 Apache Kafka Apache Kafka Apache Kafka
29. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Task #3 Task #4 Apache Kafka Apache Kafka Apache Kafka
30. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Task #3 Task #4 Task #5 Apache Kafka Apache Kafka Apache Kafka
31. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Task #3 Task #4 Task #5 OK Apache Kafka Apache Kafka Apache Kafka
32. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Task #3 Task #4 Task #5 OK Task #6 Apache Kafka Apache Kafka Apache Kafka
33. How do we evolve big microservice architectures?
34. You can’t change what you can’t test
35. Staging environment?
36. Integration tests?
37. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Change Apache Kafka Apache Kafka Apache Kafka
38. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Change MOCK MOCK MOCK MOCK Apache Kafka Apache Kafka Apache Kafka
39. Multitenancy
40. Tenancy - set of data that is isolated from the other sets of data
41. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Change Apache Kafka Apache Kafka Apache Kafka
42. Testing tenancy vs. Production tenancy
43. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Change Test Data Test Config Test Config Test Data Apache Kafka Apache Kafka Apache Kafka
44. Audit tenancy
45. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Audit tenancy traffic Check Test Config Test Config Test Data Apache Kafka Apache Kafka Apache Kafka Check Check Check
46. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Task #1 OK Task #2 Task #3 Task #4 Task #5 OK Apache Kafka Apache Kafka Apache Kafka
47. Platforms
48. Services are too concrete so it’s hard to support new use cases
49. Services need to support abstraction
50. AWS is a good example
51. Dispatch Pricing Services Tax Calc Vehicles Trips Services Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka
52. Dispatch Pricing Services Tax Calc Vehicles Orders Platform Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka
53. Dispatch Pricing Services Tax Calc Vehicles Orders Platform Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka
54. Dispatch Pricing Services Tax Calc Vehicles Orders Platform Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka
55. Hard shift in mentality in the way you design your services
56. Can you sell it outside of the company?
57. Tax Calc Payments Pricing
58. Tax Calc Payments Pricing 3.14159
59. Tax Calc Payments Pricing 3.14159 “3.14”
60. Tax Calc Payments Pricing 3.14159 “3.14” 314159
61. Common data model
64. Data extensions = schema + payload
65. How to do it? Schema service Repo Library ...
66. Dispatch Pricing Services Tax Calc Vehicles Orders Platform Users Payments Services Receipts Apache Kafka Apache Kafka Apache Kafka
67. … but there is still business specific logic.
68. Dispatch Pricing Services Tax Calc Vehicles Orders Platform Users Payments Services Receipts Where to put it? Business logic Apache Kafka Apache Kafka Apache Kafka
69. No common understanding of what should go where
70. How do we call code with business logic, presentation logic, and SQL queries all over the place?
71. We fix spaghetti code with separation of concerns
72. Presentation Logic Data
73. Layered architecture
74. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend
75. Why?
76. Common language
77. Clear separation of concerns
78. Consistent patterns
79. API and functionality discoverability
80. Engineering velocity
81. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Generic Specific
82. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Breaks everything Breaks specific thing
83. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Presentation LoB Biz Infra Core
84. https://philcalcado.com/2018/09/24/services_layers.html Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Clay Rocks
85. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Presentation
86. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend LoB
87. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Biz Infra
88. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend Core
89. Clear API
90. protobuf Apache Thrift GraphQL
91. End result: Layered multitenant architecture built with abstract platforms and a common data model
92. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Web Services Mobile Backend
93. Users Vehicles Order Platform Pricing Tax Calc Payments Receipts Rides Elevate Web Services Mobile Backend
94. Q&A Feedback:
95. Proprietary © 2019 Uber Technologies, Inc. All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber.