Microservices Architecture in the Real World - RedisConf2019

Once you decide to adopt a microservices architecture, you’ll face many more decisions and questions. Among them are some core areas that this session will cover, based on our experiences making this shift at Credit Karma. Over the past three years, Credit Karma has gone from zero to over 100 microservices, supporting over 400 engineers while serving our more than 90 million members. The talk will dive into detail about Routing (how does service A find and call service B?), Management (how do you handle hundreds of containers?), Observability (how do you know what’s going on out there?), and developer Experience (how do your developers deal with this shift?).

Transcript

1. PRESENTED BY Microservices Architecture in the Real World Mason Jones @masonoise Senior Staff Infrastructure Engineer
2. 2 Credit Karma champions financial progress
 for everyone
3. 3 900+ employees, 50% Engineering Over 90 million members Over 100 microservices now, from zero in 2016
4. 4 01 Introduction So, you've decided to adopt microservices!
7. 7 Introduction Routing Management Observability 01 02 03 04 User Experience 05 Now you need to think about...
8. 8 What you imagine your system will look like.
9. 9 What your system will really look like.
10. 10 Engineering agility Team independence System flexibility Technology choices Benefits of Microservices
11. 11 02 Routing
12. 12 How does service A make a request to service B? Seems like a simple question, right? Service Routing ROUTING
13. 13 Where are (healthy) instances running? How does service A find them? Define "healthy"? Service A calling Service B ROUTING
14. 14 Service Registry (Consul, Zookeeper…) DNS (but no port # w/o SRV record) Compute instances behind load-balancer Service A calling Service B ROUTING
15. 15 Service Registry using Redis: Hydra NodeJS package with service discovery, route discovery, pub/sub messaging, load balancing, health monitoring and more. Uses Redis to store service and route information, and for core pub/sub functionality. https://www.hydramicroservice.com/ Service A calling Service B ROUTING
16. 16 Load Balancing Timeouts Retries / Circuit Breaking Defensive Programming Sending Requests ROUTING
17. 17 Example: Rate Limiting with Redis Can use INCR on a key with EXPIRE Key can be per-user or per-service Quick overview: https://redislabs.com/redis-best-practices/basic-rate-limiting/ Sending Requests ROUTING
18. 18 Service Routing Service Mesh Client-Side Management ROUTING
19. 19 Client-Side Management ROUTING
20. 20 Service Mesh ROUTING
21. 21 Service Routing No libraries to maintain; automatic observability and security. Easier to span large systems. It's another critical piece of infrastructure to manage. Service Mesh Easy to understand, potentially easier to debug. But managing client libraries can be difficult, especially across polyglot systems. Client-Side Management ROUTING
22. 22 ? What’s the Answer? ROUTING
23. 23 03 Management
24. 24 Running many services on many servers? Trying to manage resources more efficiently? Dealing with dynamic cloud instances / auto-scaling? Service Management Are you... MANAGEMENT
25. 25 If not: Don't worry about new tools yet. Use the tools you know. Service Management MANAGEMENT
26. 26 Automate first. Think and plan ahead. Maintenance overhead will grow exponentially. Service Management But beware: MANAGEMENT
27. 27 So Many Services! Suddenly you have lots of services on lots of servers. Orchestration frameworks to the rescue. MANAGEMENT
28. 28 Kubernetes Docker Swarm Nomad Mesos + Marathon Amazon ECS / Fargate Orchestration Lots of options MANAGEMENT
29. 29 Kubernetes Docker Swarm Nomad Mesos + Marathon Amazon ECS / Fargate Orchestration Today's best option MANAGEMENT
30. 30 Primitives like Services, Deployments Pods with Sidecars (service mesh, Redis cache) Namespaces, Node Pools, access control (RBAC) Tools like Helm Orchestration Kubernetes MANAGEMENT
31. 31 04 Observability
32. There are so many questions. 32 OBSERVABILITY
33. 33 What's healthy or not? My service is slow? Slow for some customers...? Requests sometimes timing out? Services spiking the CPU? And so on... So many questions... OBSERVABILITY
34. 34 Ironically, there are fewer questions with a monolith OBSERVABILITY
35. 35 Metrics? OBSERVABILITY Metrics might not help With even 20 or 30 services, how much data can you watch? It's unrealistic to expect actionable data out of the box
36. 36 Alert on container CPU or memory? and host CPU or memory and disk & network usage and request latency (for every service???) and service metrics (for every service???) and... Metrics Which? OBSERVABILITY
37. 37 OBSERVABILITY
38. 38 Ask: Why do you care? OBSERVABILITY If your customers are happy, does a CPU spike matter? Make sure you can answer questions when something critical happens.
39. 39 Tools OBSERVABILITY Logs: ELK, Splunk, Sumo Logic, cloud offerings Metrics: Prometheus, InfluxDB, Datadog, Wavefront Tracing: Zipkin, Opentracing, Jaeger, Stackdriver
40. 40 Tools OBSERVABILITY Think very carefully about your useful signal vs noise
41. 41 Tools OBSERVABILITY Who needs what information when?
42. 42 05 User Experience
43. 43 Your users == developers You want to give them a good experience How do they currently work? What does it mean to break it apart? USER EXPERIENCE
44. 44 Environment Parity Everyone always wants dev, test, and production to be identical. Docker for Mac? Use docker-compose? Kubernetes? What about logging? Metrics? Configuration settings? Service discovery? Do we add Consul or our service mesh? Now what if our system has 50, 100, 200 services? USER EXPERIENCE
45. 45 Environment Parity Nobody has solved this perfectly...? Shared cloud environments? Mock services? Map dependencies to load them automatically? USER EXPERIENCE
46. 46 Tools Integration is hard. This is very different when you move to containers. Building and testing a monolith is conceptually easier. Build/CI Process Deploying containers is completely different from deploying a monolith. It also varies between orchestration frameworks. Deployment Integrating with whatever tools you already have for log management and monitoring is a lot of work. Logging/Monitoring Using IDEs with containers is tricky, especially debugging and integration with orchestration. Development tools USER EXPERIENCE And there's more...
47. USER EXPERIENCE 47 Be prepared for an unexpectedly large amount of change.
48. USER EXPERIENCE 48 Be prepared for an unexpectedly large amount of change. But enabling development teams to be independent is a huge benefit.
49. 49 All of this is not easy. But it will pay off!
50. Thank you! Mason Jones @masonoise