Distributed Tracing in Serverless Systems - KubeCon 2018.pdf

Serverless and FaaS naturally fit microservices architectures. Observability of such systems is very complicated, since each microservice is separated and working asynchronously from the others. Distributed tracing is a key approach to understand such systems. In serverless, there are new challenges and opportunities which make distributed tracing a very interesting and useful technique for high observability.

I will go quickly over the history of tracing and the popular tools. I will focus on the key differences between observability using distributed tracing in generic microservices environments compared to serverless. Examples from the popular cloud vendors will be shown, including full visualization of asynchronous transactions in a highly distributed serverless system, and detecting business flows across multiple asynchronous communication resources (e.g. SNS, Kinesis, and more).

Transcript

1. KubeCon + CloudNativeCon Seattle Distributed Tracing in Serverless Systems Nitzan Shapira, Epsagon
2. Nitzan Shapira (@nitzanshapira) Software engineer > 12 years Co-Founder, CEO at Epsagon Tel Aviv 2 > whoami
3. What is serverless? How is it different? What is observability for serverless? How can distributed tracing help? How will it help my job? 3 Things to discuss
4. 4 [Compute-as-a-Service] FaaS: Function-as-a-Service CaaS: Container-as-a-Service + Managed services (APIs) = Don’t manage infrastructure Focus on business logic What is serverless?
5. 5 Why serverless? Pay-per-use: reduces cloud compute cost by 90% Out-of-the-box auto-scaling DevOps à LowOps ++Developer velocity Focus on business logic – iterate faster Server Utilization
6. 6 The limitations of FaaS Limited memory Limited running time Cold starts Stateless + concurrency limit + some others…
7. 7 The properties of serverless applications Serverless is micro-services Serverless applications are - Highly distributed - Highly event-driven Utilizing managed services via APIs is key
8. A real example – HSBC 8 Source: re:Invent 2018
9. 9 The challenge in serverless SIMPLE COMPLEX Yan Cui
10. 10 What the community thinks 2018 Serverless Community Survey, serverless.com, July 2018 2017 results
11. 11 Observability – why do we need it? Track system health Troubleshoot and fix Optimize performance and cost
12. 12 Observability in serverless Let’s go one by one
13. 13 Track system health System == Functions ?
14. 14 Functions are important - Errors - Timeout - Out-of-memory - Cold start
15. 15 Track system health System > Functions ! Serverless != Functions
16. 16 Track system health System > Functions ! Functions APIs Transactions
17. 17 Troubleshoot and fix Functions are not enough Need: track asynchronous events e
18. 18 Transactions
19. 19 Tracing asynchronous invocations
20. 20 Tracing asynchronous invocations
21. 21 Tracing asynchronous invocations
22. 22 Distributed tracing …a trace tells the story of a transaction or workflow as it propagates through a (potentially distributed) system. Distributed tracing is a method used to profile and monitor applications.
23. 23 Distributed tracing Jaeger
24. 24 Implementing distributed tracing Manual tracing/instrumentation Before/after calls At the end of each micro-service High maintenance High potential of errors
25. 25 Serverless apps are very distributed Complex systems have thousands of functions What about the developer velocity?
26. 26 Can it be done differently in serverless?
27. 27 Automation can help to keep up with the development speed of serverless
28. 28 Example
29. 29 Example
30. 30 Monitoring serverless Limited memory Limited running time Cold starts Stateless
31. 31 Time is $$$
32. 32 Where do we spend the most time? Our own code API calls
33. 33 Serverless cost crisis A real-life example $$$$$$$$$$$$
34. 34 Scanning functions Scanning CloudWatch using AWS Lambda Every 5 minutes, save to RDS A new Lambda is spawned for every customer’s function Poll Spawn (async) CloudWatch
35. 35 As time flies… CloudWatch became highly throttled Requests took too much time 5K concurrent Lambdas, for 5 minutes, timing out , every 5 minutes !!!!
36. 36 Why you should care about external APIs 702ms e
37. 37 Track service health
38. 38 Business flows Subscribe Transfer Payment
39. 39 What should I optimize first?
40. 40 Remember… Serverless + Distributed Tracing = Perfect marriage (but only if you automate)
41. nitzan@epsagon.com @nitzanshapira www.epsagon.com Thank you!