Keeping an Eye on your Serverless Containers

Serverless, even for containers, means that you don't need to think about the underlying servers / clusters which takes away a lot of control but also takes away a lot of hassle around managing and patching the servers, i.e worker nodes, on the cluster. But when you go serverless how do you ensure that you still know what is going on in your container ecosystem? How do you react to unknown failure modes when the traditional means of diagnosing an application do not apply anymore? In this talk we'll try to answer these questions by sharing how you can build, run, and most importantly keep an eye on your serverless containers.

Presented together with Prateek Nayak.

Transcript

1. Keeping an eye on your Serverless Containers
2. Who are we? • Lead Engineer @InnablrTech • Co-organiser of Melbourne Kubernetes User Group • Twitter: @pnyak_ • Lead Platform Engineer @ DigIO • AWS APN Ambassador • Co-organiser of AWS User Group Melbourne • ig.nore.me • @ArjenSchwarz Arjen Prateek
3. Billion Dollar Idea!
4. Retirement home for servers https://cuddlekube.com
5. Requirements • Microservices • Containers • NO INFRASTRUCTURE
6. Serverless Containers
14. Problem • How do I docker logs / kubectl logs? • Where do I run prometheus, AlertManager, Grafana etc? • How do I get alerting on CPU, Memory for each host etc? • How do I diagnose network connectivity issues? • WHERE DO I SSH?
15. Observability
17. Logging • Timestamped record of discrete events • Forensics information required to analyse a failure mode
18. Monitoring • Actionable metrics, closely related to user experience • Manage known failure modes of our system
19. Distributed Tracing • Profile and monitor micro services via request tracing • Pinpoint where failures occur and what causes poor performance
20. How o11y? • Instrumentation • Ingestion and aggregation • Visualisation & interrogation
21. Problem • Custom instrumentation for each app • Transporting telemetry data • Dependency on third party libraries
22. Service Mesh
23. AWS App Mesh
24. Virtual Service (Public Site) Virtual Router Virtual Node Virtual Node Virtual Service (List API)
27. Demo Time
28. Some issues... • App Mesh can be a bit finicky • Not much insight into the service mesh • Hard to debug while setting up • Some things didn't work the way we expected • X-Ray needed code instrumentation
29. In conclusion • A lot out of the box • Not that much work to add instrumentation • Integration with other services • We can see what goes wrong
30. Thank you @pnyak_ @ArjenSchwarz