Docker Containers Monitoring by Leo Unbekandt

Transcript

1. Docker containers monitoring Berlin 2015
2. Léo Unbekandt @Soulou CTO at Scalingo
3. Deploy, Build, Scale, Repeat
4. Our customers want metrics to help them in their decision making
5. DOCKER AS A TOOL ?
6. DOCKER AS A TOOL • Deployment … OK
7. DOCKER AS A TOOL • Deployment … OK • Execution … OK
8. DOCKER AS A TOOL • Deployment … OK • Execution … OK • Monitoring … ?
9. - Memory usage - CPU percentage - Disk IO - Network bandwidth What metrics?
10. Docker Remote API GET /containers/:id/stats Stats API
11. Docker Remote API generates load
12. Docker Remote API is not configurable
13. Docker →is not enough So
14. Linux API Control groups Namespaces
15. Control Groups CPU, Memory, IO Namespaces Network
16. Memory usage? Memory cgroup /sys/fs/cgroup/memory/docker/:id
17. Memory cgroup Files memory.usage_in_bytes memory.limit_in_bytes memory.memsw.usage_in_bytes memory.memsw.limit_in_bytes Bytes Are Easy
18. CPU usage? cpuacct cgroup /sys/fs/cgroup/cpuacct/docker/:id
19. Cpuaccnt cgroup Files cpuacct.usage cpuacct.usage_percpu → number of nanoseconds allocated to the container since its start
20. Nanoseconds? Let's calculate!
21. How to get a CPU percentage? /proc/stat User nice system idle iowait irq softirq 7277 51882 500481 6104557 72847 217 168631 Sum of values = CPU usage since system boot (in 1/100th second)
22. Delta of container usage = 0-100% ———————————————————— Delta of system usage How to get a CPU percentage?
23. What are the IO? blkio cgroup /sys/fs/cgroup/blkio/docker/:id → Bytes read/write by device
24. blkio cgroup File blkio.io_service_bytes
25. Namespaces Networking
26. Not a control group unfortunately… Network namespace
27. Network namespace Monitoring agent in the namespace setns() Linux system call
28. Network namespace Don't worry github.com/Scalingo/go-netns
29. Interface Received bytes packets errs … eth0 12520750 102418 0 lo 7796 560 0 Interface Transmit bytes packets errs … eth0 641338 165423 0 lo 97796 250 0 Netstat in namespace
30. Enter “Acadock” Acadock monitoring Linux APIs Docker HTTP API Acadock HTTP API Data visualisation via websockets
31. Got what we want
32. Thank you Léo Unbekandt - @Soulou github.com/Scalingo/acadock-monitoring github.com/Scalingo/acadock github.com/Scalingo/go-netns github.com/Scalingo/go-netstat