Slide 1
Slide 1 text
Docker containers
monitoring
Berlin 2015
Slide 2
Slide 2 text
Léo
Unbekandt
@Soulou
CTO at Scalingo
Slide 3
Slide 3 text
Deploy, Build, Scale, Repeat
Slide 4
Slide 4 text
Our customers want metrics to
help them in their decision making
Slide 5
Slide 5 text
DOCKER AS A
TOOL
?
Slide 6
Slide 6 text
DOCKER AS A
TOOL
●
Deployment … OK
Slide 7
Slide 7 text
DOCKER AS A
TOOL
●
Deployment … OK
●
Execution … OK
Slide 8
Slide 8 text
DOCKER AS A
TOOL
●
Deployment … OK
●
Execution … OK
●
Monitoring … ?
Slide 9
Slide 9 text
- Memory usage
- CPU percentage
- Disk IO
- Network bandwidth
What metrics?
Slide 10
Slide 10 text
Docker
Remote API
GET /containers/:id/stats
Stats API
Slide 11
Slide 11 text
Docker
Remote API
generates load
Slide 12
Slide 12 text
Docker
Remote API
is not configurable
Slide 13
Slide 13 text
Docker
→is not enough
So
Slide 14
Slide 14 text
Linux API
Control groups
Namespaces
Slide 15
Slide 15 text
Control Groups
CPU, Memory, IO
Namespaces
Network
Slide 16
Slide 16 text
Memory usage?
Memory cgroup
/sys/fs/cgroup/memory/docker/:id
Slide 17
Slide 17 text
Memory cgroup
Files
memory.usage_in_bytes
memory.limit_in_bytes
memory.memsw.usage_in_bytes
memory.memsw.limit_in_bytes
Bytes Are Easy
Slide 18
Slide 18 text
CPU usage?
cpuacct cgroup
/sys/fs/cgroup/cpuacct/docker/:id
Slide 19
Slide 19 text
Cpuaccnt cgroup
Files
cpuacct.usage
cpuacct.usage_percpu
→ number of nanoseconds allocated to the
container since its start
Slide 20
Slide 20 text
Nanoseconds?
Let's calculate!
Slide 21
Slide 21 text
How to get a CPU percentage?
/proc/stat
User nice system idle iowait irq softirq
7277 51882 500481 6104557 72847 217 168631
Sum of values = CPU usage since
system boot (in 1/100th second)
Slide 22
Slide 22 text
Delta of container usage
= 0-100%
————————————————————
Delta of system usage
How to get a CPU percentage?
Slide 23
Slide 23 text
What are the IO?
blkio cgroup
/sys/fs/cgroup/blkio/docker/:id
→ Bytes read/write by device
Slide 24
Slide 24 text
blkio cgroup
File
blkio.io_service_bytes
Slide 25
Slide 25 text
Namespaces
Networking
Slide 26
Slide 26 text
Not a control group
unfortunately…
Network namespace
Slide 27
Slide 27 text
Network namespace
Monitoring agent in the
namespace
setns() Linux system call
Slide 28
Slide 28 text
Network namespace
Don't worry
github.com/Scalingo/go-netns
Slide 29
Slide 29 text
Interface Received bytes packets errs …
eth0 12520750 102418 0
lo 7796 560 0
Interface Transmit bytes packets errs …
eth0 641338 165423 0
lo 97796 250 0
Netstat in namespace
Slide 30
Slide 30 text
Enter “Acadock”
Acadock
monitoring
Linux APIs
Docker
HTTP API
Acadock
HTTP API
Data visualisation
via websockets
Slide 31
Slide 31 text
Got what we want
Slide 32
Slide 32 text
Thank you
Léo Unbekandt - @Soulou
github.com/Scalingo/acadock-monitoring
github.com/Scalingo/acadock
github.com/Scalingo/go-netns
github.com/Scalingo/go-netstat