What's inside that container? (redux)

A talk from the Docker meetup in Amsterdam all about real-world usage of containers and the need for better tools to analyse them.

1. Gareth Rushgrove What’s inside that container? (redux)
3. @garethr
4. This talk What is this all about?
5. - Some research - Some tools - Some live demos
6. - Some research into container usage - Some tools for container analysis - Some live demos of said tools
7. Setting the scene Some background
8. Configuration management camp talk
9. The New Stack coverage
10. Caveat, this data is from February
11. Let’s grab downloads from Hub $ curl -s https://registry.hub.docker.com/v2/... ...repositories/library/ubuntu/ | jq .pull_count
13. What about other popular official images? Node (32,523,647), Java (16,635,049), etc. Currently based on Debian too
14. A quick non-scientific poll
15. Data from GitHub and Google bigquery
16. The majority of people using Docker are using images containing an entire operating system filesystem
18. Scratch, or other approaches like Nix, appear to occupy a small niche
19. Alpine usage is growing more rapidly than others, but starting from a much smaller install base
20. Why does this matter?
21. Images vary in size
22. Count files on images $ find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" -type f | wc -l; done
23. Count packages in images $ dpkg -l | grep ^ii | wc -l $ dnf list installed $ rpm -qa | wc -l $ apk info | wc -l
25. Can you tell me all the versions of OpenSSL you have in production right now?
26. You mentioned tools Anything that can help with these problems?
27. Option 1: Rebuild everything
28. Option 2: Something messier
29. Skopeo from Red Hat
30. Query registry data without downloading the image $ skopeo inspect docker://docker.io/fedora
31. $ skopeo inspect docker://docker.io/fedora { "Name": "docker.io/library/fedora", "Tag": "latest", "Digest": "sha256:cfd8f0466748f522406f7ae5908d002af1b1a1", "RepoTags": [ "20", "21", "22", "23", "heisenbug", "latest", "rawhide" ],
32. But what about running containers? Or the contents of the image?
33. Lumogon
34. Open source and on GitHub
35. Get package and version information (and more) from Docker containers with: $ lumogon scan
36. $ lumogon scan {"$schema":"http://puppet.com/lumogon/core/draft-01/schema#1 ","generated":"2017-08-07 11:35:16.6517922 +0000 UTC","owner":"default","group":["default"],"client_version": {"BuildVersion":"development","BuildTime":"2017-05-11 08:24:20 UTC","BuildSHA":"a7f2943697f83ba74514a0169890ecf8ad1cfacb"}, "reportid":"c6a8731e-9681-4758-9151-9c2699769418","container s":{"8c8024760f3e4692e93c6f4f76dc56eaab879e56ace06f876afeccc 5c615ac28":{"$schema":"http://puppet.com/lumogon/containerre port/draft-01/schema#1","generated":"2017-08-07 11:35:16.1308581 +0000 UTC","container_report_id":"2e65f6e7-371d-4bae-9336-85b14b0b 19c0","container_id":"8c8024760f3e4692e93c6f4f76dc56eaab879e 56ace06f876afeccc5c615ac28","container_name":"/p
37. - Packages (rpm, apk, dpkg) - Host info (eg. linux distro) - Labels and other metadata - Extensible with new capabilities
38. How we have all the data, what can we do with it?
39. List all the debian containers $ lumogon scan | jq -r '.containers[] | select(.capabilities.host.payload.platformfamily == "debian") | .container_name'
40. $ lumogon scan | jq -r '.containers[] | .container_name + " " + .capabilities.dpkg.payload.bash + " " + .capabilities.rpm.payload.bash' /fixtures_debian-jessie_1 4.3-11+deb8u1 /fixtures_centos7_1 4.2.46-21.el7_3-x86_64 /fixtures_fedora_1 4.3.43-4.fc25-x86_64 /fixtures_debian-wheezy_1 4.2+dfsg-0.1+deb7u4 /fixtures_ubuntu-xenial_1 4.3-14ubuntu1.1 List versions of bash in all my containers
41. A hack for finding packages with CVEs
42. Query our list of package for known CVEs $ lumogon scan | findcve lumogon
43. $ lumogon scan | findcve lumogon ==> Scanning /peaceful_goldberg apt has vulnerabilities Currently installed 1.0.9.8.4 Latest version 1.0.9.8.4 CVE-2011-3374 is unimportant bash has vulnerabilities Currently installed 4.3-11+deb8u1 Latest version 4.3-11+deb8u1 CVE-2016-9401 is low** TEMP-0841856-B18BAF is unimportant coreutils has vulnerabilities Currently installed 8.23-4 Latest version 8.23-4 CVE-2016-2781 is low** dpkg has vulnerabilities
44. Manifesto from Aqua Security
45. Store and query metadata about Docker images, stored alongside the image on Hub
46. Save Lumogon data with manifesto $ ./scan-image.sh <image/name> > /tmp/lumogon.json $ manifesto put <image/name> lumogon /tmp/lumogon.json
47. See what manifesto data is available for our image $ manifesto list puppet/puppet-agent Metadata types stored for image 'puppet/puppet-agent:latest': lumogon
48. $ manifesto get puppet/puppet-agent lumogon | head -n 10 { "$schema": "http://puppet.com/lumogon/core/draft-01/schema#1", "generated": "2017-07-26 16:25:19.858671545 +0000 UTC", "owner": "default", "group": [ "default" ], "client_version": { "BuildVersion": "20170612091728-", "BuildTime": "2017-06-12 08:17:28 UTC",
49. Blog post with more examples
50. Live demo klaxon
51. Summary If all you remember is...
52. Container adoption needs to be a gentle slope, not a base jump with a parachute marked experimental
53. The real world is messier than the cloud native ideal
54. We need to consider the situation where we are running unknown artefacts
55. Small, composable tools point us to higher-level, more comprehensive, solutions
56. I heartily recommend finding out what’s inside your containers
57. Any questions? And thanks for listening