Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's inside that container? (redux)

What's inside that container? (redux)

A talk from the Docker meetup in Amsterdam all about real-world usage of containers and the need for better tools to analyse them.


Gareth Rushgrove

September 05, 2017


  1. Gareth Rushgrove What’s inside that container? (redux)

  2. None
  3. @garethr

  4. This talk What is this all about?

  5. - Some research - Some tools - Some live demos

  6. - Some research into container usage - Some tools for

    container analysis - Some live demos of said tools
  7. Setting the scene Some background

  8. Configuration management camp talk

  9. The New Stack coverage

  10. Caveat, this data is from February

  11. Let’s grab downloads from Hub $ curl -s https://registry.hub.docker.com/v2/... ...repositories/library/ubuntu/

    | jq .pull_count
  12. None
  13. What about other popular official images? Node (32,523,647), Java (16,635,049),

    etc. Currently based on Debian too
  14. A quick non-scientific poll

  15. Data from GitHub and Google bigquery

  16. The majority of people using Docker are using images containing

    an entire operating system filesystem
  17. None
  18. Scratch, or other approaches like Nix, appear to occupy a

    small niche
  19. Alpine usage is growing more rapidly than others, but starting

    from a much smaller install base
  20. Why does this matter?

  21. Images vary in size

  22. Count files on images $ find -maxdepth 1 -type d

    | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" -type f | wc -l; done
  23. Count packages in images $ dpkg -l | grep ^ii

    | wc -l $ dnf list installed $ rpm -qa | wc -l $ apk info | wc -l
  24. None
  25. Can you tell me all the versions of OpenSSL you

    have in production right now?
  26. You mentioned tools Anything that can help with these problems?

  27. Option 1: Rebuild everything

  28. Option 2: Something messier

  29. Skopeo from Red Hat

  30. Query registry data without downloading the image $ skopeo inspect

  31. $ skopeo inspect docker://docker.io/fedora { "Name": "docker.io/library/fedora", "Tag": "latest", "Digest":

    "sha256:cfd8f0466748f522406f7ae5908d002af1b1a1", "RepoTags": [ "20", "21", "22", "23", "heisenbug", "latest", "rawhide" ],
  32. But what about running containers? Or the contents of the

  33. Lumogon

  34. Open source and on GitHub

  35. Get package and version information (and more) from Docker containers

    with: $ lumogon scan
  36. $ lumogon scan {"$schema":"http://puppet.com/lumogon/core/draft-01/schema#1 ","generated":"2017-08-07 11:35:16.6517922 +0000 UTC","owner":"default","group":["default"],"client_version": {"BuildVersion":"development","BuildTime":"2017-05-11 08:24:20

    UTC","BuildSHA":"a7f2943697f83ba74514a0169890ecf8ad1cfacb"}, "reportid":"c6a8731e-9681-4758-9151-9c2699769418","container s":{"8c8024760f3e4692e93c6f4f76dc56eaab879e56ace06f876afeccc 5c615ac28":{"$schema":"http://puppet.com/lumogon/containerre port/draft-01/schema#1","generated":"2017-08-07 11:35:16.1308581 +0000 UTC","container_report_id":"2e65f6e7-371d-4bae-9336-85b14b0b 19c0","container_id":"8c8024760f3e4692e93c6f4f76dc56eaab879e 56ace06f876afeccc5c615ac28","container_name":"/p
  37. - Packages (rpm, apk, dpkg) - Host info (eg. linux

    distro) - Labels and other metadata - Extensible with new capabilities
  38. How we have all the data, what can we do

    with it?
  39. List all the debian containers $ lumogon scan | jq

    -r '.containers[] | select(.capabilities.host.payload.platformfamily == "debian") | .container_name'
  40. $ lumogon scan | jq -r '.containers[] | .container_name +

    " " + .capabilities.dpkg.payload.bash + " " + .capabilities.rpm.payload.bash' /fixtures_debian-jessie_1 4.3-11+deb8u1 /fixtures_centos7_1 4.2.46-21.el7_3-x86_64 /fixtures_fedora_1 4.3.43-4.fc25-x86_64 /fixtures_debian-wheezy_1 4.2+dfsg-0.1+deb7u4 /fixtures_ubuntu-xenial_1 4.3-14ubuntu1.1 List versions of bash in all my containers
  41. A hack for finding packages with CVEs

  42. Query our list of package for known CVEs $ lumogon

    scan | findcve lumogon
  43. $ lumogon scan | findcve lumogon ==> Scanning /peaceful_goldberg apt

    has vulnerabilities Currently installed Latest version CVE-2011-3374 is unimportant bash has vulnerabilities Currently installed 4.3-11+deb8u1 Latest version 4.3-11+deb8u1 CVE-2016-9401 is low** TEMP-0841856-B18BAF is unimportant coreutils has vulnerabilities Currently installed 8.23-4 Latest version 8.23-4 CVE-2016-2781 is low** dpkg has vulnerabilities
  44. Manifesto from Aqua Security

  45. Store and query metadata about Docker images, stored alongside the

    image on Hub
  46. Save Lumogon data with manifesto $ ./scan-image.sh <image/name> > /tmp/lumogon.json

    $ manifesto put <image/name> lumogon /tmp/lumogon.json
  47. See what manifesto data is available for our image $

    manifesto list puppet/puppet-agent Metadata types stored for image 'puppet/puppet-agent:latest': lumogon
  48. $ manifesto get puppet/puppet-agent lumogon | head -n 10 {

    "$schema": "http://puppet.com/lumogon/core/draft-01/schema#1", "generated": "2017-07-26 16:25:19.858671545 +0000 UTC", "owner": "default", "group": [ "default" ], "client_version": { "BuildVersion": "20170612091728-", "BuildTime": "2017-06-12 08:17:28 UTC",
  49. Blog post with more examples

  50. Live demo klaxon

  51. Summary If all you remember is...

  52. Container adoption needs to be a gentle slope, not a

    base jump with a parachute marked experimental
  53. The real world is messier than the cloud native ideal

  54. We need to consider the situation where we are running

    unknown artefacts
  55. Small, composable tools point us to higher-level, more comprehensive, solutions

  56. I heartily recommend finding out what’s inside your containers

  57. Any questions? And thanks for listening