Gareth Rushgrove What’s inside that container? (redux)

No content

@garethr

This talk What is this all about?

- Some research - Some tools - Some live demos

- Some research into container usage - Some tools for container analysis - Some live demos of said tools

Setting the scene Some background

Configuration management camp talk

The New Stack coverage

Caveat, this data is from February

Let’s grab downloads from Hub $ curl -s https://registry.hub.docker.com/v2/... ...repositories/library/ubuntu/ | jq .pull_count

No content

What about other popular official images? Node (32,523,647), Java (16,635,049), etc. Currently based on Debian too

A quick non-scientific poll

Data from GitHub and Google bigquery

The majority of people using Docker are using images containing an entire operating system filesystem

No content

Scratch, or other approaches like Nix, appear to occupy a small niche

Alpine usage is growing more rapidly than others, but starting from a much smaller install base

Why does this matter?

Images vary in size

Count files on images $ find -maxdepth 1 -type d | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" -type f | wc -l; done

Count packages in images $ dpkg -l | grep ^ii | wc -l $ dnf list installed $ rpm -qa | wc -l $ apk info | wc -l

No content

Can you tell me all the versions of OpenSSL you have in production right now?

You mentioned tools Anything that can help with these problems?

Option 1: Rebuild everything

Option 2: Something messier

Skopeo from Red Hat

Query registry data without downloading the image $ skopeo inspect docker://docker.io/fedora

$ skopeo inspect docker://docker.io/fedora { "Name": "docker.io/library/fedora", "Tag": "latest", "Digest": "sha256:cfd8f0466748f522406f7ae5908d002af1b1a1", "RepoTags": [ "20", "21", "22", "23", "heisenbug", "latest", "rawhide" ],

But what about running containers? Or the contents of the image?

Lumogon

Open source and on GitHub

Get package and version information (and more) from Docker containers with: $ lumogon scan

$ lumogon scan {"$schema":"http://puppet.com/lumogon/core/draft-01/schema#1 ","generated":"2017-08-07 11:35:16.6517922 +0000 UTC","owner":"default","group":["default"],"client_version": {"BuildVersion":"development","BuildTime":"2017-05-11 08:24:20 UTC","BuildSHA":"a7f2943697f83ba74514a0169890ecf8ad1cfacb"}, "reportid":"c6a8731e-9681-4758-9151-9c2699769418","container s":{"8c8024760f3e4692e93c6f4f76dc56eaab879e56ace06f876afeccc 5c615ac28":{"$schema":"http://puppet.com/lumogon/containerre port/draft-01/schema#1","generated":"2017-08-07 11:35:16.1308581 +0000 UTC","container_report_id":"2e65f6e7-371d-4bae-9336-85b14b0b 19c0","container_id":"8c8024760f3e4692e93c6f4f76dc56eaab879e 56ace06f876afeccc5c615ac28","container_name":"/p

- Packages (rpm, apk, dpkg) - Host info (eg. linux distro) - Labels and other metadata - Extensible with new capabilities

How we have all the data, what can we do with it?

List all the debian containers $ lumogon scan | jq -r '.containers[] | select(.capabilities.host.payload.platformfamily == "debian") | .container_name'

$ lumogon scan | jq -r '.containers[] | .container_name + " " + .capabilities.dpkg.payload.bash + " " + .capabilities.rpm.payload.bash' /fixtures_debian-jessie_1 4.3-11+deb8u1 /fixtures_centos7_1 4.2.46-21.el7_3-x86_64 /fixtures_fedora_1 4.3.43-4.fc25-x86_64 /fixtures_debian-wheezy_1 4.2+dfsg-0.1+deb7u4 /fixtures_ubuntu-xenial_1 4.3-14ubuntu1.1 List versions of bash in all my containers

A hack for finding packages with CVEs

Query our list of package for known CVEs $ lumogon scan | findcve lumogon

$ lumogon scan | findcve lumogon ==> Scanning /peaceful_goldberg apt has vulnerabilities Currently installed 1.0.9.8.4 Latest version 1.0.9.8.4 CVE-2011-3374 is unimportant bash has vulnerabilities Currently installed 4.3-11+deb8u1 Latest version 4.3-11+deb8u1 CVE-2016-9401 is low** TEMP-0841856-B18BAF is unimportant coreutils has vulnerabilities Currently installed 8.23-4 Latest version 8.23-4 CVE-2016-2781 is low** dpkg has vulnerabilities

Manifesto from Aqua Security

Store and query metadata about Docker images, stored alongside the image on Hub

Save Lumogon data with manifesto $ ./scan-image.sh > /tmp/lumogon.json $ manifesto put lumogon /tmp/lumogon.json

See what manifesto data is available for our image $ manifesto list puppet/puppet-agent Metadata types stored for image 'puppet/puppet-agent:latest': lumogon

$ manifesto get puppet/puppet-agent lumogon | head -n 10 { "$schema": "http://puppet.com/lumogon/core/draft-01/schema#1", "generated": "2017-07-26 16:25:19.858671545 +0000 UTC", "owner": "default", "group": [ "default" ], "client_version": { "BuildVersion": "20170612091728-", "BuildTime": "2017-06-12 08:17:28 UTC",

Blog post with more examples

Live demo klaxon

Summary If all you remember is...

Container adoption needs to be a gentle slope, not a base jump with a parachute marked experimental

The real world is messier than the cloud native ideal

We need to consider the situation where we are running unknown artefacts

Small, composable tools point us to higher-level, more comprehensive, solutions

I heartily recommend finding out what’s inside your containers

Any questions? And thanks for listening