What's inside that container? (redux)

Transcript

1. Gareth Rushgrove What’s inside that container? (redux)

2. None

3. @garethr

4. This talk What is this all about?

5. - Some research - Some tools - Some live demos

6. - Some research into container usage - Some tools for

   container analysis - Some live demos of said tools

7. Setting the scene Some background

8. Configuration management camp talk

9. The New Stack coverage

10. Caveat, this data is from February

11. Let’s grab downloads from Hub $ curl -s https://registry.hub.docker.com/v2/... ...repositories/library/ubuntu/

    | jq .pull_count
12. None

13. What about other popular official images? Node (32,523,647), Java (16,635,049),

    etc. Currently based on Debian too

14. A quick non-scientific poll

15. Data from GitHub and Google bigquery

16. The majority of people using Docker are using images containing

    an entire operating system filesystem
17. None

18. Scratch, or other approaches like Nix, appear to occupy a

    small niche

19. Alpine usage is growing more rapidly than others, but starting

    from a much smaller install base

20. Why does this matter?

21. Images vary in size

22. Count files on images $ find -maxdepth 1 -type d

    | while read -r dir; do printf "%s:\t" "$dir"; find "$dir" -type f | wc -l; done

23. Count packages in images $ dpkg -l | grep ^ii

    | wc -l $ dnf list installed $ rpm -qa | wc -l $ apk info | wc -l
24. None

25. Can you tell me all the versions of OpenSSL you

    have in production right now?

26. You mentioned tools Anything that can help with these problems?

27. Option 1: Rebuild everything

28. Option 2: Something messier

29. Skopeo from Red Hat

30. Query registry data without downloading the image $ skopeo inspect

    docker://docker.io/fedora

31. $ skopeo inspect docker://docker.io/fedora { "Name": "docker.io/library/fedora", "Tag": "latest", "Digest":

    "sha256:cfd8f0466748f522406f7ae5908d002af1b1a1", "RepoTags": [ "20", "21", "22", "23", "heisenbug", "latest", "rawhide" ],

32. But what about running containers? Or the contents of the

    image?

33. Lumogon

34. Open source and on GitHub

35. Get package and version information (and more) from Docker containers

    with: $ lumogon scan

36. $ lumogon scan {"$schema":"http://puppet.com/lumogon/core/draft-01/schema#1 ","generated":"2017-08-07 11:35:16.6517922 +0000 UTC","owner":"default","group":["default"],"client_version": {"BuildVersion":"development","BuildTime":"2017-05-11 08:24:20

    UTC","BuildSHA":"a7f2943697f83ba74514a0169890ecf8ad1cfacb"}, "reportid":"c6a8731e-9681-4758-9151-9c2699769418","container s":{"8c8024760f3e4692e93c6f4f76dc56eaab879e56ace06f876afeccc 5c615ac28":{"$schema":"http://puppet.com/lumogon/containerre port/draft-01/schema#1","generated":"2017-08-07 11:35:16.1308581 +0000 UTC","container_report_id":"2e65f6e7-371d-4bae-9336-85b14b0b 19c0","container_id":"8c8024760f3e4692e93c6f4f76dc56eaab879e 56ace06f876afeccc5c615ac28","container_name":"/p

37. - Packages (rpm, apk, dpkg) - Host info (eg. linux

    distro) - Labels and other metadata - Extensible with new capabilities

38. How we have all the data, what can we do

    with it?

39. List all the debian containers $ lumogon scan | jq

    -r '.containers[] | select(.capabilities.host.payload.platformfamily == "debian") | .container_name'

40. $ lumogon scan | jq -r '.containers[] | .container_name +

    " " + .capabilities.dpkg.payload.bash + " " + .capabilities.rpm.payload.bash' /fixtures_debian-jessie_1 4.3-11+deb8u1 /fixtures_centos7_1 4.2.46-21.el7_3-x86_64 /fixtures_fedora_1 4.3.43-4.fc25-x86_64 /fixtures_debian-wheezy_1 4.2+dfsg-0.1+deb7u4 /fixtures_ubuntu-xenial_1 4.3-14ubuntu1.1 List versions of bash in all my containers

41. A hack for finding packages with CVEs

42. Query our list of package for known CVEs $ lumogon

    scan | findcve lumogon

43. $ lumogon scan | findcve lumogon ==> Scanning /peaceful_goldberg apt

    has vulnerabilities Currently installed 1.0.9.8.4 Latest version 1.0.9.8.4 CVE-2011-3374 is unimportant bash has vulnerabilities Currently installed 4.3-11+deb8u1 Latest version 4.3-11+deb8u1 CVE-2016-9401 is low** TEMP-0841856-B18BAF is unimportant coreutils has vulnerabilities Currently installed 8.23-4 Latest version 8.23-4 CVE-2016-2781 is low** dpkg has vulnerabilities

44. Manifesto from Aqua Security

45. Store and query metadata about Docker images, stored alongside the

    image on Hub

46. Save Lumogon data with manifesto $ ./scan-image.sh <image/name> > /tmp/lumogon.json

    $ manifesto put <image/name> lumogon /tmp/lumogon.json

47. See what manifesto data is available for our image $

    manifesto list puppet/puppet-agent Metadata types stored for image 'puppet/puppet-agent:latest': lumogon

48. $ manifesto get puppet/puppet-agent lumogon | head -n 10 {

    "$schema": "http://puppet.com/lumogon/core/draft-01/schema#1", "generated": "2017-07-26 16:25:19.858671545 +0000 UTC", "owner": "default", "group": [ "default" ], "client_version": { "BuildVersion": "20170612091728-", "BuildTime": "2017-06-12 08:17:28 UTC",

49. Blog post with more examples

50. Live demo klaxon

51. Summary If all you remember is...

52. Container adoption needs to be a gentle slope, not a

    base jump with a parachute marked experimental

53. The real world is messier than the cloud native ideal

54. We need to consider the situation where we are running

    unknown artefacts

55. Small, composable tools point us to higher-level, more comprehensive, solutions

56. I heartily recommend finding out what’s inside your containers

57. Any questions? And thanks for listening