Shipping Manifests, Bill of Lading and Docker - Metadata and Container

Shipping Manifests, Bill of Lading and Docker
Metadata and Container
Gareth Rushgrove
Senior Software Engineer, Puppet Labs

View Slide

@garethr

View Slide

Senior Engineer at Puppet Labs

View Slide

Creator of the Puppet Docker module

View Slide

This Talk
The introduction

View Slide

Shipping containers are cool

View Slide

But nothing without all the
paper work

View Slide

A manifest or ship's manifest
is a document listing the
cargo, passengers, and crew
of a ship, aircraft, or vehicle,
for the use of customs and
other oﬃcials.

View Slide

A bill of lading is a
document issued by a carrier
which details a shipment of
merchandise and gives title
of that shipment to a
speciﬁed party.

View Slide

State of the Software Supply Chain

View Slide

—State of the Software
Supply Chain 2015
A once safe component
may be found to be
vulnerable at any time”
“

View Slide

—State of the Software
Supply Chain 2015
Defective components…
end up in our software
largely unnoticed”
“

View Slide

Lets apply the same
principles to a diﬀerent type
of container

View Slide

Docker Labels
Docker builtin metadata capabilities

View Slide

Added in 1.6
One meta data to rule them all

View Slide

Labels on Docker
Engines

View Slide

Provide information about the host
$ docker daemon \
--label com.example.environment="production" \
--label com.example.storage="ssd"

View Slide

Labels to guide Swarm scheduling
$ docker run -d -P \
-e constraint:storage==ssd --name db mysql

View Slide

Labels on Docker
images

View Slide

LABEL [.][=] ...
Dockerﬁle Label instruction

View Slide

Don’t do this - new layer per label
LABEL vendor=ACME\ Incorporated
LABEL com.example.version.is-beta
LABEL com.example.version="0.0.1-beta"
LABEL com.example.release-date="2015-02-12"

View Slide

Better - only one layer
LABEL vendor="ACME\ Incorporated" \
com.example.is-beta \
com.example.version="0.0.1-beta" \
com.example.release-date="2015-02-12"

View Slide

$ docker inspect 4fa6e0f0c678
...
"Labels": {
"vendor": "ACME Incorporated",
"com.example.is-beta": "",
"com.example.version": "0.0.1-beta",
"com.example.release-date": "2015-02-12"
}
... Access labels via inspect

View Slide

Containers can have
additional labels too

View Slide

Add labels at docker run time
$ docker run \
-d \
--label com.example.group="webservers" \
--label com.example.environment="production" \
busybox \
top

View Slide

Query based on labels
with ﬁlters

View Slide

Filter images by label
$ docker images --filter "label=com.example.is-beta"

View Slide

Filter containers by label
$ docker ps --filter "label=com.example.is-beta"

View Slide

Added in v1.17
Lots of label support in the API

View Slide

But what to store in
Labels?

View Slide

A quick aside
Package Managers

View Slide

I like system packages

View Slide

The power of system
packages lies not in the
ﬁle format but in the
metadata

View Slide

DPKG and RPM

View Slide

Debian New Maintainers’ Guide

View Slide

Fedora packaging guidelines

View Slide

Summary: A CD player app that rocks!
Name: cdplayer
Version: 1.0
Release: 1
Copyright: GPL
Group: Applications/Sound
Source: ftp://ftp.gnomovision.com/pub/cdplayer/cdplayer
URL: http://www.gnomovision.com/cdplayer/cdplayer.html
Distribution: WSS Linux
Vendor: White Socks Software, Inc.
Packager: Santa Claus
%description
It slices! It dices! It's a CD player app that
can't be beat. By using the resonant frequency
of the CD itself, it is able to simulate 20X
Example RPM spec ﬁle

View Slide

Given standard
metadata what
can we do?

View Slide

$ dpkg -L lynx
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/lynx
/usr/share/doc/lynx/copyright
/usr/share/doc/lynx/changelog.gz
/usr/share/doc/lynx/changelog.Debian.gz
List ﬁles from package

View Slide

What installed that ﬁle?
$ rpm -qf /usr/bin/mysqlaccess
MySQL-client-3.23.57-1

View Slide

Find unmet dependencies
$ apt-cache unmet
Package libdataobjects-sqlite3-ruby1.9.1 version
0.10.1.1-1 has an unmet dep:
Depends: libdataobjects-ruby1.9

View Slide

$ rpm -qdf /usr/bin/mysqlaccess
/usr/share/man/man1/mysql.1.gz
/usr/share/man/man1/mysqlaccess.1.gz
/usr/share/man/man1/mysqladmin.1.gz
/usr/share/man/man1/mysqldump.1.gz
/usr/share/man/man1/mysqlshow.1.gz
Find documentation

View Slide

$ apticron
The following packages are currently pending an
upgrade:
xfree86-common 4.3.0.dfsg.1-14sarge3
libice6 4.3.0.dfsg.1-14sarge3
libsm6 4.3.0.dfsg.1-14sarge3
xlibs-data 4.3.0.dfsg.1-14sarge3
libx11-6 4.3.0.dfsg.1-14sarge3
libxext6 4.3.0.dfsg.1-14sarge3
libxpm4 4.3.0.dfsg.1-14sarge3
Find outdated packages

View Slide

Standards
The power of agreement

View Slide

Docker oﬃcial label guidance

View Slide

All (third-party) tools should
preﬁx their keys with the
reverse DNS notation of a
domain controlled by the
author. For example,
com.example.some-label.
1

View Slide

The com.docker.*, io.docker.*
and org.dockerproject.*
namespaces are reserved for
Docker’s internal use. 2

View Slide

Keys should only consist of
lower-cased alphanumeric
characters, dots and dashes
(for example, [a-z0-9-.]). 3

View Slide

Keys should start and end
with an alpha numeric
character.
4

View Slide

Keys may not contain
consecutive dots or dashes.
5

View Slide

Keys without namespace
(dots) are reserved for CLI use.
6

View Slide

How widely adhered to?

View Slide

< 20% from a small sample

View Slide

But some folks care

View Slide

Unify the labels for Docker images

View Slide

Merged

View Slide

The problem with
inconsistent metadata

View Slide

Without complete
metadata we can’t trust
the tools built on top

View Slide

Docker Label Inspector

View Slide

Check against Docker guidelines
$ dli lint
========> Check all labels have namespaces
[WARN] Label 'vendor' should use a namespace based
on reverse DNS notation
========> Check labels don't use reserved namespaces
========> Check labels only use valid characters
========> Check labels start and end with alpanumeric
characters
========> Check labels for double dots and dashes

View Slide

$ dli validate
========> Check labels based on schema in 'schema.json'
[ERROR] u'com.example.is-beta' is a required property
Check against a schema

View Slide

{
"title": "Dockerfile schema",
"type": "object",
"properties": {
"com.example.release-date": {
"type": "string"
},
"com.example.is-beta": {
"type": "string"
},
"com.example.version": {
"description": "Version",
"type": "integer",
"minimum": 0
}
},
"required": ["com.example.is-beta",
"com.example.version"]
}
Deﬁne labels in JSON schema

View Slide

Stand well back
LIVE DEMO

View Slide

Runtime Metadata
A missing piece, and some ideas

View Slide

What temperature is a
refrigerated shipping
containers at?
*Not all shipping containers are refrigerated

View Slide

docker exec as an API

View Slide

Dockerﬁle example
FROM alpine
LABEL net.morethanseven.dockerfile="/Dockerfile" \
net.morethanseven.exec.packages="apk info -vv"
RUN apk add --update bash && rm -rf /var/cache/apk/*
COPY Dockerfile /

View Slide

Discover out API
$ docker inspect -f "{{json .Config.Labels }}" \
garethr/alpine \
| jq
{
"net.morethanseven.dockerfile": "/Dockerfile",
"net.morethanseven.exec.packages": "apk info -vv"
}

View Slide

Read the Dockerﬁle
$ docker run -i -t garethr/alpine cat /Dockerfile
FROM alpine
LABEL net.morethanseven.dockerfile="/Dockerfile" \
net.morethanseven.exec.packages="apk info -vv"
RUN apk add --update bash && rm -rf /var/cache/apk/*
COPY Dockerfile /

View Slide

$ docker run -i -t garethr/alpine apk info -vv
musl-1.1.11-r2 - the musl c library (libc) implementati
busybox-1.23.2-r0 - Size optimized toolbox of many comm
alpine-baselayout-2.3.2-r0 - Alpine base dir structure
openrc-0.15.1-r3 - OpenRC manages the services, startup
alpine-conf-3.2.1-r6 - Alpine configuration management
List installed packages

View Slide

Shipping Manifests, Bill of Lading and Docker - Metadata and Container

Shipping Manifests, Bill of Lading and Docker - Metadata and Container

Gareth Rushgrove

More Decks by Gareth Rushgrove

Other Decks in Technology

Featured

Transcript