Slide 1
Slide 1 text
VSHN – The DevOps Company
Adrian Kosmaczewski
Sharing CLI Tools with
Containers
1
Slide 2
Slide 2 text
VSHN – The DevOps Company
Pronounced ˈvɪʒn – like "vision"
Founded 2014 in Zürich
Switzerland’s leading DevOps, Docker & Kubernetes
expert with 24/7 support
ISO 27001 certi ed & ISAE 3402 Report Type 1 veri ed
First Swiss Kubernetes Certi ed Service Provider
/ /
vshn.ch @vshn_ch vshn.tv
2
Slide 3
Slide 3 text
VSHN – The DevOps Company 3
Slide 4
Slide 4 text
VSHN – The DevOps Company 4
Slide 5
Slide 5 text
VSHN – The DevOps Company
1. "Lightweight virtual machines"
2. Docker Compose
3. Kubernetes
Containers at Work
5
Slide 6
Slide 6 text
VSHN – The DevOps Company
Containers as a tool distribution mechanism
Think Di erent
6
Slide 7
Slide 7 text
VSHN – The DevOps Company
1. The Challenge of Internal Tools
2. Encapsulating and Sharing Tools
3. Reusing Tools in CI/CD Pipelines
4. More Container Tool Ideas
5. "Gotchas"
Agenda
7
Slide 8
Slide 8 text
VSHN – The DevOps Company
1. The Challenge of Internal Tools
8
Slide 9
Slide 9 text
VSHN – The DevOps Company 9
Slide 10
Slide 10 text
VSHN – The DevOps Company
retool.com/blog/state-of-internal-tools-2020
10
Slide 11
Slide 11 text
VSHN – The DevOps Company
retool.com/blog/state-of-internal-tools-2020
11
Slide 12
Slide 12 text
VSHN – The DevOps Company
Lots of
Each VSHNeer
Linux, Mac, Windows
Choose the "best" programming language for the job
Go, Java, Python, JavaScript, shell script
VSHN
internal tools
can use their preferred OS
12
Slide 13
Slide 13 text
VSHN – The DevOps Company
Go, Rust, C, C++
Great for x-platform
CLI tools!
Complex to build
Ruby, Python, JavaScript
Agile: fast prototyping,
great libraries
Big runtimes to install
and libraries to download
Programming Languages
13
Slide 14
Slide 14 text
VSHN – The DevOps Company
2. Encapsulating and Sharing Tools
14
Slide 15
Slide 15 text
VSHN – The DevOps Company
TypeScript application
npm download internet
Example: "Greeter"
gitlab.com/akosma/greeter
15
Slide 16
Slide 16 text
VSHN – The DevOps Company
Greeter in Action
$ podman run --rm quay.io/akosma/greeter:1.0 "WeAreDevelopers"
_______________________
< Hello WeAreDevelopers >
-----------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
16
Slide 17
Slide 17 text
VSHN – The DevOps Company
Options:
-V, --version output the version number
-c, --character greeter (choices: "C3PO", "R2-D2", "ackbar", "ar
"bunny", "cat", "default", "doge", "mona-lisa",
-a, --action thing to do (choices: "talk", "think", default:
-h, --help display help for command
asciinema $ node out/src/index.js WeAreDevelopers!
________________________
< Hello WeAreDevelopers! >
------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
asciinema $
00:00
gitlab.com/akosma/greeter
asciinema.org/a/lTo0VDVmMJyD6eF9tu56XLbeQ
17
Slide 18
Slide 18 text
VSHN – The DevOps Company
Multi-Step Builds
# Step 1: Builder image
FROM node:14.15.1-alpine3.12 AS builder
COPY [".eslintrc.js", ".eslintignore", "tsconfig.json", "gulpfile.js", "package.json",
COPY src /command/src
COPY spec /command/spec
WORKDIR /command
RUN npm install
RUN npm test
RUN npm run release
# ..
# Step 2: Runtime image
FROM alpine:3.12
RUN apk add --no-cache libstdc++
COPY --from=builder /command/greeter.bin /usr/local/bin/greeter
ENTRYPOINT [ "greeter" ]
gitlab.com/akosma/greeter/-/blob/master/Docker le
18
Slide 19
Slide 19 text
VSHN – The DevOps Company
asciinema $ podman build -t quay.io/akosma/greeter:1.0 .
STEP 1: FROM node:14.15.1-alpine3.12 AS builder
Completed short name "node" with unqualified-search registries (origin: /etc/co
s.conf)
Getting image source signatures
Copying blob e75f007ed29c skipped: already exists
Copying blob 05e7bc50f07f skipped: already exists
Copying blob 94058c629e48 skipped: already exists
Copying blob 6bd5f70f12c5 [--------------------------------------] 0.0b / 0.0b
Copying config bc9a7579ff done
Writing manifest to image destination
Storing signatures
00:00
gitlab.com/akosma/greeter
asciinema.org/a/qS0X1vBrZtMjHZhI8q55p4JfD
19
Slide 20
Slide 20 text
VSHN – The DevOps Company
1. podman build -t quay.io/akosma/greeter:1.0 .
2. podman images
3. podman login quay.io
4. podman push quay.io/akosma/greeter:1.0
5.
Recap'
quay.io/repository/akosma/greeter
20
Slide 21
Slide 21 text
VSHN – The DevOps Company 21
Slide 22
Slide 22 text
VSHN – The DevOps Company
0:00
22
Slide 23
Slide 23 text
VSHN – The DevOps Company 23
Slide 24
Slide 24 text
VSHN – The DevOps Company
3. Reusing Tools in CI/CD Pipelines
24
Slide 25
Slide 25 text
VSHN – The DevOps Company
.gitlab-ci.yml
build:
image: docker:19.03.12
stage: build
services:
- docker:19.03.12-dind
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker build -t $CI_REGISTRY/akosma/greeter:latest .
- docker push $CI_REGISTRY/akosma/greeter:latest
gitlab.com/akosma/greeter/-/blob/master/.gitlab-ci.yml
25
Slide 26
Slide 26 text
VSHN – The DevOps Company
Pipeline Run
26
Slide 27
Slide 27 text
VSHN – The DevOps Company
Embedded Container Registry
27
Slide 28
Slide 28 text
VSHN – The DevOps Company
$ docker login registry.gitlab.com
...
$ podman login registry.gitlab.com
...
28
Slide 29
Slide 29 text
VSHN – The DevOps Company
Reuse
gitlab.com/akosma/fortune
29
Slide 30
Slide 30 text
VSHN – The DevOps Company
Docker le
# Step 1: print a greeting
FROM registry.gitlab.com/akosma/greeter:latest
RUN /usr/local/bin/greeter Dockerfile --character snoopy --action think
# Step 2: build runtime image
FROM python:3.7-alpine
RUN apk add fortune
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py /usr/src/app
COPY templates /usr/src/app/templates/
USER 1001
EXPOSE 9090
CMD [ "python", "app.py" ]
gitlab.com/akosma/fortune/-/blob/master/Docker le
30
Slide 31
Slide 31 text
VSHN – The DevOps Company
.gitlab-ci.yml
stages:
- build
.base:
stage: build
image: docker:stable
services:
- docker:dind
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
docker-build:
extends: .base
script:
- docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" .
- docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
only:
- master
gitlab.com/akosma/fortune/-/blob/master/.gitlab-ci.yml
31
Slide 32
Slide 32 text
VSHN – The DevOps Company 32
Slide 33
Slide 33 text
VSHN – The DevOps Company
4. More Container Tool Ideas
33
Slide 34
Slide 34 text
VSHN – The DevOps Company
Linting
Black-box testing
API documentation generation
Code Tasks
34
Slide 35
Slide 35 text
VSHN – The DevOps Company
Find
using vale
Create or
Generate for docs
for documentation
Documentation Tasks
dead links in HTML
Check writing style
Spell checking
PDF EPUB
search engine index
Live preview
35
Slide 36
Slide 36 text
VSHN – The DevOps Company
5. "Gotchas"
36
Slide 37
Slide 37 text
VSHN – The DevOps Company
1. Creating CLI tools
2. Building containers
3. Programming language-speci c tips
Types of Gotchas
37
Slide 38
Slide 38 text
VSHN – The DevOps Company
Follow the
Create man page for your tools using
Pass secrets to tools as environment variables
Document, document, document
Use Make les to standardize projects
5.1 Creating CLI Tools Gotchas
Command Line Interface Guidelines
Asciidoctor
38
Slide 39
Slide 39 text
VSHN – The DevOps Company
pages := $(shell find . -type f -name '*.adoc')
out_dir := ./docs
docker_cmd ?= podman
docker_opts ?= --rm --tty # add --user "$$(id -u)" if using docker!
antora_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/antora antora/anto
vale_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/docsrc/modules/ROOT/pa
hunspell_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/spell vshn/hunspe
preview_cmd ?= $(docker_cmd) run --rm --publish 35729:35729 --publish 2020:2020 --volu
htmltest_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/$(out_dir):/test w
.PHONY: all
all: html
.PHONY: clean
clean:
rm -rf $(out_dir)
.PHONY: vale
vale:
$(vale_cmd)
.PHONY: spell
spell: html
$(hunspell cmd)
39
Slide 40
Slide 40 text
VSHN – The DevOps Company
Beware of base images using musl instead of glibc
…like Alpine!
Free some disk space regularly with podman image prune
ENTRYPOINT > CMD
COPY > ADD
Use other container registries
Inspect your images using
5.2 Gotchas when Building
Containers
dive
40
Slide 41
Slide 41 text
VSHN – The DevOps Company
Self-hosted: , , , and
SaaS: , , ,
( ), and ( )
Embedded: and
Container Registries
kraken Harbor Docker distribution
Sonatype Nexus
Docker Pro or Team plans Quay AWS ECR GitHub
Packages ghcr.io Google Container Registry gcr.io
OpenShift GitLab
41
Slide 42
Slide 42 text
VSHN – The DevOps Company
1. Python
2. Go
3. JavaScript
4. Shell scripts
5.3 Language-speci c tips
42
Slide 43
Slide 43 text
VSHN – The DevOps Company
Use virtual environments!
5.3.1 Python Tips
# During development...
$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install PyYAML
$ pip install PyGithub
$ pip freeze > requirements.txt
# In the Dockerfile...
RUN pip install --no-cache-dir -r requirements.txt
43
Slide 44
Slide 44 text
VSHN – The DevOps Company
Use the base image!
5.3.2 Go Tips
distroless
44
Slide 45
Slide 45 text
VSHN – The DevOps Company
Use TypeScript!
npm install --prod
Use to create standalone binaries
Use the base image for minimalistic, small
nal container images ( ).
5.3.3 JavaScript Tips
pkg
scratch-node
GitHub
45
Slide 46
Slide 46 text
VSHN – The DevOps Company
Use
And if you use Visual Studio Code, install the
Beware of base images that do not contain /bin/bash and
only have /bin/sh (like with )
Either make sure your shell scripts work with /bin/sh,
or install bash in your target image
5.3.4 Shell Scripting Tips
ShellCheck
ShellCheck VS extension
Alpine Busybox
46
Slide 47
Slide 47 text
VSHN – The DevOps Company
Wrap command line tools in containers
Use a public or private registries to share your tools
Document them properly: README + man page + Antora
website + …
Beware of common issues
Summary
47
Slide 48
Slide 48 text
VSHN – The DevOps Company
Christian Cremer
Tobias Brunner
Aarno Aukia
Acknowledgements
ccremer on GitHub
@tobruzh on Twitter
@aarnoaukia on Twitter
48
Slide 49
Slide 49 text
VSHN – The DevOps Company
Adrian Kosmaczewski – –
VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 –
– –
Thanks!
@akosma [email protected]
[email protected]
vshn.ch @vshn_ch vshn.tv
49