Reusing Apps between Teams and Environments through Containers

Presentation shown in the WeAreDevelopers Container Day, Wednesday, February 3rd, 2021. Video: https://youtu.be/dudny94KWpo?t=3629

Transcript

1. VSHN – The DevOps Company
   Adrian Kosmaczewski
   Sharing CLI Tools with
   Containers
   1
   

   View Slide

2. VSHN – The DevOps Company
   Pronounced ˈvɪʒn – like "vision"
   Founded 2014 in Zürich
   Switzerland’s leading DevOps, Docker & Kubernetes
   expert with 24/7 support
   ISO 27001 certi ed & ISAE 3402 Report Type 1 veri ed
   First Swiss Kubernetes Certi ed Service Provider
   / /
   vshn.ch @vshn_ch vshn.tv
   2
   

   View Slide

3. VSHN – The DevOps Company 3
   

   View Slide

4. VSHN – The DevOps Company 4
   

   View Slide

5. VSHN – The DevOps Company
   1. "Lightweight virtual machines"
   2. Docker Compose
   3. Kubernetes
   Containers at Work
   5
   

   View Slide

6. VSHN – The DevOps Company
   Containers as a tool distribution mechanism
   Think Di erent
   6
   

   View Slide

7. VSHN – The DevOps Company
   1. The Challenge of Internal Tools
   2. Encapsulating and Sharing Tools
   3. Reusing Tools in CI/CD Pipelines
   4. More Container Tool Ideas
   5. "Gotchas"
   Agenda
   7
   

   View Slide

8. VSHN – The DevOps Company
   
   1. The Challenge of Internal Tools
   8
   

   View Slide

9. VSHN – The DevOps Company 9
   

   View Slide

10. VSHN – The DevOps Company
    retool.com/blog/state-of-internal-tools-2020
    10
    

    View Slide

11. VSHN – The DevOps Company
    retool.com/blog/state-of-internal-tools-2020
    11
    

    View Slide

12. VSHN – The DevOps Company
    Lots of
    Each VSHNeer
    Linux, Mac, Windows
    Choose the "best" programming language for the job
    Go, Java, Python, JavaScript, shell script
    VSHN
    internal tools
    can use their preferred OS
    12
    

    View Slide

13. VSHN – The DevOps Company
    Go, Rust, C, C++
     Great for x-platform
    CLI tools!
     Complex to build
    Ruby, Python, JavaScript
     Agile: fast prototyping,
    great libraries
     Big runtimes to install
    and libraries to download
    Programming Languages
    13
    

    View Slide

14. VSHN – The DevOps Company
    2. Encapsulating and Sharing Tools
    14
    

    View Slide

15. VSHN – The DevOps Company
    TypeScript application
    npm download internet
    Example: "Greeter"
    gitlab.com/akosma/greeter
    15
    

    View Slide

16. VSHN – The DevOps Company
    Greeter in Action
    $ podman run --rm quay.io/akosma/greeter:1.0 "WeAreDevelopers"
    _______________________
    < Hello WeAreDevelopers >
    -----------------------
    \ ^__^
    \ (oo)\_______
    (__)\ )\/\
    ||----w |
    || ||
    16
    

    View Slide

17. VSHN – The DevOps Company
    Options:
    -V, --version output the version number
    -c, --character greeter (choices: "C3PO", "R2-D2", "ackbar", "ar
    "bunny", "cat", "default", "doge", "mona-lisa",
    -a, --action thing to do (choices: "talk", "think", default:
    -h, --help display help for command
    asciinema $ node out/src/index.js WeAreDevelopers!
    ________________________
    < Hello WeAreDevelopers! >
    ------------------------
    \ ^__^
    \ (oo)\_______
    (__)\ )\/\
    ||----w |
    || ||
    asciinema $
    00:00
    gitlab.com/akosma/greeter
    asciinema.org/a/lTo0VDVmMJyD6eF9tu56XLbeQ
    17
    

    View Slide

18. VSHN – The DevOps Company
    Multi-Step Builds
    # Step 1: Builder image
    FROM node:14.15.1-alpine3.12 AS builder
    COPY [".eslintrc.js", ".eslintignore", "tsconfig.json", "gulpfile.js", "package.json",
    COPY src /command/src
    COPY spec /command/spec
    WORKDIR /command
    RUN npm install
    RUN npm test
    RUN npm run release
    # ..
    # Step 2: Runtime image
    FROM alpine:3.12
    RUN apk add --no-cache libstdc++
    COPY --from=builder /command/greeter.bin /usr/local/bin/greeter
    ENTRYPOINT [ "greeter" ]
    gitlab.com/akosma/greeter/-/blob/master/Docker le
    18
    

    View Slide

19. VSHN – The DevOps Company
    asciinema $ podman build -t quay.io/akosma/greeter:1.0 .
    STEP 1: FROM node:14.15.1-alpine3.12 AS builder
    Completed short name "node" with unqualified-search registries (origin: /etc/co
    s.conf)
    Getting image source signatures
    Copying blob e75f007ed29c skipped: already exists
    Copying blob 05e7bc50f07f skipped: already exists
    Copying blob 94058c629e48 skipped: already exists
    Copying blob 6bd5f70f12c5 [--------------------------------------] 0.0b / 0.0b
    Copying config bc9a7579ff done
    Writing manifest to image destination
    Storing signatures
    00:00
    gitlab.com/akosma/greeter
    asciinema.org/a/qS0X1vBrZtMjHZhI8q55p4JfD
    19
    

    View Slide

20. VSHN – The DevOps Company
    1. podman build -t quay.io/akosma/greeter:1.0 .
    2. podman images
    3. podman login quay.io
    4. podman push quay.io/akosma/greeter:1.0
    5.
    Recap'
    quay.io/repository/akosma/greeter
    20
    

    View Slide

21. VSHN – The DevOps Company 21
    

    View Slide

22. VSHN – The DevOps Company
    0:00
    22
    

    View Slide

23. VSHN – The DevOps Company 23
    

    View Slide

24. VSHN – The DevOps Company
    3. Reusing Tools in CI/CD Pipelines
    24
    

    View Slide

25. VSHN – The DevOps Company
    .gitlab-ci.yml
    build:
    image: docker:19.03.12
    stage: build
    services:
    - docker:19.03.12-dind
    script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $CI_REGISTRY/akosma/greeter:latest .
    - docker push $CI_REGISTRY/akosma/greeter:latest
    gitlab.com/akosma/greeter/-/blob/master/.gitlab-ci.yml
    25
    

    View Slide

26. VSHN – The DevOps Company
    Pipeline Run
    26
    

    View Slide

27. VSHN – The DevOps Company
    Embedded Container Registry
    27
    

    View Slide

28. VSHN – The DevOps Company
    $ docker login registry.gitlab.com
    ...
    $ podman login registry.gitlab.com
    ...
    28
    

    View Slide

29. VSHN – The DevOps Company
    Reuse
    gitlab.com/akosma/fortune
    29
    

    View Slide

30. VSHN – The DevOps Company
    Docker le
    # Step 1: print a greeting
    FROM registry.gitlab.com/akosma/greeter:latest
    RUN /usr/local/bin/greeter Dockerfile --character snoopy --action think
    # Step 2: build runtime image
    FROM python:3.7-alpine
    RUN apk add fortune
    WORKDIR /usr/src/app
    COPY requirements.txt ./
    RUN pip install --no-cache-dir -r requirements.txt
    COPY app.py /usr/src/app
    COPY templates /usr/src/app/templates/
    USER 1001
    EXPOSE 9090
    CMD [ "python", "app.py" ]
    gitlab.com/akosma/fortune/-/blob/master/Docker le
    30
    

    View Slide

31. VSHN – The DevOps Company
    .gitlab-ci.yml
    stages:
    - build
    .base:
    stage: build
    image: docker:stable
    services:
    - docker:dind
    before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
    docker-build:
    extends: .base
    script:
    - docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" .
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
    only:
    - master
    gitlab.com/akosma/fortune/-/blob/master/.gitlab-ci.yml
    31
    

    View Slide

32. VSHN – The DevOps Company 32
    

    View Slide

33. VSHN – The DevOps Company
    
    4. More Container Tool Ideas
    33
    

    View Slide

34. VSHN – The DevOps Company
    Linting
    Black-box testing
    API documentation generation
    Code Tasks
    34
    

    View Slide

35. VSHN – The DevOps Company
    Find
    using vale
    Create or
    Generate for docs
    for documentation
    Documentation Tasks
    dead links in HTML
    Check writing style
    Spell checking
    PDF EPUB
    search engine index
    Live preview
    35
    

    View Slide

36. VSHN – The DevOps Company
    
    5. "Gotchas"
    36
    

    View Slide

37. VSHN – The DevOps Company
    1. Creating CLI tools
    2. Building containers
    3. Programming language-speci c tips
    Types of Gotchas
    37
    

    View Slide

38. VSHN – The DevOps Company
    Follow the
    Create man page for your tools using
    Pass secrets to tools as environment variables
    Document, document, document
    Use Make les to standardize projects
    5.1 Creating CLI Tools Gotchas
    Command Line Interface Guidelines
    Asciidoctor
    38
    

    View Slide

39. VSHN – The DevOps Company
    pages := $(shell find . -type f -name '*.adoc')
    out_dir := ./docs
    docker_cmd ?= podman
    docker_opts ?= --rm --tty # add --user "$$(id -u)" if using docker!
    antora_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/antora antora/anto
    vale_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/docsrc/modules/ROOT/pa
    hunspell_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}":/spell vshn/hunspe
    preview_cmd ?= $(docker_cmd) run --rm --publish 35729:35729 --publish 2020:2020 --volu
    htmltest_cmd ?= $(docker_cmd) run $(docker_opts) --volume "$${PWD}"/$(out_dir):/test w
    .PHONY: all
    all: html
    .PHONY: clean
    clean:
    rm -rf $(out_dir)
    .PHONY: vale
    vale:
    $(vale_cmd)
    .PHONY: spell
    spell: html
    $(hunspell cmd)
    39
    

    View Slide

40. VSHN – The DevOps Company
    Beware of base images using musl instead of glibc
    …like Alpine!
    Free some disk space regularly with podman image prune
    ENTRYPOINT > CMD
    COPY > ADD
    Use other container registries
    Inspect your images using
    5.2 Gotchas when Building
    Containers
    dive
    40
    

    View Slide

41. VSHN – The DevOps Company
    Self-hosted: , , , and
    SaaS: , , ,
    ( ), and ( )
    Embedded: and
    Container Registries
    kraken Harbor Docker distribution
    Sonatype Nexus
    Docker Pro or Team plans Quay AWS ECR GitHub
    Packages ghcr.io Google Container Registry gcr.io
    OpenShift GitLab
    41
    

    View Slide

42. VSHN – The DevOps Company
    1. Python
    2. Go
    3. JavaScript
    4. Shell scripts
    5.3 Language-speci c tips
    42
    

    View Slide

43. VSHN – The DevOps Company
     Use virtual environments!
    5.3.1 Python Tips
    # During development...
    $ python3 -m venv .venv
    $ source .venv/bin/activate
    $ pip install PyYAML
    $ pip install PyGithub
    $ pip freeze > requirements.txt
    # In the Dockerfile...
    RUN pip install --no-cache-dir -r requirements.txt
    43
    

    View Slide

44. VSHN – The DevOps Company
     Use the base image!
    5.3.2 Go Tips
    distroless
    44
    

    View Slide

45. VSHN – The DevOps Company
    Use TypeScript!
    npm install --prod
    Use to create standalone binaries
    Use the base image for minimalistic, small
    nal container images ( ).
    5.3.3 JavaScript Tips
    pkg
    scratch-node
    GitHub
    45
    

    View Slide

46. VSHN – The DevOps Company
    Use
    And if you use Visual Studio Code, install the
    Beware of base images that do not contain /bin/bash and
    only have /bin/sh (like with )
    Either make sure your shell scripts work with /bin/sh,
    or install bash in your target image
    5.3.4 Shell Scripting Tips
    ShellCheck
    ShellCheck VS extension
    Alpine Busybox
    46
    

    View Slide

47. VSHN – The DevOps Company
    Wrap command line tools in containers
    Use a public or private registries to share your tools
    Document them properly: README + man page + Antora
    website + …
    Beware of common issues
    Summary
    47
    

    View Slide

48. VSHN – The DevOps Company
    Christian Cremer
    Tobias Brunner
    Aarno Aukia
    Acknowledgements
    ccremer on GitHub
    @tobruzh on Twitter
    @aarnoaukia on Twitter
    48
    

    View Slide

49. VSHN – The DevOps Company
    Adrian Kosmaczewski – –
    VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 –
    – –
    Thanks!
    @akosma adrian@vshn.ch
    info@vshn.ch
    vshn.ch @vshn_ch vshn.tv
    49
    

    View Slide