Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 1 Create multi-stage, multi-arch Go container images for your heterogeneous Kubernetes clusters 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Golang 2 Simple and helpful… Go was designed at Google ] by Robert Griesemer, Rob Pike, and Ken Thompson in 2007 to improve programming productivity in an era of multicore, networked machines and extremely large codebases. Container technologies like Docker and Kubernetes powered by golang 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Why build GO apps 
 with containers? 3 •Use or implement OpenSource projects that written in Go •Ship your apps with container is state of the art •Build software for different environments (OS, CPU, …) •Use multiple golang version in parallel •Solve your golang dependency problems •Reduce your size of your app shipments for better scale 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion What’s the problem? 4 •compiling is slow What we need? •Speed up compiling •Reduce size •Multi Arch •Pipeline Builds •Running at Kubernetes 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Native build 5 FROM golang:1.11 RUN go get github.com/golang/dep/cmd/dep WORKDIR /go/src/github.com/bee42/whoamI ADD . . # Download deps RUN dep ensure # Install RUN go install github.com/bee42/whoamI ENTRYPOINT /go/bin/whoamI >50 sec 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion With cached deps 6 FROM golang:1.11 RUN go get github.com/golang/dep/cmd/dep WORKDIR /go/src/github.com/bee42/whomaI # Download deps ADD Gopkg.* ./ RUN dep ensure --vendor-only # Install source ADD . . RUN go install github.com/bee42/whomaI # Keep the container open ENTRYPOINT tail -f /dev/null 24 secs 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion With cached obj - I 7 # Start builder ARG baseImage="golang:1.11" FROM ${baseImage} as builder RUN go get github.com/golang/dep/cmd/dep WORKDIR /go/src/github.com/bee42/whoamI # Download deps ADD Gopkg.* ./ RUN dep ensure --vendor-only # Install source ADD . . RUN go install github.com/bee42/whoamI # Done builder
 … 24 secs 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion With cached obj - II 8 # Start builder ARG baseImage="golang:1.11" FROM ${baseImage} as builder … # Start obj-cache FROM golang:1.11 as obj-cache COPY --from=builder /root/.cache /root/.cache # Done obj-cache # Start main FROM builder ENTRYPOINT /go/bin/whoamI # Done main 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Use cached obj 9 cacheobjs-base: if [ "$(shell docker images bee42/whoamI/cacheobjs-base -q)" = "" ]; then \ docker build -t bee42/whoamI/cacheobjs-base -f Dockerfile.cacheobjs --target=obj-cache .; \ fi; cacheobjs: cacheobjs-base docker build --build-arg baseImage=bee42/whoamI/cacheobjs-base \ -t bee42/whoamI/cacheobjs \ -f Dockerfile.cacheobjs . docker run --rm -it bee42/whoamI/cacheobjs 8 secs 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Taily Build 10 FROM golang:1.11 RUN go get github.com/golang/dep/cmd/dep WORKDIR /go/src/github.com/bee42/whoamI # Download deps ADD Gopkg.* ./ RUN dep ensure --vendor-only # Install source ADD . . RUN go install github.com/bee42/whoamI # Keep the container open ENTRYPOINT tail -f /dev/null 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Use taily build 11 tailybuild-base: if [ "$(shell docker ps --filter=name=tailybuild -q)" = "" ]; then \ docker build -t bee42/whomaI/tailybuild-base -f Dockerfile.tailybuild .; \ docker run --name tailybuild -d bee42/whoami/tailybuild-base; \ fi; tailybuild: tailybuild-base docker exec -it tailybuild rm -fR whoamI docker cp cmd tailybuild:/go/src/github.com/bee42/whoamI/ docker exec -it tailybuild go install github.com/bee42/whoamI docker exec -it tailybuild /go/bin/whomaI 3 secs 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Taily Mount 12 FROM golang:1.11 RUN go get github.com/golang/dep/cmd/dep WORKDIR /go/src/github.com/bee42/whoamI # Download deps ADD Gopkg.* ./ RUN dep ensure --vendor-only # Install source ADD . . RUN go install github.com/bee42/whoamI # Keep the container open ENTRYPOINT tail -f /dev/null 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Use taily mount 13 tailymount-base: if [ "$(shell docker ps --filter=name=tailymount -q)" = "" ]; then \ docker build -t bee42/whoamI/tailymount-base -f Dockerfile.tailymount .; \ docker run --mount type=bind,source=$(shell pwd)/cmd,target=/go/src/github.com/bee42/whoamI \ --name tailymount -d bee42/whoamI/tailymount-base; \ fi; tailymount: tailymount-base docker exec -it tailymount go install github.com/bee42/whoamI docker exec -it tailymount /go/bin/whoamI 2.4 secs 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Yeah > Save life time… 14 • https://medium.com/windmill-engineering/tips-tricks-for-making-your-golang-container-builds-10x-faster-4cc618a43827 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Reduce Size 15 •Prepare a Tool Container •Multi Stage Build •Compression •Improve quality 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Choose the right base images 16 https://www.codacy.com/blog/five-ways-to-slim-your-docker-images/ 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Strategies to reduce image size 17 •Think Carefully About Your Application’s Needs •Use a Small Base Image •Use as Few Layers As Possible •Use .dockerignore files •Squash Docker Images 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion go dep tool base container 18 ARG BASE_IMAGE=${BASE_IMAGE:-golang:1.11.5-alpine3.8} FROM ${BASE_IMAGE} LABEL maintainer="Peter Rossbach " ARG DEP_VERSION=${DEP_VERSION:-0.5.0} RUN apk update; \ apk add --no-cache \ ca-certificates \ curl \ git \ make \ openssl; \ curl -L -s https://github.com/golang/dep/releases/download/v${DEP_VERSION}/dep-linux-amd64 \ -o /bin/dep; \ chmod +x /bin/dep; \ rm -rf /var/cache/apk/*; \ rm -rf /tmp/*; 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Builder Image 19 FROM bee42.com/containers/tools/go-dep:1.11.5-alpine3.8 as builder ARG TARGET_ARCH=${TARGET_ARCH:-amd64} ARG APP=${APP:-bee42.com/containers/examples/k8s-client/blinkt} ENV CGO_ENABLED=0 ENV APP_GOPATH $GOPATH/src/$APP WORKDIR $APP_GOPATH RUN mkdir -p $APP_GOPATH COPY vendor/ $APP_GOPATH/vendor/ COPY Gopkg.* $APP_GOPATH/ COPY *.go $APP_GOPATH RUN cd $APP_GOPATH && \ GOOS=linux GOARCH=${TARGET_ARCH} GOARM=${GOARM:-7} go build -a --installsuffix cgo -- ldflags="-s" -o blinkt # Resulting App … 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Resulting Image 20 FROM bee42.com/containers/tools/go-dep:1.11.5-alpine3.8 as builder … # Resulting App FROM alpine:v3.8 COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=builder /app/blinkt /app/blinkt WORKDIR /app ENTRYPOINT ["/app/blinkt"] 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion UPX 21 https://upx.github.io UPX achieves an excellent compression ratio and offers very fast decompression. 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion UPX Package Compression 22 # Optimize binary size FROM alpine:v3.8 as packager ARG APP=${APP:-bee42.com/containers/examples/k8s-client/blinkt} ENV APP_GOPATH /go/src/$APP ARG TARGET_ARCH=${TARGET_ARCH:-amd64} ARG UPX_VERSION=${UPX_VERSION:-3.95} RUN apk add --no-cache xz binutils curl && echo ${TARGET_ARCH} RUN curl -sL -o /tmp/upx-${UPX_VERSION}-${TARGET_ARCH}_linux.tar.xz \ https://github.com/upx/upx/releases/download/v${UPX_VERSION}/upx-${UPX_VERSION}-$ {TARGET_ARCH}_linux.tar.xz && \ xz -d -c /tmp/upx-${UPX_VERSION}-${TARGET_ARCH}_linux.tar.xz | \ tar -xOf - upx-${UPX_VERSION}-${TARGET_ARCH}_linux/upx > /bin/upx && \ chmod a+x /bin/upx && \ rm /tmp/upx-${UPX_VERSION}-${TARGET_ARCH}_linux.tar.xz COPY --from=builder $APP_GOPATH/blinkt /app/blinkt RUN cd /app && \ strip --strip-unneeded blinkt && \ upx blinkt # Resulting App 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Use Upx package compression 23 # Optimize binary size FROM alpine:v3.8 as packager … # Resulting App FROM alpine:v3.8 COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=packager /app/blinkt /app/blinkt WORKDIR /app ENTRYPOINT ["/app/blinkt"] Safe 20-40% image size 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 24 Build Cross Compiled Binaries: 
 qemu static docker run --rm --privileged multiarch/qemu-user-static:register --reset https://hub.docker.com/r/multiarch/qemu-user-static for target_arch in aarch64 arm x86_64; do wget -N https://github.com/multiarch/qemu-user-static/releases/download/v2.9.1-1/x86_64_qemu-$ {target_arch}-static.tar.gz tar -xvf x86_64_qemu-${target_arch}-static.tar.gz done https://lobradov.github.io/Building-docker-multiarch-images/ Registry kernel modules Build with emulation binary 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Multiarch Docker hub 25 https://hub.docker.com/u/multiarch/ https://github.com/multiarch 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Multi Arch build matrix 26 https://doi-janky.infosiftr.net/job/multiarch/job/arm32v7/job/httpd/ 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Build with separate Dockerfiles 27 cat >Dockerfile.amd64 <Dockerfile.arm32v6 <

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Resulting Image 28 for arch in amd64 arm32v6; do docker build -f Dockerfile.${arch} -t bee42/nginx:${arch}-latest . docker push bee42/nginx:${arch}-latest done 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Builder Image with Args 29 ARG GOLANG_TARGET=${GOLANG_TARGET:-bee42.com/containers/tools/go-dep:1.11.5-alpine3.8} ARG TARGET=${TARGET:-multiarch/alpine:armhf-v3.8} FROM ${GOLANG_TARGET} as builder ARG TARGET_ARCH=${TARGET_ARCH:-arm} ARG APP=${APP:-bee42.com/containers/examples/k8s-client/blinkt} ENV CGO_ENABLED=0 ENV APP_GOPATH $GOPATH/src/$APP WORKDIR $APP_GOPATH RUN mkdir -p $APP_GOPATH COPY vendor/ $APP_GOPATH/vendor/ COPY Gopkg.* $APP_GOPATH/ COPY *.go $APP_GOPATH RUN cd $APP_GOPATH && \ GOOS=linux GOARCH=${TARGET_ARCH} GOARM=${GOARM:-7} go build -a --installsuffix cgo -- ldflags="-s" -o blinkt # Resulting App … 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Resulting Image 30 ARG GOLANG_TARGET=${GOLANG_TARGET:-bee42.com/containers/tools/go-dep:1.11.5-alpine3.8} ARG TARGET=${TARGET:-multiarch/alpine:armhf-v3.8} FROM ${GOLANG_TARGET} as builder … # Resulting App FROM ${TARGET} COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=buillder /app/blinkt /app/blinkt WORKDIR /app ENTRYPOINT ["/app/blinkt"] 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Build with Multi Arch Images 31 build: case $${arch} in \ amd64 ) target_image="multiarch/alpine:amd64-v3.8" ;; \ arm ) target_image="multiarch/alpine:armhf-v3.8" ;; \ arm64 ) target_image="multiarch/alpine:arm64-v3.8" ;; \ esac ; \ docker image build --no-cache \ --build-arg TARGET=$${target_image} \ --build-arg TARGET_ARCH=$${arch} \ -t $(DOCKER_IMAGE):$(DOCKER_TAG)-$${arch} . ; \ 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Build with Mutli Arch Images 32 build-multiarch: @for arch in $(MULTIARCH); do \ case $${arch} in \ amd64 ) target_image="multiarch/alpine:amd64-v3.8" ;; \ arm ) target_image="multiarch/alpine:armhf-v3.8" ;; \ arm64 ) target_image="multiarch/alpine:arm64-v3.8" ;; \ esac ; \ docker image build --no-cache \ --build-arg TARGET=$${target_image} \ --build-arg TARGET_ARCH=$${arch} \ --build-arg VERSION=`cat VERSION` \ --build-arg VCS_REF=$(DOCKER_TAG) \ --build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` \ -t $(DOCKER_IMAGE):$(DOCKER_TAG)-$${arch} . ; \ done 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 33 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Create Multiarch Manifest 34 push-multiarch: @echo "Create and push multiarch manifest: " @for arch in $(MULTIARCH); do \ docker image push $(DOCKER_IMAGE):$(DOCKER_TAG)-$${arch} ; \ done @docker manifest create $(DOCKER_IMAGE):$(MANIFEST_TAG) \ $(DOCKER_IMAGE):$(DOCKER_TAG)-amd64 \ $(DOCKER_IMAGE):$(DOCKER_TAG)-arm \ $(DOCKER_IMAGE):$(DOCKER_TAG)-arm64 @for arch in $(MULTIARCH); do \ case $${arch} in \ amd64 ) manifest_annotate="" ;; \ arm ) manifest_annotate="--os linux --arch arm" ;; \ arm64 ) manifest_annotate="--os linux --arch arm64 --variant armv8" ;; \ esac ; \ docker manifest annotate $(DOCKER_IMAGE):$(MANIFEST_TAG) $(DOCKER_IMAGE):$(DOCKER_TAG)- $${arch} $${manifest_annotate} ;\ done @docker manifest push $(DOCKER_IMAGE):$(MANIFEST_TAG) 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Docker 18.09 
 client experimental feature 35 mkdir -p ~/.docker cat > "$HOME/.docker/config.json" <

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Add Metadata 36 # Metadata ARG VCS_REF ARG BUILD_DATE ARG VERSION # Metadata LABEL maintainer="bee42 cloud native crew " \ org.opencontainers.image.title="blinkt" \ org.opencontainers.image.version="${VERSION}" \ org.opencontainers.image.revision="${VCS_REF}" \ org.opencontainers.image.created="${BUILD_DATE}" \ org.opencontainers.image.url="https://r-gitlab.bee42.com/containers/examples/k8s-client/blinkt/" \ org.opencontainers.image.source="https://gitlab.bee42.com/containers/examples/k8s-client/blinkt/" \ org.opencontainers.image.authors="bee42 cloud native crew " \ org.opencontainers.image.vendor="bee42 solutions gmbh" \ org.opencontainers.image.licenses="Apache-2.0" \ com.bee42.image.type="service-stateless" \ https://github.com/opencontainers/image-spec/blob/master/annotations.md 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 37 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 38 • Kubernetes is a container orchestrator. • It’s how to run containers at scale. • It’s a very active open-source platform with lots of contributors, start at 6. June 2014 • Originally developed by Google and 
 donated to Cloud Native Computing Foundation 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 39 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 40 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 41 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 42 https://github.com/bee42/kubernetes-on-embedded https://blog.hypriot.com/post/setup-kubernetes-raspberry-pi-cluster/ Blinkt - Demo https://github.com/apprenda/blinkt-k8s-controller https://github.com/StefanScherer/swarm-monitor 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 43 Your Mac Ethernet Switch bee42-crew— 03-001 bee42- crew-03-002 bee42- crew-03-003 Edge Max DNS 192.168.42.31 192.168.42.32 192.168.42.33 192.168.42.101 192.168.42.1 Master Nodes Raspberry PI 3+ armv7 bee42-crew— 03-004 192.168.42.34 Nodes Raspberry PI 3+ arm64 bee42-crew— 03-005 192.168.42.35 Nodes UP Board amd64 192.168.1.230 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Kubernetes Deploy Blinkt 44 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Facts of the multistage 
 and multi arch adventure 45 • Speed up with golang inside container is possible. • Caching the dependencies • Use compiling inside run container • Reduce of images size with multistage build (copy resulting binary) • Reduce size with UPX • Copy Binary • Multi Arch image building • Use qemu-user-static at intel • Sometimes you need native builder instances • Create image manifests to use same image reference inside your deployment resources • Add metadata to your images • Setup and maintain a heterogeneous machine cluster is hard. • Compiling your own kernel • Manuell setup of machines • Wait for features 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion rethink IT Build herogenous kubernetes cluster with embedded machines is a funny adventure… 46 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Kubernetes poster pre registration started https://tinyurl.com/y9js3p7w 47 delivery starts at 42ten day of the year 2019 PREVIEW PREVIEW WE 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Cloud Native System Architect & bee42 founder Peter Roßbach @PRossbach [email protected] https://bee42.com https://devops-gathering.io 48 Save the date… #DOG19 11.-13. March 2019 at Bochum 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion bee42 Trainings 49 https://bee42.com/de/trainings 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 50 We hiring :-) 
 https://bit.ly/2K8DtRu 
 [email protected]
 @bee42solutions