Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Terence Lee @hone02

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

$ git push heroku master

Slide 5

Slide 5 text

Buildpacks: Heroku for Everything (2011) Opinionated, app aware, source-centric way to build your apps.

Slide 6

Slide 6 text

Buildpack Overview ● bin/detect ● bin/compile ○ (CF) bin/supply ○ (CF) bin/finalize ● bin/release Slug Tarball Stack Image ABI Compatibility Guarantee

Slide 7

Slide 7 text

Ruby Buildpack Steps ● installing Ruby ● installing and running Bundler to manage gem dependencies ● injecting database configuration ● compiling Rails assets Comprehensive Support ○ 7 years of battle hardened usage ○ Used in production by millions of apps ○ supported MRI as old as 1.8.7 to 2.6.3 (on release day) ○ Rails 2.x-5.2 ○ Minimize buildpack upgrade pain/burden

Slide 8

Slide 8 text

Buildpack Ecosystem (Buildpacks) ● Languages ○ .NET Core ○ Elixir ○ R ● Frontend ○ create-react-app ○ Meteor ○ Jekyll ● Tools ○ NGINX ○ OpenCV ● Off the Shelf Software ○ Metabase ○ Spree ○ Minecraft

Slide 9

Slide 9 text

Buildpack Ecosystem (Providers)

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

“Writing a quality Dockerfile is still my users' biggest point of friction” - David Dollar, CEO, Convox

Slide 12

Slide 12 text

“Top ten most popular docker images each contain at least 30 vulnerabilities” “'Mystery meat' OpenJDK builds strike again" in *official* openjdk Docker images on Docker Hub

Slide 13

Slide 13 text

Leaky Abstraction FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . CMD [ "python", "./your-daemon-or-script.py" ]

Slide 14

Slide 14 text

Least Privileged User FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . RUN useradd pythonista USER pythonista CMD [ "python", "./your-daemon-or-script.py" ]

Slide 15

Slide 15 text

Reducing Image Size

Slide 16

Slide 16 text

Composability How do we combine two Docker images?

Slide 17

Slide 17 text

Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY . /app

Slide 18

Slide 18 text

Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app

Slide 19

Slide 19 text

Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=java /app/target /app/target

Slide 20

Slide 20 text

Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app RUN ./mvnw clean install FROM openjdk:11-jre as jre FROM ruby COPY --from=jre /docker-java-home /docker-java-home COPY --from=jre /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jre /usr/share/java/ /usr/share/java/ COPY --from=jre /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jre /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jre /usr/bin/java /usr/bin/java COPY --from=jre /usr/bin/jps /usr/bin/jps COPY --from=jre /usr/bin/jshell /usr/bin/jshell COPY --from=jre /usr/bin/jcmd /usr/bin/jcmd COPY --from=jre /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=jdk /app/target /app/target

Slide 21

Slide 21 text

Composability (Multi-stage Builds) ● No environment variables ● Doesn’t follow symlinks ● Limited interface for copying ○ No support for globs, often need many copy statements ■ COPY --from=0 /n1 /n1 ■ COPY --from=0 /n2 /n2 ■ COPY --from=0 /n3 /n3

Slide 22

Slide 22 text

Don’t leak sensitive information to Docker images FROM ubuntu as intermediate WORKDIR /app COPY secret/key /tmp/ RUN scp -i /tmp/key build@acme/files . FROM ubuntu WORKDIR /app COPY --from=intermediate /app .

Slide 23

Slide 23 text

meet developers where they are: their app source code

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

Day-2

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

Optimized Builds – How it Works ● Only re-builds and uploads layers when necessary ● OCI image specification: content addressable layers ● Docker Registry v2: cross repository blob mounting Result: Fast builds, minimal data transfer, layer “rebasing” directly on the registry!

Slide 51

Slide 51 text

New Buildpack API bin/detect bin/supply bin/finalize bin/build bin/detect Old Buildpack Interface New Buildpack Interface plan TOML bin/release

Slide 52

Slide 52 text

New Buildpack API Build Detect Analysis Export where metadata about OCI layers generated during a previous build are made available to buildpacks where the remote layers are replaced by the generated layers where an optimal selection of compatible buildpacks is chosen and a build plan is created where buildpacks use that metadata to generate only the OCI layers that need to be replaced

Slide 53

Slide 53 text

bin/detect ruby_buildpack Ruby + Node.js App Gemfile package.json app/ ✓ plan.toml bin/detect nodejs_buildpack ✓ bin/build bin/build [ruby] version = "2.5.1" [ruby.metadata] launch = true [node] version = "8.12.0" [node.metadata] launch = true Node.js layer Ruby layer Node modules layer Multiple Buildpack Support Buildpack Group Gems layer

Slide 54

Slide 54 text

Build Steps – New API Build Install ruby bundle install Install node npm install OCI Image Ruby + Node.js App First Build Analysis First build, nothing to do Export Create nodejs layer Create node_modules layer Create app layer Create mri layer Create gem layer Create OS layer mri layer modules layer app layer configuration layer gems layer nodejs layer ubuntu:18.04

Slide 55

Slide 55 text

Build (w/ cache + metadata) Read metadata from disk bundle install (with cached gems) npm install (with cached modules) Ruby + Node.js App node modules updated gems updated Second Build Analysis Read metadata about layers Write metadata to disk for build Export Update app layer Update gems layer Update modules layer OCI Image mri layer modules layer app layer configuration layer gems layer nodejs layer ubuntu:18.04 app layer modules layer gems layer

Slide 56

Slide 56 text

● ○ ○ ○ ● ○ ● ○