Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intro: Cloud Native Buildpacks - KubeCon EU 2019

Intro: Cloud Native Buildpacks - KubeCon EU 2019

B87c43d4be875c9b41cd436f5c364f75?s=128

hone

May 21, 2019
Tweet

More Decks by hone

Other Decks in Programming

Transcript

  1. None
  2. Terence Lee @hone02

  3. None
  4. $ git push heroku master

  5. Buildpacks: Heroku for Everything (2011) Opinionated, app aware, source-centric way

    to build your apps.
  6. Buildpack Overview • bin/detect • bin/compile ◦ (CF) bin/supply ◦

    (CF) bin/finalize • bin/release Slug Tarball Stack Image ABI Compatibility Guarantee
  7. Ruby Buildpack Steps • installing Ruby • installing and running

    Bundler to manage gem dependencies • injecting database configuration • compiling Rails assets Comprehensive Support ◦ 7 years of battle hardened usage ◦ Used in production by millions of apps ◦ supported MRI as old as 1.8.7 to 2.6.3 (on release day) ◦ Rails 2.x-5.2 ◦ Minimize buildpack upgrade pain/burden
  8. Buildpack Ecosystem (Buildpacks) • Languages ◦ .NET Core ◦ Elixir

    ◦ R • Frontend ◦ create-react-app ◦ Meteor ◦ Jekyll • Tools ◦ NGINX ◦ OpenCV • Off the Shelf Software ◦ Metabase ◦ Spree ◦ Minecraft
  9. Buildpack Ecosystem (Providers)

  10. None
  11. “Writing a quality Dockerfile is still my users' biggest point

    of friction” - David Dollar, CEO, Convox
  12. “Top ten most popular docker images each contain at least

    30 vulnerabilities” “'Mystery meat' OpenJDK builds strike again" in *official* openjdk Docker images on Docker Hub
  13. Leaky Abstraction FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN

    pip install --no-cache-dir -r requirements.txt COPY . . CMD [ "python", "./your-daemon-or-script.py" ]
  14. Least Privileged User FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./

    RUN pip install --no-cache-dir -r requirements.txt COPY . . RUN useradd pythonista USER pythonista CMD [ "python", "./your-daemon-or-script.py" ]
  15. Reducing Image Size

  16. Composability How do we combine two Docker images?

  17. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY . /app
  18. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app
  19. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=java /app/target /app/target
  20. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM openjdk:11-jre as jre FROM ruby COPY --from=jre /docker-java-home /docker-java-home COPY --from=jre /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jre /usr/share/java/ /usr/share/java/ COPY --from=jre /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jre /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jre /usr/bin/java /usr/bin/java COPY --from=jre /usr/bin/jps /usr/bin/jps COPY --from=jre /usr/bin/jshell /usr/bin/jshell COPY --from=jre /usr/bin/jcmd /usr/bin/jcmd COPY --from=jre /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=jdk /app/target /app/target
  21. Composability (Multi-stage Builds) • No environment variables • Doesn’t follow

    symlinks • Limited interface for copying ◦ No support for globs, often need many copy statements ▪ COPY --from=0 /n1 /n1 ▪ COPY --from=0 /n2 /n2 ▪ COPY --from=0 /n3 /n3
  22. Don’t leak sensitive information to Docker images FROM ubuntu as

    intermediate WORKDIR /app COPY secret/key /tmp/ RUN scp -i /tmp/key build@acme/files . FROM ubuntu WORKDIR /app COPY --from=intermediate /app .
  23. meet developers where they are: their app source code

  24. None
  25. Day-2

  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. None
  44. None
  45. None
  46. None
  47. None
  48. None
  49. None
  50. Optimized Builds – How it Works • Only re-builds and

    uploads layers when necessary • OCI image specification: content addressable layers • Docker Registry v2: cross repository blob mounting Result: Fast builds, minimal data transfer, layer “rebasing” directly on the registry!
  51. New Buildpack API bin/detect bin/supply bin/finalize bin/build bin/detect Old Buildpack

    Interface New Buildpack Interface plan TOML bin/release
  52. New Buildpack API Build Detect Analysis Export where metadata about

    OCI layers generated during a previous build are made available to buildpacks where the remote layers are replaced by the generated layers where an optimal selection of compatible buildpacks is chosen and a build plan is created where buildpacks use that metadata to generate only the OCI layers that need to be replaced
  53. bin/detect ruby_buildpack Ruby + Node.js App Gemfile package.json app/ ✓

    plan.toml bin/detect nodejs_buildpack ✓ bin/build bin/build [ruby] version = "2.5.1" [ruby.metadata] launch = true [node] version = "8.12.0" [node.metadata] launch = true Node.js layer Ruby layer Node modules layer Multiple Buildpack Support Buildpack Group Gems layer
  54. Build Steps – New API Build Install ruby bundle install

    Install node npm install OCI Image Ruby + Node.js App First Build Analysis First build, nothing to do Export Create nodejs layer Create node_modules layer Create app layer Create mri layer Create gem layer Create OS layer mri layer modules layer app layer configuration layer gems layer nodejs layer ubuntu:18.04
  55. Build (w/ cache + metadata) Read metadata from disk bundle

    install (with cached gems) npm install (with cached modules) Ruby + Node.js App node modules updated gems updated Second Build Analysis Read metadata about layers Write metadata to disk for build Export Update app layer Update gems layer Update modules layer OCI Image mri layer modules layer app layer configuration layer gems layer nodejs layer ubuntu:18.04 app layer modules layer gems layer
  56. • ◦ ◦ ◦ • ◦ • ◦