Upgrade to Pro — share decks privately, control downloads, hide ads and more …

KubeCon 2018 - Intro: Cloud Native Buildpacks

hone
December 11, 2018

KubeCon 2018 - Intro: Cloud Native Buildpacks

Presented by Joe Kutner and myself.
Video here: https://videos.videoken.com/index.php/videos/intro-cloud-native-buildpacks-terence-lee-joe-kutner-salesforce-heroku/

You're great at running containers but you shouldn't have to be great at building them. In this talk, you'll learn about Cloud Native Buildpacks, a higher-level abstraction for building apps compared to Dockerfiles.

Buildpacks are a standardized tool for creating images in a secure, reproducible, and efficient manner. As an app developer, you don't need to know best practices around ordering commands for layer reuse. As an operator, you don't need to worry about exposing developers to the responsibilities that come with Dockerfile.

Come learn how buildpacks meet developers at their source code, automate the delivery of both OS-level and application-level dependency upgrades, and help you efficiently handle day-2 app operations.

hone

December 11, 2018
Tweet

More Decks by hone

Other Decks in Programming

Transcript

  1. Dockerfile (Ruby) FROM ruby COPY . /app WORKDIR /app RUN

    bundle install EXPOSE 5000 CMD bundle exec ruby ./app.rb
  2. Building Docker Images $ docker build . Step 1/6 :

    FROM ruby ---> 88666731c3e1 Step 2/6 : COPY . /app ---> 173624d82900 Step 3/6 : WORKDIR /app ---> Running in 0649f4408d91 ---> 850a46b3ec29 Step 4/6 : RUN bundle install ---> Running in ed644f258949 ---> 499852b8c318 Step 5/6 : EXPOSE 5000 ---> Running in fc0958926d74 ---> 4d4fbc35cde5 Step 6/6 : CMD bundle exec ruby ./app.rb ---> Running in 0e901e0910db ---> 881870f15126 Successfully built 881870f15126
  3. Dockerfile (Rails) FROM ruby COPY . /app WORKDIR /app RUN

    bundle install EXPOSE 5000 CMD bin/rails
  4. Dockerfile (Rails) FROM ruby RUN apt-get update -qq \ &&

    apt-get install -y nodejs libpq-dev build-essential COPY . /app WORKDIR /app RUN bundle install RUN bundle exec rake assets:precompile EXPOSE 5000 CMD bin/rails
  5. Build image best practices? • Reduce image size • Speed

    up incremental builds • Pick base image
  6. Dockerfile Layers Step 1/6 : FROM ruby ---> 88666731c3e1 Step

    2/6 : COPY . /app ---> 173624d82900 Step 3/6 : WORKDIR /app ---> Running in 0649f4408d91 ---> 850a46b3ec29 Step 4/6 : RUN bundle install ---> Running in ed644f258949 ---> 499852b8c318 Step 5/6 : EXPOSE 5000 ---> Running in fc0958926d74 ---> 4d4fbc35cde5 Step 6/6 : CMD bundle exec ruby ./app.rb ---> Running in 0e901e0910db ---> 881870f15126 Successfully built 881870f15126 FROM ruby:latest COPY . /app WORKDIR /app RUN bundle install EXPOSE 5000 CMD bundle exec ruby ./app.rb
  7. Reduce image size FROM ruby RUN apt-get update -qq \

    && apt-get install -y nodejs libpq-dev build-essential \ && apt-get clean autoclean && apt-get autoremove -y \ && rm -rf /var/lib/apt /var/lib/dpkg /var/lib/cache /var/lib/log COPY . /app WORKDIR /app RUN bundle install RUN bundle exec rake assets:precompile \ && rm -rf /app/tmp/cache/assets/ EXPOSE 5000 CMD bin/rails
  8. Speed Up Incremental Builds FROM ruby RUN apt-get update -qq

    \ && apt-get install -y nodejs libpq-dev build-essential \ && apt-get clean autoclean && apt-get autoremove -y \ && rm -rf /var/lib/apt /var/lib/dpkg /var/lib/cache /var/lib/log ADD Gemfile /app/ ADD Gemfile.lock /app/ RUN bundle install COPY . /app WORKDIR /app RUN bundle exec rake assets:precompile \ && rm -rf /app/tmp/cache/assets/ EXPOSE 5000 CMD bin/rails
  9. Pick Base Image FROM ruby RUN apt-get update -qq \

    && apt-get install -y nodejs libpq-dev build-essential COPY . /app WORKDIR /app RUN bundle install RUN bundle exec rake assets:precompile EXPOSE 5000 CMD bin/rails
  10. “Writing a quality Dockerfile is still my users' biggest point

    of friction” - David Dollar, CEO, Convox
  11. Dockerfile Shortcomings • Maintenance / Day 2 Operations (Security) •

    Not App Aware • Composability • Leaky Abstraction
  12. Maintenance / Day 2 Operations FROM ruby:latest RUN mkdir /usr/src/app

    ADD . /usr/src/app/ WORKDIR /usr/src/app/ RUN bundle install CMD ["/usr/src/app/main.rb"]
  13. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY . /app
  14. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app
  15. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=java /app/target /app/target
  16. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM openjdk:11-jre as jre FROM ruby COPY --from=jre /docker-java-home /docker-java-home COPY --from=jre /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jre /usr/share/java/ /usr/share/java/ COPY --from=jre /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jre /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jre /usr/bin/java /usr/bin/java COPY --from=jre /usr/bin/jps /usr/bin/jps COPY --from=jre /usr/bin/jshell /usr/bin/jshell COPY --from=jre /usr/bin/jcmd /usr/bin/jcmd COPY --from=jre /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=jdk /app/target /app/target
  17. Composability (Multi-stage Builds) • No environment variables • Doesn’t follow

    symlinks • Only copying FS layers manually ◦ Can’t copy arbitrary layers/files/dirs ▪ COPY --from=0 /n1 /n1 ▪ COPY --from=0 /n2 /n2 ▪ COPY --from=0 /n3 /n3
  18. Leaky Abstraction Poor tool for app developers who just want

    to write code. Authoring a good Dockerfile requires too much knowledge of the underlying mechanisms. Mix of operation and app developer concerns.
  19. Ruby Buildpack • Steps ◦ installing Ruby ◦ installing and

    running Bundler to manage gem dependencies ◦ injecting database configuration ◦ compiling Rails assets • Comprehensive Support ◦ 7 years of battle hardened usage ◦ Used in production by millions of apps ◦ supported MRI as old as 1.8.7 to 2.6.0-rc1 (on release day) ◦ Rails 2.x-5.2 ◦ Minimize buildpack upgrade pain/burden
  20. Buildpack Ecosystem • Languages ◦ .NET Core ◦ Elixir ◦

    R • Frontend ◦ create-react-app ◦ Meteor ◦ Jekyll • Tools ◦ NGINX ◦ OpenCV • Off the Shelf Software ◦ Metabase ◦ Spree ◦ Minecraft
  21. Cloud Native Buildpacks (2018) Combine the power of buildpacks with

    the benefits of containers. Cloud Native Sandbox Project (CNCF) Incorporate learnings from Pivotal and Heroku. Let developers focus on their app and not piecing together a build pipeline.
  22. Pack - Buildpack CLI application developers to use buildpacks to

    convert code into runnable images • Building app images with build • Updating app images using rebase
  23. Docker Image Manifest • tarballs of layers • configuration for

    container ◦ Command ◦ Entrypoint ◦ Env Vars
  24. Docker Image Manifest { "schemaVersion": 2, "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "config": {

    "mediaType": "application/vnd.docker.container.image.v1+json", "size": 7023, "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7" }, "layers": [ { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 32654, "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 16724, "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 73109, "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736" } ] }
  25. Docker Image Manifest { "schemaVersion": 2, "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "config": {

    "mediaType": "application/vnd.docker.container.image.v1+json", "size": 7023, "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7" }, "layers": [ { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 32654, "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 16724, "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 73109, "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736" } ] }
  26. Buildpack Image Building • Day 2 Operations • App Aware

    Image Builder • Composability • Higher Level Abstraction
  27. Day 2 Operations • Fast as a feature • Stack

    image updates • Unified build pipelines
  28. App Aware Image Builder • Intentional about docker layers •

    Intelligent about caching • Smart defaults: memory, concurrency, commands
  29. Try Buildpacks today! • Github repo ◦ Specification (WIP): https://github.com/buildpack/spec

    ◦ Lifecycle: https://github.com/buildpack/lifecycle ◦ Pack CLI: https://github.com/buildpack/pack • Slack ◦ https://slack.buildpacks.io/ • Samples ◦ https://github.com/buildpack/samples ◦ https://github.com/heroku/java-buildpack ◦ https://github.com/cloudfoundry/nodejs-cnb ◦ https://github.com/cloudfoundry/npm-cnb ◦ https://github.com/jkutner/python-buildpack
  30. Deep Dive • Thursday @ 2:35pm • Tahoma 5 @

    TCC Joe Kutner @codefiner Terence Lee @hone02