Upgrade to Pro — share decks privately, control downloads, hide ads and more …

KubeCon 2018 - Intro: Cloud Native Buildpacks

B87c43d4be875c9b41cd436f5c364f75?s=47 hone
December 11, 2018

KubeCon 2018 - Intro: Cloud Native Buildpacks

Presented by Joe Kutner and myself.
Video here: https://videos.videoken.com/index.php/videos/intro-cloud-native-buildpacks-terence-lee-joe-kutner-salesforce-heroku/

You're great at running containers but you shouldn't have to be great at building them. In this talk, you'll learn about Cloud Native Buildpacks, a higher-level abstraction for building apps compared to Dockerfiles.

Buildpacks are a standardized tool for creating images in a secure, reproducible, and efficient manner. As an app developer, you don't need to know best practices around ordering commands for layer reuse. As an operator, you don't need to worry about exposing developers to the responsibilities that come with Dockerfile.

Come learn how buildpacks meet developers at their source code, automate the delivery of both OS-level and application-level dependency upgrades, and help you efficiently handle day-2 app operations.

B87c43d4be875c9b41cd436f5c364f75?s=128

hone

December 11, 2018
Tweet

More Decks by hone

Other Decks in Programming

Transcript

  1. Cloud Native Buildpacks Intro

  2. Joe Kutner @codefinger

  3. Terence Lee @hone02

  4. None
  5. k8s is a great platform to run images

  6. What's the best way to build images?

  7. None
  8. Containerize a Ruby App

  9. Dockerfile (Ruby) FROM ruby COPY . /app WORKDIR /app RUN

    bundle install EXPOSE 5000 CMD bundle exec ruby ./app.rb
  10. Building Docker Images $ docker build .

  11. Building Docker Images $ docker build . Step 1/6 :

    FROM ruby ---> 88666731c3e1 Step 2/6 : COPY . /app ---> 173624d82900 Step 3/6 : WORKDIR /app ---> Running in 0649f4408d91 ---> 850a46b3ec29 Step 4/6 : RUN bundle install ---> Running in ed644f258949 ---> 499852b8c318 Step 5/6 : EXPOSE 5000 ---> Running in fc0958926d74 ---> 4d4fbc35cde5 Step 6/6 : CMD bundle exec ruby ./app.rb ---> Running in 0e901e0910db ---> 881870f15126 Successfully built 881870f15126
  12. Containerize Rails?

  13. Dockerfile (Rails) FROM ruby COPY . /app WORKDIR /app RUN

    bundle install EXPOSE 5000 CMD bin/rails
  14. Dockerfile (Rails) FROM ruby RUN apt-get update -qq \ &&

    apt-get install -y nodejs libpq-dev build-essential COPY . /app WORKDIR /app RUN bundle install RUN bundle exec rake assets:precompile EXPOSE 5000 CMD bin/rails
  15. Build image best practices? • Reduce image size • Speed

    up incremental builds • Pick base image
  16. Dockerfile Layers Step 1/6 : FROM ruby ---> 88666731c3e1 Step

    2/6 : COPY . /app ---> 173624d82900 Step 3/6 : WORKDIR /app ---> Running in 0649f4408d91 ---> 850a46b3ec29 Step 4/6 : RUN bundle install ---> Running in ed644f258949 ---> 499852b8c318 Step 5/6 : EXPOSE 5000 ---> Running in fc0958926d74 ---> 4d4fbc35cde5 Step 6/6 : CMD bundle exec ruby ./app.rb ---> Running in 0e901e0910db ---> 881870f15126 Successfully built 881870f15126 FROM ruby:latest COPY . /app WORKDIR /app RUN bundle install EXPOSE 5000 CMD bundle exec ruby ./app.rb
  17. Reduce image size FROM ruby RUN apt-get update -qq \

    && apt-get install -y nodejs libpq-dev build-essential \ && apt-get clean autoclean && apt-get autoremove -y \ && rm -rf /var/lib/apt /var/lib/dpkg /var/lib/cache /var/lib/log COPY . /app WORKDIR /app RUN bundle install RUN bundle exec rake assets:precompile \ && rm -rf /app/tmp/cache/assets/ EXPOSE 5000 CMD bin/rails
  18. Speed Up Incremental Builds FROM ruby RUN apt-get update -qq

    \ && apt-get install -y nodejs libpq-dev build-essential \ && apt-get clean autoclean && apt-get autoremove -y \ && rm -rf /var/lib/apt /var/lib/dpkg /var/lib/cache /var/lib/log ADD Gemfile /app/ ADD Gemfile.lock /app/ RUN bundle install COPY . /app WORKDIR /app RUN bundle exec rake assets:precompile \ && rm -rf /app/tmp/cache/assets/ EXPOSE 5000 CMD bin/rails
  19. Pick Base Image FROM ruby RUN apt-get update -qq \

    && apt-get install -y nodejs libpq-dev build-essential COPY . /app WORKDIR /app RUN bundle install RUN bundle exec rake assets:precompile EXPOSE 5000 CMD bin/rails
  20. hub.docker.com/_/ruby/

  21. “Writing a quality Dockerfile is still my users' biggest point

    of friction” - David Dollar, CEO, Convox
  22. Dockerfile Shortcomings • Maintenance / Day 2 Operations (Security) •

    Not App Aware • Composability • Leaky Abstraction
  23. Maintenance / Day 2 Operations FROM ruby:latest RUN mkdir /usr/src/app

    ADD . /usr/src/app/ WORKDIR /usr/src/app/ RUN bundle install CMD ["/usr/src/app/main.rb"]
  24. Composability How do we combine two Docker images?

  25. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY . /app
  26. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app
  27. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM ruby COPY --from=jdk /docker-java-home /docker-java-home COPY --from=jdk /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jdk /usr/share/java/ /usr/share/java/ COPY --from=jdk /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jdk /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jdk /usr/bin/java /usr/bin/java COPY --from=jdk /usr/bin/jps /usr/bin/jps COPY --from=jdk /usr/bin/jshell /usr/bin/jshell COPY --from=jdk /usr/bin/jcmd /usr/bin/jcmd COPY --from=jdk /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=java /app/target /app/target
  28. Composability FROM openjdk:11-jdk as jdk COPY . /app WORKDIR /app

    RUN ./mvnw clean install FROM openjdk:11-jre as jre FROM ruby COPY --from=jre /docker-java-home /docker-java-home COPY --from=jre /usr/lib/jvm/ /usr/lib/jvm/ COPY --from=jre /usr/share/java/ /usr/share/java/ COPY --from=jre /usr/share/ca-certificates-java/ /usr/share/ca-certificates-java/ COPY --from=jre /etc/java-11-openjdk/ /etc/java-11-openjdk/ COPY --from=jre /usr/bin/java /usr/bin/java COPY --from=jre /usr/bin/jps /usr/bin/jps COPY --from=jre /usr/bin/jshell /usr/bin/jshell COPY --from=jre /usr/bin/jcmd /usr/bin/jcmd COPY --from=jre /usr/bin/jar /usr/bin/jar ENV JAVA_HOME /docker-java-home ENV JAVA_VERSION 11.0.1 ENV JAVA_DEBIAN_VERSION 11.0.1+13-3 COPY . /app COPY --from=jdk /app/target /app/target
  29. Composability (Multi-stage Builds) • No environment variables • Doesn’t follow

    symlinks • Only copying FS layers manually ◦ Can’t copy arbitrary layers/files/dirs ▪ COPY --from=0 /n1 /n1 ▪ COPY --from=0 /n2 /n2 ▪ COPY --from=0 /n3 /n3
  30. Leaky Abstraction Poor tool for app developers who just want

    to write code. Authoring a good Dockerfile requires too much knowledge of the underlying mechanisms. Mix of operation and app developer concerns.
  31. Dockerfile Alternatives?

  32. None
  33. $ git push heroku master

  34. Buildpacks: Heroku for Everything Opinionated, app aware, source-centric way to

    build your apps.
  35. Buildpack Overview • bin/detect • bin/compile • bin/release Slug Tarball

    Stack Image ABI Compatibility Guarantee
  36. Ruby Buildpack • Steps ◦ installing Ruby ◦ installing and

    running Bundler to manage gem dependencies ◦ injecting database configuration ◦ compiling Rails assets • Comprehensive Support ◦ 7 years of battle hardened usage ◦ Used in production by millions of apps ◦ supported MRI as old as 1.8.7 to 2.6.0-rc1 (on release day) ◦ Rails 2.x-5.2 ◦ Minimize buildpack upgrade pain/burden
  37. Buildpack Ecosystem • Languages ◦ .NET Core ◦ Elixir ◦

    R • Frontend ◦ create-react-app ◦ Meteor ◦ Jekyll • Tools ◦ NGINX ◦ OpenCV • Off the Shelf Software ◦ Metabase ◦ Spree ◦ Minecraft
  38. Buildpack Ecosystem (Providers)

  39. Cloud Native Buildpacks (2018) Combine the power of buildpacks with

    the benefits of containers. Cloud Native Sandbox Project (CNCF) Incorporate learnings from Pivotal and Heroku. Let developers focus on their app and not piecing together a build pipeline.
  40. Pack - Buildpack CLI application developers to use buildpacks to

    convert code into runnable images • Building app images with build • Updating app images using rebase
  41. Build $ pack build <image name>

  42. None
  43. How does it work? 1. Detect 2. Analyze 3. Build

    4. Export
  44. Detect

  45. Analyze

  46. Build

  47. Export

  48. Image Rebasing $ pack rebase <image name>

  49. Day 2 Operations / Security Patching

  50. Docker Image Manifest • tarballs of layers • configuration for

    container ◦ Command ◦ Entrypoint ◦ Env Vars
  51. Docker Image Manifest { "schemaVersion": 2, "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "config": {

    "mediaType": "application/vnd.docker.container.image.v1+json", "size": 7023, "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7" }, "layers": [ { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 32654, "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 16724, "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 73109, "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736" } ] }
  52. Docker Image Manifest { "schemaVersion": 2, "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "config": {

    "mediaType": "application/vnd.docker.container.image.v1+json", "size": 7023, "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7" }, "layers": [ { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 32654, "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 16724, "digest": "sha256:3c3a4604a545cdc127456d94e421cd355bca5b528f4a9c1905b15da2eb4a4c6b" }, { "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "size": 73109, "digest": "sha256:ec4b8955958665577945c89419d1af06b5f7636b4ac3da7f12184802ad867736" } ] }
  53. Buildpack Image Building • Day 2 Operations • App Aware

    Image Builder • Composability • Higher Level Abstraction
  54. Day 2 Operations • Fast as a feature • Stack

    image updates • Unified build pipelines
  55. App Aware Image Builder • Intentional about docker layers •

    Intelligent about caching • Smart defaults: memory, concurrency, commands
  56. Composability Buildpacks are decoupled and cohesive units

  57. Higher Level Abstraction

  58. meet developers where they are, their app source code

  59. Try Buildpacks today! • Github repo ◦ Specification (WIP): https://github.com/buildpack/spec

    ◦ Lifecycle: https://github.com/buildpack/lifecycle ◦ Pack CLI: https://github.com/buildpack/pack • Slack ◦ https://slack.buildpacks.io/ • Samples ◦ https://github.com/buildpack/samples ◦ https://github.com/heroku/java-buildpack ◦ https://github.com/cloudfoundry/nodejs-cnb ◦ https://github.com/cloudfoundry/npm-cnb ◦ https://github.com/jkutner/python-buildpack
  60. Deep Dive • Thursday @ 2:35pm • Tahoma 5 @

    TCC Joe Kutner @codefiner Terence Lee @hone02