Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A DevOps Pipeline - BuildStuff Ukraine 2016

A DevOps Pipeline - BuildStuff Ukraine 2016

A guided tour of using Docker and other best-of-breed tools to deliver quality code from developers to production in a fast and scalable process. What does a modern pipeline look like in the context of Docker?

We’ll cover:

the state of the ecosystem
tools of the trade
process overview
key automations
integrations and hosting
deployment, operations and scaling
working with Docker without losing your mind :)
Target audience: new and intermediate engineers working with/interested in Docker.

4fe11dd39e16a687cc15ea45d2fbb829?s=128

Steve Pereira

November 23, 2016
Tweet

More Decks by Steve Pereira

Other Decks in Technology

Transcript

  1. @steveElsewhere PIPELINE A DEVOPS @steveElsewhere

  2. @steveElsewhere ME

  3. @steveElsewhere CONTEXT VALUE

  4. @steveElsewhere WHY DO WE NEED A PIPELINE?

  5. @steveElsewhere CREATING VALUE WHY WOULD WE SHIP 100 TIMES A

    DAY?
  6. @steveElsewhere ITERATION

  7. @steveElsewhere PIPELINE SO, ABOUT THAT @steveElsewhere

  8. @steveElsewhere ‣ NATURAL START AND END ‣ STARTS WITH SOURCE

    CONTROL ‣ INITIATE WITH EACH CHANGE ‣ RUN IN PARALLEL AND/OR SEQUENTIALLY ‣ SUCCESS MEANS CODE IN PRODUCTION PIPELINE CHARACTERISTICS
  9. @steveElsewhere CONTINUOUS DELIVERY CHANGE STABILIZE VALIDATE RELEASE

  10. @steveElsewhere PROCESS CODE TEST BUILD DEPLOY RUN

  11. @steveElsewhere STAGES OF MATURITY DEPLOY DIRECTLY/TEST CONTINUOUS INTEGRATION CONTINUOUS DELIVERY

    CONTINUOUS DEPLOYMENT
  12. @steveElsewhere IMPORTANT? WHY IS IT @steveElsewhere

  13. @steveElsewhere THE WALL @steveElsewhere

  14. @steveElsewhere DOCKER LETS TALK ABOUT

  15. @steveElsewhere CAVEATS LETS TALK ABOUT

  16. @steveElsewhere

  17. @steveElsewhere ADVANTAGES LETS TALK ABOUT

  18. @steveElsewhere LIGHT WEIGHT REUSE CONSISTENCY WHY DOCKER? PUSH/PULL VERSIONING PORTABILITY

  19. @steveElsewhere 1. TRIGGER 2. RUN UNIT TESTS 3. BUILD THE

    IMAGE THAT WE USE TO BUILD OUR APP 4. RUN THE BUILD CONTAINER AND COMPILING OUR APP 5. BUILD THE IMAGE THAT WE RUN AND DEPLOY 6. PUSH THE FINAL IMAGE TO A DOCKER REGISTRY 7. (TRIGGER) 8. PULL AND RUN OVERVIEW
  20. @steveElsewhere SOME
 PREREQUISITES

  21. @steveElsewhere PROVISIONING AND CONFIG

  22. @steveElsewhere LERN YOU SOME CONTAINER TYPES

  23. @steveElsewhere BASE VOLUME BUILD RUN CONTAINER TYPES

  24. @steveElsewhere OS SOURCE REPOS PACKAGES CLEANUP THE BASE CONTAINER

  25. @steveElsewhere MINIMAL BARE OS FILE TOOLING FILES THE VOLUME CONTAINER

  26. @steveElsewhere THE BUILD CONTAINER BASE CONTAINER CODE COPY COMPILE CLEANUP

  27. @steveElsewhere BUILT CONTAINER RUN COMMAND THE RUN CONTAINER

  28. @steveElsewhere CASCADE CHANGE CHANGE BASE REBUILD BUILD REBUILD BASE REBUILD

    RUN RUN
  29. @steveElsewhere QUALITY ASSURANCE

  30. @steveElsewhere BEYOND BASICS

  31. @steveElsewhere QUALITY ASSURANCE CODE COMMIT TEST ENV PRODUCTION

  32. @steveElsewhere QUALITY ASSURANCE LOCAL Unit Mock COMMIT/CI Regression Cross-Device Functional

    Static Analysis Integration TEST ENV Exploratory Acceptance Performance Smoke Security PROD Smoke A / B
  33. @steveElsewhere QUALITY ASSURANCE

  34. @steveElsewhere CI ONSITE ‣ JENKINS ‣ GITLAB CI ‣ GO.CD

    ‣ STRIDER ‣ TEAM FOUNDATION SERVER
  35. @steveElsewhere CI ONLINE ‣ SHIPPABLE ‣ CODESHIP ‣ CIRCLE CI

    ‣ TRAVIS ‣ APPVEYOR
  36. @steveElsewhere BUILD ‣ DOCKER (LOCAL) ‣ PACKER (LOCAL) ‣ DOCKER

    HUB ‣ QUAY.IO ‣ ANY CI TOOL
  37. @steveElsewhere DEPLOYMENT PULL FROM HUB CONFIG MANAGEMENT RUN

  38. @steveElsewhere THE #! CODE CHANGE PUSH TRIGGER TEST PULL TRIGGER

  39. @steveElsewhere THE #! PULL TRIGGER CODE CHANGE PUSH TRIGGER TEST

  40. @steveElsewhere THE #! PULL TRIGGER CODE CHANGE PUSH TRIGGER TEST

  41. @steveElsewhere WHERE WE GO FROM HERE

  42. @steveElsewhere MEASUREMENT ‣ CYCLE TIME (MTTR) ‣ ITS OK TO

    ADD TESTS ‣ FAILURE RATE ‣ TEST TYPE COVERAGE ‣ CONFIG COVERAGE ‣ MANUAL EDGES
  43. @steveElsewhere VALUE STREAM MAP

  44. @steveElsewhere AUTOMATION

  45. @steveElsewhere THE BOTS AGE OF

  46. @steveElsewhere COLLABO- TOMATION CHATOPS

  47. @steveElsewhere VISIBILITY CHATOPS

  48. @steveElsewhere MY ADVICE

  49. @steveElsewhere THE EDGES THINK ABOUT

  50. @steveElsewhere SCHEDULING STORAGE SERVICE - DISCOVERY OTHER CONSIDERATIONS: LOGGING MONITORING

    NETWORKING
  51. @steveElsewhere HEALTHCHECK ROLLBACK GRACEFUL FAILURE OTHER CONSIDERATIONS: SECRETS SCALING CRON

  52. @steveElsewhere CHANGE AS LITTLE AS POSSIBLE GO CRAZY.

  53. @steveElsewhere SPECIFICS AIM FOR

  54. @steveElsewhere THINK ABOUT SECURITY

  55. @steveElsewhere START NOW

  56. @steveElsewhere THE STACK STATFLO.COM CONVOX

  57. @steveElsewhere CIRCLE

  58. @steveElsewhere --- MACHINE: SERVICES: - DOCKER DEPENDENCIES: PRE: - DOCKER

    PULL FEBUILD - PIP INSTALL --UPGRADE AWSCLI TEST: OVERRIDE: - 'MAKE VERIFICATION-ISOLATION' DEPLOYMENT: PRODUCTION: BRANCH: MASTER COMMANDS: - $(AWS ECR GET-LOGIN) - MAKE JS CSS BUNDLE-LEGACY - MAKE META - ./CONVOX LOGIN CONSOLE.CONVOX.COM - ./CONVOX SWITCH STATFLO/PROD - ./CONVOX DEPLOY --WAIT CIRCLE.YML
  59. @steveElsewhere 1. TRIGGER 2. RUN UNIT TESTS 3. BUILD THE

    IMAGE THAT WE USE TO BUILD OUR APP 4. RUN THE BUILD CONTAINER AND COMPILING OUR APP 5. BUILD THE IMAGE THAT WE RUN AND DEPLOY 6. PUSH THE FINAL IMAGE TO A DOCKER REGISTRY 7. (TRIGGER) 8. PULL AND RUN OVERVIEW AS CI PIPELINE
  60. @steveElsewhere PIPELINE: STATFLO.COM - REUSABLE CONTAINERS - AUTOMATED TESTS -

    CONSTANT DEPLOYMENT STEVE@STATFLO.COM
  61. @steveElsewhere CONTINUOUS AWESOME STATFLO.COM STEVE@STATFLO.COM

  62. @steveElsewhere WE’RE HIRING! STATFLO.COM STEVE@STATFLO.COM

  63. @steveElsewhere WE’RE HIRING! STATFLO.COM - DATA - PYTHON - AWS

    STEVE@STATFLO.COM
  64. @steveElsewhere ДЯКУЮ J.MP/TALKRESOURCES