ITT 2019 - Michael Hüttermann - Continuous Delivery/DevOps Live Cooking Show

In this session, we will interactively set up a holistic Continuous Delivery pipeline with the use of DevOps enabler tools, based on Cloud, microservices and container technologies. We’ll package a Java EE web application. Its development versions will be inspected and promoted towards production environment in the Cloud, with Docker. Jenkins 2 with its native delivery pipeline features will serve as the major backbone of our DevOps toolchain gluing together other tools of the overall ecosystem. This session discusses concepts and shows tools in action. Happy cooking!

Transcript

1. Continuous Delivery/DevOps
   Michael Hüttermann
   Freelancing DevOps Consultant
   http://huettermann.net
   https://github.com/michaelhuettermann/sandbox/tree/master/all
   https://git.io/fxoch
   

   View Slide

2. Context
   Disclaimer
   • More than one solution
   • Consider individual requirements and use cases
   • DevOps: Mix of goals, concepts and tools
   • Not an in-depth tool discussion
   • Giving colored zoo of appetizers
   Objectives
   § Learn good practices, derived from real world success stories.
   § Learn how to address common project challenges.
   § Learn DevOps context and building blocks.
   § Learn about tools, and how they can be integrated.
   § Learn about microservices, cloud, and Oracle Container Cloud Service.
   § Motivate and initially prepare to zoom in yourselves, later on. :-)
   @huettermann
   

   View Slide

3. Agenda
   § Setting the stage
   § Deriving pipelines (and the cycle time)
   § Demoing a roundtrip
   @huettermann
   

   View Slide

4. Agenda
   § Setting the stage
   § Deriving pipelines (and the cycle time)
   § Demoing a roundtrip
   @huettermann
   

   View Slide

5. DevOps
   § Holistic /shared goals
   § Holistic /shared processes
   § Holistic / shared tools
   @huettermann
   

   View Slide

6. Cycle Time
   § Spanning different functions (yep, it is holistic)
   § Measuring the time from start to end of a process
   § Creating your own definition helps
   Example: Time from Git Push to availability in production
   § Managing, with pipelines
   § Setting this into context (often a tradeoff)
   @huettermann
   

   View Slide

7. Agenda
   § Setting the stage
   § Deriving pipelines (and the cycle time)
   § Demoing a roundtrip
   @huettermann
   

   View Slide

8. Continuous Delivery
   Start with a value stream map
   

   View Slide

9. Accelerate Cycle Time, Part I
   DevOps: Set of goals, processes, tools
   SCM Build QA UAT
   Build PROD
   Here DevOps becomes often obvious!
   @huettermann
   

   View Slide

10. Accelerate Cycle Time, Part II
    Bottlenecks are normal!
    "a chain is no stronger than its weakest link"
    Build
    Station #1 Build
    Station #2 Build
    Station #3
    1p / hour 5p / hour 10p / hour
    @huettermann
    

    View Slide

11. The Theory of Constraints!
    Build
    Station #1 Build
    Station #2 Build
    Station #3
    1p / hour 5p / hour 10p / hour
    Build
    Station #1 Build
    Station #2 Build
    Station #3
    7p / hour 5p / hour 10p / hour
    before optimizing
    after optimizing
    Accelerate Cycle Time, Part II
    Bottlenecks are normal!
    @huettermann
    bottleneck
    

    View Slide

12. Accelerate Cycle Time, Part III
    Key success factors
    § Form pipelines to be doughnuts, not tubes
    § Glue together existing tools, e.g. based on Jenkins
    § High quality is a must! (Utilize quality gates)
    § Implement high degree of automation1
    § Form functional+technical consistent releases
    § Eliminate local optimization, strive for holistic approaches
    1 Consider the “Pitfalls of Automation”, see Hüttermann, DevOps for Developers (Apress, 2012), pg. 41
    @huettermann
    

    View Slide

13. Continuous Delivery
    Visualize and optimize cycle time
    @huettermann
    

    View Slide

14. Continuous Delivery
    Visualize and optimize cycle time
    @huettermann
    

    View Slide

15. Continuous Delivery
    Visualize and optimize cycle time
    @huettermann
    

    View Slide

16. Dev Build
    Continuous Delivery
    Workflow, and different environments / build types
    Continuous
    Build
    RC Build GA Build
    Dev Dev
    SCM
    Business Business
    Dev
    @huettermann
    

    View Slide

17. Checkout Compile
    Pipeline for Continuous Build
    Run
    Unit Tests
    @huettermann
    

    View Slide

18. Setup
    Checkout
    Release
    Version
    Provision
    Build Env
    Compile &
    Run Unit Tests
    Run
    Integration
    Tests
    Certify
    WAR
    Migrate
    Database
    Source Code
    Inspection
    Distribute
    WAR
    Build Docker
    Image
    Run Docker
    Container
    Binaries
    Inspection
    Distribute
    Docker
    Image
    Certify
    Sanity Check
    Container
    Checkout Compile
    Pipeline for Continuous Build
    Pipeline for Dev versions
    Run
    Unit Tests
    @huettermann
    

    View Slide

19. Setup
    Checkout
    Release
    Version
    Provision
    Build Env
    Compile &
    Run Unit Tests
    Run
    Integration
    Tests
    Certify
    WAR
    Migrate
    Database
    Source Code
    Inspection
    Distribute
    WAR
    Build Docker
    Image
    Run Docker
    Container
    Binaries
    Inspection
    Distribute
    Docker
    Image
    Certify
    Sanity Check
    Container
    Promote
    Checkout Compile
    Pipeline for Continuous Build
    RC
    Pipeline for Dev versions
    Run
    Unit Tests
    cherry pick
    Deploy
    Certify
    Deliverables
    @huettermann
    

    View Slide

20. Certify
    Deliverables
    Setup
    Checkout
    Release
    Version
    Provision
    Build Env
    Compile &
    Run Unit Tests
    Run
    Integration
    Tests
    Certify
    WAR
    Migrate
    Database
    Source Code
    Inspection
    Distribute
    WAR
    Build Docker
    Image
    Run Docker
    Container
    Binaries
    Inspection
    Distribute
    Docker
    Image
    Certify
    Sanity Check
    Container
    Promote
    Promote
    Checkout Compile
    Pipeline for Continuous Build
    RC
    GA
    Pipeline for Dev versions
    Run
    Unit Tests
    cherry pick cherry pick
    Deploy
    Deploy
    Certify
    Deliverables
    @huettermann
    

    View Slide

21. Certify
    Deliverables
    Setup
    Checkout
    Release
    Version
    Provision
    Build Env
    Compile &
    Run Unit Tests
    Run
    Integration
    Tests
    Certify
    WAR
    Migrate
    Database
    Source Code
    Inspection
    Distribute
    WAR
    Build Docker
    Image
    Run Docker
    Container
    Binaries
    Inspection
    Distribute
    Docker
    Image
    Certify
    Sanity Check
    Container
    Promote
    Promote
    Checkout Compile
    Binary repository and distribution management, with JFrog Artifactory, Oracle Cloud
    Containerized infrastructure, with Docker, Kubernetes, JFrog Artifactory, Oracle Cloud
    Cloud-enabled setup, with GitHub, Oracle Cloud, SonarCloud
    Continuous Inspection, with SonarQube (+SonarLint) and Twistlock
    Supporting and cross-cutting tools, including Chef, Selenium 2, Maven, Cargo, Flyway
    Technologies and middleware, such as Java EE, Tomcat, MySQL
    Functional monitoring, with ELK
    Automation engine: Jenkins
    Pipeline for Continuous Build
    RC
    GA
    Pipeline for Dev versions
    Run
    Unit Tests
    cherry pick cherry pick
    Ecosystem
    Deploy
    Deploy
    Oracle Cloud Infrastructure: Container Service Classic, Compute, OCIR
    Certify
    Deliverables
    @huettermann
    

    View Slide

22. Certify
    Deliverables
    Setup
    Checkout
    Release
    Version
    Provision
    Build Env
    Compile &
    Run Unit Tests
    Run
    Integration
    Tests
    Certify
    WAR
    Migrate
    Database
    Source Code
    Inspection
    Distribute
    WAR
    Build Docker
    Image
    Run Docker
    Container
    Binaries
    Inspection
    Distribute
    Docker
    Image
    Certify
    Sanity Check
    Container
    Promote
    Promote
    Checkout Compile
    Binary repository and distribution management, with JFrog Artifactory, Oracle Cloud
    Containerized infrastructure, with Docker, Kubernetes, JFrog Artifactory, Oracle Cloud
    Cloud-enabled setup, with GitHub, Oracle Cloud, SonarCloud
    Continuous Inspection, with SonarQube (+SonarLint) and Twistlock
    Supporting and cross-cutting tools, including Chef, Selenium 2, Maven, Cargo, Flyway
    Technologies and middleware, such as Java EE, Tomcat, MySQL
    Functional monitoring, with ELK
    Automation engine: Jenkins
    Pipeline for Continuous Build
    RC
    GA
    Pipeline for Dev versions
    Run
    Unit Tests
    cherry pick cherry pick
    Ecosystem
    Deploy
    Deploy
    Oracle Cloud Infrastructure: Container Service Classic, Compute, OCIR
    Certify
    Deliverables
    @huettermann
    

    View Slide

23. Security tool #1
    SonarQube sonarqube.org
    § Identifies issues across languages
    § Cloud native, with fine integration points, configurable
    § Categories:
    § Vulnerabilities (security, common weakness
    enumeration CWE)
    § Reliability (bugs)
    § Maintainability (code smells)
    @huettermann
    

    View Slide

24. Security tool #2
    Twistlock twistlock.com
    § Identifies issues, based on primitives, transitively
    § Cloud native, with fine integration points, configurable
    § Categories:
    § Vulnerability management (common vulnerabilities
    and exposures, CVE)
    § Compliance
    § Runtime defense
    @huettermann
    

    View Slide

25. Certify
    Deliverables
    Setup
    Checkout
    Release
    Version
    Provision
    Build Env
    Compile &
    Run Unit Tests
    Run
    Integration
    Tests
    Certify
    WAR
    Migrate
    Database
    Source Code
    Inspection
    Distribute
    WAR
    Build Docker
    Image
    Run Docker
    Container
    Binaries
    Inspection
    Distribute
    Docker
    Image
    Certify
    Sanity Check
    Container
    Promote
    Promote
    Checkout Compile
    Binary repository and distribution management, with JFrog Artifactory, Oracle Cloud
    Containerized infrastructure, with Docker, Kubernetes, JFrog Artifactory, Oracle Cloud
    Cloud-enabled setup, with GitHub, Oracle Cloud, SonarCloud
    Continuous Inspection, with SonarQube (+SonarLint) and Twistlock
    Supporting and cross-cutting tools, including Chef, Selenium 2, Maven, Cargo, Flyway
    Technologies and middleware, such as Java EE, Tomcat, MySQL
    Functional monitoring, with ELK
    Automation engine: Jenkins
    Pipeline for Continuous Build
    RC
    GA
    Pipeline for Dev versions
    Run
    Unit Tests
    cherry pick cherry pick
    Ecosystem
    Deploy
    Deploy
    Oracle Cloud Infrastructure: Container Service Classic, Compute, OCIR
    Certify
    Deliverables
    @huettermann
    1
    2
    

    View Slide

26. Agenda
    § Setting the stage
    § Deriving pipelines (and the cycle time)
    § Demoing a roundtrip
    @huettermann
    

    View Slide

27. View Slide

28. View Slide

29. View Slide

30. Thank you.
    

    View Slide