ITT 2019 - Michael Hüttermann - Continuous Delivery/DevOps Live Cooking Show

In this session, we will interactively set up a holistic Continuous Delivery pipeline with the use of DevOps enabler tools, based on Cloud, microservices and container technologies. We’ll package a Java EE web application. Its development versions will be inspected and promoted towards production environment in the Cloud, with Docker. Jenkins 2 with its native delivery pipeline features will serve as the major backbone of our DevOps toolchain gluing together other tools of the overall ecosystem. This session discusses concepts and shows tools in action. Happy cooking!

Transcript

1. Continuous Delivery/DevOps Michael Hüttermann Freelancing DevOps Consultant http://huettermann.net https://github.com/michaelhuettermann/sandbox/tree/master/all https://git.io/fxoch
2. Context Disclaimer • More than one solution • Consider individual requirements and use cases • DevOps: Mix of goals, concepts and tools • Not an in-depth tool discussion • Giving colored zoo of appetizers Objectives § Learn good practices, derived from real world success stories. § Learn how to address common project challenges. § Learn DevOps context and building blocks. § Learn about tools, and how they can be integrated. § Learn about microservices, cloud, and Oracle Container Cloud Service. § Motivate and initially prepare to zoom in yourselves, later on. :-) @huettermann
3. Agenda § Setting the stage § Deriving pipelines (and the cycle time) § Demoing a roundtrip @huettermann
4. Agenda § Setting the stage § Deriving pipelines (and the cycle time) § Demoing a roundtrip @huettermann
5. DevOps § Holistic /shared goals § Holistic /shared processes § Holistic / shared tools @huettermann
6. Cycle Time § Spanning different functions (yep, it is holistic) § Measuring the time from start to end of a process § Creating your own definition helps Example: Time from Git Push to availability in production § Managing, with pipelines § Setting this into context (often a tradeoff) @huettermann
7. Agenda § Setting the stage § Deriving pipelines (and the cycle time) § Demoing a roundtrip @huettermann
8. Continuous Delivery Start with a value stream map
9. Accelerate Cycle Time, Part I DevOps: Set of goals, processes, tools SCM Build QA UAT Build PROD Here DevOps becomes often obvious! @huettermann
10. Accelerate Cycle Time, Part II Bottlenecks are normal! "a chain is no stronger than its weakest link" Build Station #1 Build Station #2 Build Station #3 1p / hour 5p / hour 10p / hour @huettermann
11. The Theory of Constraints! Build Station #1 Build Station #2 Build Station #3 1p / hour 5p / hour 10p / hour Build Station #1 Build Station #2 Build Station #3 7p / hour 5p / hour 10p / hour before optimizing after optimizing Accelerate Cycle Time, Part II Bottlenecks are normal! @huettermann bottleneck
12. Accelerate Cycle Time, Part III Key success factors § Form pipelines to be doughnuts, not tubes § Glue together existing tools, e.g. based on Jenkins § High quality is a must! (Utilize quality gates) § Implement high degree of automation1 § Form functional+technical consistent releases § Eliminate local optimization, strive for holistic approaches 1 Consider the “Pitfalls of Automation”, see Hüttermann, DevOps for Developers (Apress, 2012), pg. 41 @huettermann
13. Continuous Delivery Visualize and optimize cycle time @huettermann
14. Continuous Delivery Visualize and optimize cycle time @huettermann
15. Continuous Delivery Visualize and optimize cycle time @huettermann
16. Dev Build Continuous Delivery Workflow, and different environments / build types Continuous Build RC Build GA Build Dev Dev SCM Business Business Dev @huettermann
17. Checkout Compile Pipeline for Continuous Build Run Unit Tests @huettermann
18. Setup Checkout Release Version Provision Build Env Compile & Run Unit Tests Run Integration Tests Certify WAR Migrate Database Source Code Inspection Distribute WAR Build Docker Image Run Docker Container Binaries Inspection Distribute Docker Image Certify Sanity Check Container Checkout Compile Pipeline for Continuous Build Pipeline for Dev versions Run Unit Tests @huettermann
19. Setup Checkout Release Version Provision Build Env Compile & Run Unit Tests Run Integration Tests Certify WAR Migrate Database Source Code Inspection Distribute WAR Build Docker Image Run Docker Container Binaries Inspection Distribute Docker Image Certify Sanity Check Container Promote Checkout Compile Pipeline for Continuous Build RC Pipeline for Dev versions Run Unit Tests cherry pick Deploy Certify Deliverables @huettermann
20. Certify Deliverables Setup Checkout Release Version Provision Build Env Compile & Run Unit Tests Run Integration Tests Certify WAR Migrate Database Source Code Inspection Distribute WAR Build Docker Image Run Docker Container Binaries Inspection Distribute Docker Image Certify Sanity Check Container Promote Promote Checkout Compile Pipeline for Continuous Build RC GA Pipeline for Dev versions Run Unit Tests cherry pick cherry pick Deploy Deploy Certify Deliverables @huettermann
21. Certify Deliverables Setup Checkout Release Version Provision Build Env Compile & Run Unit Tests Run Integration Tests Certify WAR Migrate Database Source Code Inspection Distribute WAR Build Docker Image Run Docker Container Binaries Inspection Distribute Docker Image Certify Sanity Check Container Promote Promote Checkout Compile Binary repository and distribution management, with JFrog Artifactory, Oracle Cloud Containerized infrastructure, with Docker, Kubernetes, JFrog Artifactory, Oracle Cloud Cloud-enabled setup, with GitHub, Oracle Cloud, SonarCloud Continuous Inspection, with SonarQube (+SonarLint) and Twistlock Supporting and cross-cutting tools, including Chef, Selenium 2, Maven, Cargo, Flyway Technologies and middleware, such as Java EE, Tomcat, MySQL Functional monitoring, with ELK Automation engine: Jenkins Pipeline for Continuous Build RC GA Pipeline for Dev versions Run Unit Tests cherry pick cherry pick Ecosystem Deploy Deploy Oracle Cloud Infrastructure: Container Service Classic, Compute, OCIR Certify Deliverables @huettermann
22. Certify Deliverables Setup Checkout Release Version Provision Build Env Compile & Run Unit Tests Run Integration Tests Certify WAR Migrate Database Source Code Inspection Distribute WAR Build Docker Image Run Docker Container Binaries Inspection Distribute Docker Image Certify Sanity Check Container Promote Promote Checkout Compile Binary repository and distribution management, with JFrog Artifactory, Oracle Cloud Containerized infrastructure, with Docker, Kubernetes, JFrog Artifactory, Oracle Cloud Cloud-enabled setup, with GitHub, Oracle Cloud, SonarCloud Continuous Inspection, with SonarQube (+SonarLint) and Twistlock Supporting and cross-cutting tools, including Chef, Selenium 2, Maven, Cargo, Flyway Technologies and middleware, such as Java EE, Tomcat, MySQL Functional monitoring, with ELK Automation engine: Jenkins Pipeline for Continuous Build RC GA Pipeline for Dev versions Run Unit Tests cherry pick cherry pick Ecosystem Deploy Deploy Oracle Cloud Infrastructure: Container Service Classic, Compute, OCIR Certify Deliverables @huettermann
23. Security tool #1 SonarQube sonarqube.org § Identifies issues across languages § Cloud native, with fine integration points, configurable § Categories: § Vulnerabilities (security, common weakness enumeration CWE) § Reliability (bugs) § Maintainability (code smells) @huettermann
24. Security tool #2 Twistlock twistlock.com § Identifies issues, based on primitives, transitively § Cloud native, with fine integration points, configurable § Categories: § Vulnerability management (common vulnerabilities and exposures, CVE) § Compliance § Runtime defense @huettermann
25. Certify Deliverables Setup Checkout Release Version Provision Build Env Compile & Run Unit Tests Run Integration Tests Certify WAR Migrate Database Source Code Inspection Distribute WAR Build Docker Image Run Docker Container Binaries Inspection Distribute Docker Image Certify Sanity Check Container Promote Promote Checkout Compile Binary repository and distribution management, with JFrog Artifactory, Oracle Cloud Containerized infrastructure, with Docker, Kubernetes, JFrog Artifactory, Oracle Cloud Cloud-enabled setup, with GitHub, Oracle Cloud, SonarCloud Continuous Inspection, with SonarQube (+SonarLint) and Twistlock Supporting and cross-cutting tools, including Chef, Selenium 2, Maven, Cargo, Flyway Technologies and middleware, such as Java EE, Tomcat, MySQL Functional monitoring, with ELK Automation engine: Jenkins Pipeline for Continuous Build RC GA Pipeline for Dev versions Run Unit Tests cherry pick cherry pick Ecosystem Deploy Deploy Oracle Cloud Infrastructure: Container Service Classic, Compute, OCIR Certify Deliverables @huettermann 1 2
26. Agenda § Setting the stage § Deriving pipelines (and the cycle time) § Demoing a roundtrip @huettermann
30. Thank you.