Continuous Delivery in Practice

Transcript

1. CONTINUOUS DELIVERY IN PRACTICE VIVEK PRAHLAD
2. ABOUT ME ▸ Technologist for 19 years ▸ 13 years at ThoughtWorks ▸ Head of Technology at ThoughtWorks Singapore, 2012 - 2015 ▸ Technology strategy and leadership at product companies ▸ Practicing and teaching Continuous Delivery for ~ 10 years ▸ Languages that I find interesting: Clojure and Rust (watch this space)
3. AGENDA ▸ What is Continuous Delivery? ▸ Continuous Delivery in practice ▸ Release patterns ▸ Adopting Continuous Delivery
4. WHAT IS CONTINUOUS DELIVERY?
5. The ability to get changes:
 features, fixes & configuration into the hands of users safely quickly in a sustainable way
6. The ability to get changes:
 features, fixes & configuration into the hands of users safely quickly in a sustainable way
7. The ability to get changes:
 features, fixes & configuration into the hands of users safely quickly in a sustainable way
8. The ability to get changes:
 features, fixes & configuration into the hands of users safely quickly in a sustainable way
9. A SUPPLY CHAIN FOR SOFTWARE DELIVERY
10. CD CHANGES THE ECONOMICS OF SOFTWARE DEVELOPMENT
11. WATER-SCRUM-FALL STUDY+
 APPROVAL DESIGN + PLANNING 1 2 3 ANALYSIS DEVELOPMENT TESTING INTEGRATION UAT QA RELEASE AND OPERATIONS CENTRALISED QA IT OPERATIONS ‘AGILE’ TEAMS PMO + FINANCE The fuzzy front end The last mile
13. THE PRINCIPLES OF CONTINUOUS DELIVERY ▸ Build quality in ▸ Work in small batches ▸ Computers do repetitive tasks, people solve problems ▸ Relentlessly pursue continuous improvement ▸ Everyone is responsible
14. KEY MEASURES ▸ Lead time for changes ▸ Frequency of release ▸ Mean Time to Restore Service https://puppet.com/resources/whitepaper/2014-state-devops-report
15. LEAD TIME AND CYCLE TIME Idea Start
 development Release
 to users LEAD TIME CYCLE TIME
16. FIRMS WITH HIGH PERFORMING IT ORGANISATIONS WERE TWICE AS LIKELY TO ACHIEVE THEIR PROFITABILITY, MARKET SHARE AND PRODUCTIVITY GOALS
17. STRONG IT PERFORMANCE IS A COMPETITIVE ADVANTAGE
18. DEVOPS PRACTICES IMPROVE IT PERFORMANCE
19. THE BUILD PIPELINE BUILD TEST PACKAGE Environment &
 Application Configuration TEST ENV TEST ENV TEST ENV STAGING PRODUCTION Docker Image Machine Image Application installer SOURCE CONTROL
20. A SUPPLY CHAIN FOR SOFTWARE DELIVERY
21. CONTINUOUS INTEGRATION CONTINUOUS DEPLOYMENT CONTINUOUS DELIVERY Integration whenever code changes. Everyone commits to trunk at least once a day Deploy as the last stage of integration Software is always in a deployable state
22. What to build How to build it How to validate it How to run it
23. What to build How to build it How to validate it How to run it => Product => Development => Testing => Operations
24. AGENDA ▸ What is Continuous Delivery? ▸ Continuous Delivery in practice ▸ Release patterns ▸ Adopting Continuous Delivery
25. CONTINUOUS DELIVERY FOR TEAMS
26. CULTURE EATS STRATEGY FOR BREAKFAST Peter Drucker
27. WESTRUM’S MODEL: HOW EFFECTIVELY ORGANISATIONS PROCESS INFORMATION
28. Pathological (power-oriented) Bureaucratic (rule-oriented) Generative 
 (performance-oriented) Low cooperation Modest cooperation High cooperation Messengers shot Messengers neglected Messengers trained Responsibilities shirked Narrow responsibilities Risks are shared Bridging discouraged Bridging tolerated Bridging encouraged Failure leads to scapegoating Failure leads to justice Failure leads to enquiry Novelty crushed Novelty leads to problems Novelty implemented
30. ORGANISATIONS WHICH DESIGN SYSTEMS.. ARE CONSTRAINED TO PRODUCE DESIGNS THAT ARE COPIES OF THE COMMUNICATION STRUCTURES OF THOSE ORGANISATIONS Melvin Conway CONWAY’S LAW
31. UI MIDDLEWARE DATABASE ESB TEAMS SYSTEMS
32. What to build How to build it How to validate it How to run it => Product => Development => Testing => Operations
33. PRODUCT MANAGEMENT
34. INCREMENTAL ITERATIVE https://gojko.net/2007/12/04/waterfall-trap/
35. LEARN BUILD MEASURE IDEAS CODE DATA http://theleanstartup.com/principles
36. STORY TREES
37. USER PROFILES RECOMMENDATIONS SKILLS CERTIFICATIONS JOBS
38. STORY MAPS
39. ACTIVITY TASK SUB-TASK TIME NECESSITY
40. FEATURE TOGGLES
 AND A/B TESTING
41. CONTINUOUS DELIVERY AND DEVELOPMENT
42. KEY GOAL: EVERY COMMIT SHOULD BE DEPLOYABLE
43. Onsite Customer Metaphor Refactoring Simple design Testing Continuous Integration Coding standards Collective Ownership Pair programming Sustainable Pace Planning game Small releases
44. EMPHASIS ON REDUCING COST OF CHANGE
45. KEY GOALS: TESTABILITY AND DEPLOYABILITY
46. THE 12 FACTOR APP CODEBASE DEPENDENCIES CONFIG BACKING SERVICES BUILD, RELEASE, RUN PROCESSES CONCURRENCY DISPOSABILITY PORT BINDING DEV/PROD PARITY LOGS ADMIN PROCESSES https://12factor.net/
47. BRANCHING STRATEGIES
48. BRANCHING STRATEGIES?
49. TRUNK BASED DEVELOPMENT WITH FEATURE TOGGLES
50. DATA MANAGEMENT
51. Onsite Customer Metaphor Refactoring Simple design Testing Continuous Integration Coding standards Collective Ownership Pair programming Sustainable Pace Planning game Small releases
52. THE ‘EXPAND AND CONTRACT’ PATTERN ORDER
 - ADDRESS ORDER
 - ADDRESS CUSTOMER
 - ADDRESS ORDER
 - ADDRESS CUSTOMER
 - ADDRESS CUSTOMER
 - ADDRESS USE ORDER.ADRESS USE ORDER.ADRESS USE CUSTOMER.
 ADDRESS USE CUSTOMER.
 ADDRESS
54. CONTINUOUS DELIVERY AND TESTING
55. FUNCTIONAL ACCEPTANCE INTEGRATION UNIT SHOWCASES USABILITY EXPLORATORY NONFUNCTIONAL 
 ACCEPTANCE /
 QOS Technology facing Business facing Technology facing Critique project Automated Automated Automated Manual Manual / automated
56. THE TEST PYRAMID Unit tests Integration tests UI
 tests FAST SLOW MANY FEW
57. CI AND ROBUST TESTING IS THE FOUNDATION FOR CD
58. CONTINUOUS DELIVERY AND INFRASTRUCTURE
59. INFRASTRUCTURE AS CODE
60. Onsite Customer Metaphor Refactoring Simple design Testing Continuous Integration Coding standards Collective Ownership Pair programming Sustainable Pace Planning game Small releases
61. IMMUTABLE INFRASTRUCTURE
62. IAC MATURITY MODEL
63. Practice Build management & CI Environments & Deployment Release management & compliance Testing Data management Level 3 (Optimising) Continual improvement and optimisation of IaC based on evolving industry standards Continuous improvement of tests at various levels Zero-downtime provisioning of infrastructure
 Ability to automatically roll back Changes Infrastructure is self-healing, self- configurable, and self-optimising Metrics are regularly reviewed Level 2 (Quantitatively managed) All changes tracked in an Application Lifecycle Management (ALM) tool Builds not left broken
 Changes promoted through a consistent path to production Ability to manually roll back changes quickly and efficiently Infrastructure is highly available and fault-tolerant Automated alerting based on active monitoring
 IaC processes and practices are documented and available Level 1 (Consistent) All infrastructure defined as code Industry standard tooling is used to write code declaratively CI server to build, test and public IaC artifacts
 Automated tests run for every check in Provisioned infrastructure the result of an automated delivery pipeline Provisioning is idempotent Immutable infrastructure (no SSHing into boxes) Infrastructure is reliable and performs predictably Metrics calculated automatically, but not regularly reviewed.
 Centralised infrastructure monitoring and logging Level 0 (Repeatable) Infrastructure partly automated using scripts. Automation tooling is ad-hoc Some IaC tests Tests are run locally Provisioning is scripted but executed ad-hoc Patching and upgrades are done through provisioning processes Metrics are defined, but no way to collect and consistently measure Level -1 (Regressive) Nothing stored in version control
 Scripts stored on infrastructure No Continuous Integration server
 No written IaC tests No way to test infrastructure before provisioning Infrastructure is built manually from command line or GUI Existing infrastructure cannot be rebuilt
 Provisioning new infra is slow, inconsistent and painful Existing infrastructure is brittle and unreliable. Patching and upgrades done directly on running infrastructure. Trouble shooting on running infrastructure No defined infrastructure metrics: SLAs, KPIs, CSFs Monitoring and logging done directly on running infrastructure No automated alerting Practice Development Continuous Integration Provisioning and Configuration Management Observability
64. AGENDA ▸ What is Continuous Delivery? ▸ Continuous Delivery in practice ▸ Release patterns ▸ Adopting Continuous Delivery
65. BLUE-GREEN DEPLOYMENT ROUTER
66. DARK LAUNCHING
67. CANARY RELEASES ROUTER
68. A/B TESTING ROUTER
69. AGENDA ▸ What is Continuous Delivery? ▸ Continuous Delivery in practice ▸ Release patterns ▸ Adopting Continuous Delivery
70. ADOPTING CONTINUOUS DELIVERY
71. WHERE DO YOU BEGIN?
72. Practice Build management & CI Environments & Deployment Release management & compliance Testing Data management Configuration management Level 3 (Optimising) Teams meet to discuss integration problems resolve via automation, visibility, feedback All environments managed effectively. Provisioning fully automated. DevOps culture, collaboration to reduce risks, cycle time Production rollbacks rare. Defects found and fixed immediately Release to release feedback loop of database performance & deployment process Zero downtime provisioning of infrastructure. Automatic rollback, self service Level 2 (Quantitatively managed) Build metrics gathered, made visible and acted on. Builds are not left broken. Orchestrated deployments managed. Release and rollback processes tested. Environment and application health proactively managed. Cycle time monitored Quality metrics and trends tracked. Non-functional requirements defined, measured Database upgrades & rollbacks tested with deployment. Database performance monitored, optimised Ability to manually roll back changes quickly and safely Level 1 (Consistent) Automated build and test cycle when a change is committed. Artifacts managed Fully automated, push-button deployment. Same artifact, process for each environment. Change management and approval processes defined, enforced. Automated unit and acceptance tests. Testing part of development. Database changes performed automatically as part of deployment Provisioned infrastructure result of an automated delivery pipeline. Idempotent. Level 0 (Repeatable) Regular automated build and testing. Any build can be recreated from source Automated deploys to some environments. Configuration versioned Painful, infrequent, but reliable releases. Limited traceability from requirements Automated tests written as part of story development Changes to databases done with version controlled scripts. Provisioning scripted but executed ad-hoc Level -1 (Regressive) Manual processes for building software. No management of artifacts and reports Manual deployment. Environment-specific binaries. Manual environment creation Infrequent and unreliable releases Manual testing after development Data migrations unversioned and performed manually Infrastructure built manually, existing infrastructure cannot be rebuilt. Provision painful. Practice Build management & CI Environments & Deployment Release management & compliance Testing Data management Configuration management
73. VALUE STREAM MAPPING
74. IDEA PRODUCT
 BACKLOG SPRINT
 BACKLOG DEVELOP TEST IN PRODUCTION
75. SUMMARY ▸ Continuous Delivery: think about the big picture ▸ Optimise cycle time - use a value stream map ▸ Engineering, product management, testing and ops practices are crucial ▸ Make deployments easy
76. THANK YOU!
77. QUESTIONS?