Continuous Delivery Principles

Slide 1

Slide 1 text

Continuous Delivery Foundations

Slide 2

Slide 2 text

Continuous Delivery – Agenda Configuration Management Version Control Dependency Management Continuous Integration Deployment Pipeline Scripting Managing Infrastructure Managing Data

Slide 3

Slide 3 text

Continuous Delivery – Today’s Agenda Introduction Configuration Management Version Control Continuous Integration Testing Strategies The Deployment Pipeline

Slide 4

Slide 4 text

Introduction

Slide 5

Slide 5 text

Continuous Delivery - What is it all about? “ How quickly can we publish a single line of code change to production ? ” Most software development methodologies focus primarily on requirement management and its impact on the development effort. BUT

Slide 6

Slide 6 text

Continuous Delivery - Definition • The Lean Movement  Rapid delivery of high-quality products, focusing on the removal of waste and the reduction of cost • Continuous Deployment  Continuous deployment is deploying every change that passes the automated tests and some optional QA tests. • Continuous Delivery  Continuous deployment on production  Continuous delivery is about putting the release schedule in the hands of the business, not in the hands of IT In the world of continuous delivery, developers aren’t done with a feature when they hand some code over to testers, or when the feature is “QA passed”

Slide 7

Slide 7 text

Continuous Delivery - Principles Principles of Software Delivery • Repeatable & Reliable Release Process • Automate almost everything • Keep everything in Version Control • Bring the Pain foreword • Build Quality-In • Done means released • Bring down the Barriers = Everyone has a global overview = Everyone is responsible • Continuous Improvement = Study what has gone badly and discuss it

Slide 8

Slide 8 text

Continuous Delivery – How to do it? It is a development practice as well as a pattern Commit Stage Automated Acceptance Testing Exploratory Testing Automated Performance Testing Deployment “ Continuous delivery builds on a simple foundation, continuous integration, automation, and version control. But in order to get it to work you must version control and automate everything. It's that simple! “ Continuous Delivery uses a Continuous Integration server to drive the Deployment Pipeline

Slide 9

Slide 9 text

Continuous Delivery – How was your Release Day ? Release is a manually intensive process. 1. Dev, IS or Operations team craft the environments that host the software. 2. Third-party software is installed 3. The software artifacts of the application are copied to the production host environments. 4. Configuration information is copied or created through the admin consoles of web servers, applications servers, or other third-party components of the system. 5. Reference data is copied. 6. Finally the application is started Many things can go wrong in the process!

Slide 10

Slide 10 text

Continuous Delivery - Common Release Anti-Patterns • Deploy Software Manually There should be two tasks for a human being to perform to deploy software into a development, test, or production environment: to pick the version and environment and to press the “deploy” button • Deploy to a Production-like Environment at the end Integrate the testing, deployment, and release activities into the development process • Manual Configuration Management of Production Environments All aspects of each of our testing, staging, and production environments, specifically the configuration of any third-party elements of our system, should be applied from version control through an automated process

Slide 11

Slide 11 text

Continuous Delivery - A Better Release Day The Goal should still always be to deliver high-quality, valuable software in an efficient, fast, and reliable manner. Low cycle and high quality can be achieved by performing releases : 1. Frequently 2. In an automated way And then getting feedback and acting on it. We are going to see some exceptional release strategies such as: • Canary releasing • Blue-green deployment

Slide 12

Slide 12 text

Continuous Delivery - Benefits • A Pull-System that empowers teams It allows testers, operations or support personnel to self-service the version of the application they want into the environment of their choice. • Reducing Errors Specifically those introduced into production by poor configuration management • Lowering Stress Better working hours, less fear of the unexpected • Deployment Flexibility For example we could be running our multi-component enterprise software on a single laptop • Practice makes Perfect Integrate the deployment process into the development process. Use the same deployment approach whatever the deployment target

Slide 13

Slide 13 text

Configuration Management

Slide 14

Slide 14 text

Continuous Delivery – What makes a good CM? Having a good Configuration Management means answering Yes to the following: • Can I exactly reproduce any of my environments?  Operating System  Patch Level  Network Configuration  Software Stack  Applications & Configuration • Can I easily make an incremental change and deploy it anywhere ? • Can I easily track changes to any of the above items? • Can I satisfy all of the compliance regulations that I am subject to? • Is it easy for every member of the team(s) ?

Slide 15

Slide 15 text

How important Flexibility is? Configurable software may enhance flexibility but needs to be managed very carefully. There are no inherent protections against erroneous configuration. The two most common pitfalls a dev-team can fall into are: • Analysis paralysis. Too much thought and solve nothing at the end. • Oversized complexity. Custom development could require less effort then. Continuous Delivery – Don’t overdo it The desire to achieve flexibility may lead to the common anti-pattern of “ultimate configurability” Focus on delivering high-value functionality. Add configuration later but plan it early on

Slide 16

Slide 16 text

Continuous Delivery – When is it applied ? Application = Binaries + Data + Configuration Configuration information can be used to change the behavior of software at : • Build scripts can pull configuration in and incorporate it into your binaries at build time. • Packaging software can inject configuration at packaging time, such as when creating assemblies, wars, ears, or ruby gems. • Deployment scripts or installers can fetch the necessary information and pass it to your application at deployment time. • The application itself can fetch configuration at startup time or run time. Build time Packaging time Deploy time Startup or Run time

Slide 17

Slide 17 text

Continuous Delivery – Questions that define it Three questions 1. How do we represent our configuration information? 2. How do our deployment scripts access it? 3. How does it vary between environments, applications, and versions of applications?

Slide 18

Slide 18 text

Continuous Delivery – CM Model & Representation Model • As a Set of Tuples • 3-D = Application, Version, Environment Representation • Key Value Pairs (Properties Files) • XML • YAML (Ruby) • Dynamic Languages (Groovy, Javascript etc) • Version 1.0 of our Reporting application will have a set of tuples different from 2.2 • Or from version 1.0 of our portfolio management application • The database server used by the application in UAT will typically be different from that used in production and may even vary between developer machines • Same for external integration points—an update service used by your application will be different when running integration tests than from production.

Slide 19

Slide 19 text

Continuous Delivery – Where do we store Conf/on? Database Registries / Directory Services Filesystem Version Control • Make sure to keep the history of changes to configuration for the purposes of audit and rollback • With version control you get the history of your configuration over time for free • Keep the configuration options in the same repository as the source code • Keep the actual configuration information in a separate repository* Note that the place where you store configuration is not the same as the mechanism by which your application accesses it

Slide 20

Slide 20 text

Continuous Delivery – How do we access Conf/on? Central Service • The most efficient way • Common for every application and environment Filesystem • Cross-platform and supported in every language • Problem of keeping in sync Keep the details secret using a Façade • getThisProperty() • setThisProperty() 1. Fetch configuration from a centralized repository such as a RDBMS, LDAP, or a web service. 2. Pass the environment name to your deployment scripts (via a property, command-line switch, or environment variable). 3. Scripts fetch the appropriate configuration make it available to the application, perhaps as a file on the filesystem.

Slide 21

Slide 21 text

• The configuration system should be able to provide different values based on the application, its version and the environment it is being deployed to. • Use clear naming conventions • Use the DRY principle but also ensure that your configuration information is modular and encapsulated. (This is not so easy). • Test your configuration. It‘s part of the application. Test references to external services are good at deployment time.  Run smoke-tests during deployment or right after installing. Failure should cancel deployment or stop the application Continuous Delivery - Application Conf/on Principles (2/2)

Slide 22

Slide 22 text

Continuous Delivery - Application Conf/on Principles (1/2) • Be simple and minimal • Consider where during the lifecycle to inject a specific option. • Build script should generate a configuration options report, otherwise should be documented in a wiki etc. • Keep available configuration options in the same repository as its source code, but the values somewhere else. • Configuration should always be performed by automated processes using values taken from your configuration repository.

Slide 23

Slide 23 text

Continuous Delivery - Managing Environment Conf/on • Don’t deal with it on ad-hoc basis • Define & manage the change process • Check the op system’s configuration into version control. • Being able to reproduce your environments is essential  No risk of significant downtime due to scattered random pieces of infrastructure.  Predictable amount of time for deploying/fixing.  Easily create exact copies of your environments. • Environment configuration includes the following:  Operating System, Version, Patch Level, Settings  Additional Software Packages, Version, Settings  Network Topology  External Services, Databases

Slide 24

Slide 24 text

Continuous Delivery - Configuration Tools Application Configuration • Escape • LDAP • DNS Configuration Monitoring • Nagios • OpenNMS • HP OpenView Environment Management • Puppet • CfEngine • Chef

Slide 25

Slide 25 text

Continuous Delivery – Puppet / CfEngine / Virtualization • Patches • Files • User Access • Stored in VCS Declarative Definitions • Pull latest configuration from VCS • Update Operating System • Update Software Local Agents • Instead of creating a new environment from scratch using an automated process, you can simply take a copy of each box in your environment and store it as a baseline. Virtualization

Slide 26

Slide 26 text

Continuous Delivery – Puppet

Slide 27

Slide 27 text

Continuous Delivery – ESCAPE Tool It provides a RESTful interface for both the setting and getting of environment configuration. • GET requests will retrieve configuration • POST/PUT requests will create new entries or update them. URL - http://escape Value returned / User interface. No API available here. /environments/ A list of all the available environments. environments/production/ A list of all the applications in the "production" environment. environments/production/mywebapp All keys and values for the "mywebapp" application in the "production" environment. environments/production/mywebapp/thiskey The value of "thiskey" for the "mywebapp" application in the "production" environment.

Slide 28

Slide 28 text

Continuous Delivery – ESCAPE Tool MYENV = $1 export TARGET_HOSTS=`wget -q http://escape/environments/$MYENV/myapplication/target_hosts --output-document=-` for host in $TARGET_HOSTS do run_installation_on $host done Bash Script *There also ready to use clients for Java, Python & Ruby A common usage example

Slide 29

Slide 29 text

Version Control SCCS – RCS – CVS – SVN Mercurial, GIT

Slide 30

Slide 30 text

Continuous Delivery – Why Version Control? 1. It retains, and provides access to, every version of every file that has ever been stored in it 2. It allows teams that may be distributed across space and time to collaborate 3. Supports metadata And helps answering to the following questions : 1. What constitutes a particular version of my software. 2. How can I reproduce a particular state of the software’s binaries & configuration 3. What was done when, by whom, and for what reason

Slide 31

Slide 31 text

Continuous Delivery – Keep in Version Control What should we keep in Version Control ? Everything • Source Code • Tests • Database Scripts • Build and Deployment Scripts • Documentation • Libraries* • Configuration Files • Binary Images of Application Servers • Compilers • Virtual Machines … and everything else needed to recreate a test or production Environment

Slide 32

Slide 32 text

Continuous Delivery – Check-In When and where should we check in? • On trunk • Regularly Points of Interest • Checking in is a form of publication. Would you publish anything? • Checking in regularly means small merges • Use meaningful comments • Use commit messages • Run unit-tests locally before checking in • Develop new features incrementally!!

Slide 33

Slide 33 text

Continuous Delivery - Branching To Branch or Not to Branch ? Well, almost NO • It is antithetical to continuous integration as it defers it • The more branches the harder the merge. You actually make it harder by postponing it. • Automated merging tools don’t solve semantic problems • You cannot Refactor the codebase on a branch as you will make merging too difficult. • If you have to, create long-lived branches only on release. • It’s better to defer the problem of merging branches into integrating the loose-coupled components of a well designed application 3 Special Branching Strategies: • Branch for Release • Branch by Feature • Branch by Team

Slide 34

Slide 34 text

Continuous Delivery – A common Release Features Development Bug Fixing Release 2.3 But the aim of continuous delivery is for the application to always be in a releasable state This process generally results in weeks or months between releases Hotfixes TRUNK

Slide 35

Slide 35 text

Continuous Delivery – Develop on mainline The only strategy which enables you to perform continuous integration. How can you manage if you check every change in to mainline? 1. Embrace feature hiding which is not easy but pays off rewarding you with a zero integration time. 2. Do incremental development 3. Use good componentization of your software 4. Use branch by abstraction for large-scale changes • All code is continuously integrated • Developers pick up each others changes immediately • Avoid “merge hell” and “integration hell” at the end

Slide 36

Slide 36 text

Continuous Delivery – Instead of Branching How is it possible to have everybody working on mainline, and still keep your application in a releasable state at all times ? • Put in the new features, but make them inaccessible to users • Turn access to them on and off by means of configuration settings* • Dependencies and integration phases do not need to be introduced into the project plan Hide new functionality until it is finished • It is often tempting, when making large changes, to branch the source code and make the change on the branch • Solve the problem of keeping the application working as you go along, preventing pain at the end • Analysis plays an important part as the thought process that goes into it is the same thought process used to break a requirement down into smaller tasks Make All Changes Incrementally

Slide 37

Slide 37 text

Continuous Delivery – Instead of Branching • An abstraction layer is created over the piece to be changed • A new implementation is then created in parallel with the existing implementation, and then when it is complete, the original implementation and (optionally) the abstraction layer are removed • Use a configuration option that can be modified at deploy time or even run time to choose implementation • Dependency Injection makes it easier • It is better not to create an abstraction API up front at the start of a project. Create your first implementation, then a second one, and factor out the API afterwards Use branch by abstraction for large-scale changes • … that change at different rates Use components to decouple parts … When incremental changes are too hard to make

Slide 38

Slide 38 text

Branch By Feature • For large teams to work simultaneously on features while … • Keeping mainline always releasable • Every story or feature is developed on a separate branch • Only after a story is accepted by testers, it is merged to mainline • It makes version control history more semantically rich • Long lived feature branches is a major anti-pattern Many developers don’t like to have their work exposed and publicly available until they are completely done. Continuous Delivery – Branching Exceptions There are conventions though for this to work well • Any changes from mainline must be merged onto every branch daily • Branches must be short-lived, never more than an iteration • Number of active branches = Number of stories in play • Testers should accept stories before they are merged • Refactorings should be merged immediately

Slide 39

Slide 39 text

Branch By Team • Mainline is always releasable while every team works on separate branch • Merge into trunk only when branch is stable • Any merge of any branch should be immediately pulled from all other branches • Every branch has its own owner & policy Although trunk is kept in releasable state, this strategy has some serious drawbacks. • Every branch has its own deployment pipeline • Scope of work becomes the branch and not a smaller item • Less branches than BbF, quicker diversions Continuous Delivery – Branching Exceptions

Slide 40

Slide 40 text

Continuous Delivery – Branching Exceptions Branch For Release • Features are always developed on mainline • A branch is created when your code is feature-complete for a particular release • Only fixes for critical defects are committed on branches, and they are merged into mainline immediately • When you perform an actual release, this branch is optionally tagged • Always branch from the mainline Some cons are that it doesn’t work very well for really large (monolithic) projects and it is not suitable for frequent release strategies.

Slide 41

Slide 41 text

Continuous Delivery – Distributed Version Control

Slide 42

Slide 42 text

Continuous Delivery – Distributed Version Control

Slide 43

Slide 43 text

Continuous Delivery – Distributed Version Control

Slide 44

Slide 44 text

Continuous Delivery – Branching with DVCS Develop F6 F7 F8 F9 F10 F4 F5 F3 F2 Master Initial production version F1 Next production release Release 2 Next production release Release 3 Next production release Release 1 Release branches ! Fix! Hot-fix branches Feature branches

Slide 45

Slide 45 text

Continuous Delivery – Components & Modules Third Party Libraries • Always in binary form except for interpreted languages • Never changed by the development team* • Change infrequently • Could be Version Controlled • Best practice mandates that they are persisted in a local repository Components & Modules • Monolithic projects are a bad practice for large sized applications • When project is split into several modules, you have to decide if they are going to be different build/deployment pipelines for each • It is safer from a continuous delivery POV to use them as binary dependencies, on the other hand it makes it harder to reproduce the entire build process on a developers workstation. • Using binary dependencies can make it hard to track back a breakage to the source code change that caused it, but a good CI server can help with that.

Slide 46

Slide 46 text

Continuous Integration

Slide 47

Slide 47 text

Continuous Delivery - Continuous Integration Without continuous integration, your software is broken until somebody proves it works, usually during a testing or integration stage With continuous integration, your software is proven to work with every new change—and you know the moment it breaks and can fix it immediately

Slide 48

Slide 48 text

Continuous Delivery - Continuous Integration What you need before you start 1. Version Control Code, tests, database scripts, build and deployment scripts, and anything else needed to create, install, run, and test the application 2. Automated Build Treated as the codebase. It must be possible for either a person or a computer to run the build, test, and deployment process in an automated fashion via the command line 3. Agreement of the Team Apart from the various tools, CI is a practice and requires commitment and discipline from the development team. The highest priority task on the project is to fix any change that breaks the application

Slide 49

Slide 49 text

Continuous Delivery - Continuous Integration

Slide 50

Slide 50 text

Continuous Delivery - Continuous Integration Usage Basic CI Usage 1. Check to see if the build is running. If so, wait for it to finish. No need for this if CI supports concurrent builds. 2. Once it has finished successfully , update to the successful version 3. Build and Test on developer machine or use CI personal build if available 4. If local build passes, check in. 5. Wait for the CI tool to run the build with your changes 6. If it fails, stop everything and fix the problem on the developer machine, then go to step 3. 7. If the build passes, move on to your next task

Slide 51

Slide 51 text

Continuous Delivery - Continuous Integration Requirements Requirements for Continuous Integration (1/2) • Check in Regularly to mainline  Makes changes smaller  Always have a recent version to revert to  Enforces Refactoring discipline  Minimizes conflicts  Allows devs to be more explorative as it makes discarding easier  Provides the necessary breaks to stretch your muscles • Create a comprehensive Automated Test Suite  Unit Tests (should run between 1-5 mins)  Component Tests (may hit databases, services)  Acceptance Tests in a production-like environment (could take a day)

Slide 52

Slide 52 text

Continuous Delivery - Continuous Integration Requirements Requirements for Continuous Integration (2/2) • Keep the Build & Test Short! *Commit Stage of the deployment pipeline  So that people keep doing full builds  So that you won’t have multiple commits between builds and know which commit broke the build  So that you promote frequent check ins  Include a quick smoke test • Carefully manage your Development Workspace  So that starting from the last “good-known” version  you are able to run the build  … execute the automated tests  … and deploy in an environment under your control  using the same automated processes used in the CI environment

Slide 53

Slide 53 text

Continuous Delivery - Continuous Integration Practices Practices for Continuous Integration (1/3) • Don’t check In on a Broken Build • Always run all commit tests locally prior committing or use the CI server’s personal build or pre-tested commit features • Wait for Commit Tests to Pass before moving on • Never go home on a broken build • Be prepared to Revert to a previous version • Time-Box Fixing before Reverting

Slide 54

Slide 54 text

Practices for Continuous Integration (2/3) • When a bug is identified, instead of fixing the bug, write a test to expose it first, then fix it • Take Responsibility for All Breakages That Result from Your Changes • Don’t Comment Out Failing Tests • As developers embrace Test Driven Development • Refactoring is a cornerstone of effective software development and CI makes it safe and easy • Fail a build for Architectural Breaches • Failing the Build for Slow Tests Continuous Delivery - Continuous Integration Practices

Slide 55

Slide 55 text

Practices for Continuous Integration (3/3) • Fail a build for Architectural Breaches ? • Fail the Build for Slow Tests ? • Fail the Build for Warnings and Code Style Breaches ? What’s “ratcheting”? • OMG! The Nagging Is Worth It after All. Continuous Delivery - Continuous Integration Practices

Slide 56

Slide 56 text

Testing Strategy

Slide 57

Slide 57 text

Continuous Delivery - Testing Strategy “Build quality into the product in the first place” In the ideal project, testers collaborate with devs & users to write automated tests from the start of the project. These tests are written before devs start work on the features that they test. In order to achieve this we need to define our Testing Strategy

Slide 58

Slide 58 text

Continuous Delivery - Testing Strategy The design of a testing strategy is primarily a process of identifying project risks and deciding what actions to take to mitigate them. Major positive effects of a good testing strategy • Fewer Bugs • Reduced Support Costs • Improved Reputation • Constraints enforce good development practices • Provides the most complete and up-to-date form of documentation

Slide 59

Slide 59 text

Continuous Delivery - Types Of Tests

Slide 60

Slide 60 text

Continuous Delivery – Functional Acceptance Tests Developers : How do I know when I am done? Users : Did I get what I wanted? • When the acceptance tests pass, whatever requirements or stories they are testing can be said to be complete. • In an ideal project, users would write the functional acceptance tests. • Modern automated functional testing tools like Cucumber, JBehave, Concordion and Twist can bring devs and users closer. • Happy, Alternative and Sad paths. Acceptance tests should be run in production-like systems

Slide 61

Slide 61 text

What to test? • The most important automated test to write is the main happy path test used individually by developers as smoke tests. • When the application is stable and time is something you have, alternate paths should be your priority. • If your application is buggy and crashes often, strategic application of sad path testing Continuous Delivery – Functional Acceptance Tests How much to test automatically ? • More than 80% should be the rule of thumb • What about regression tests?

Slide 62

Slide 62 text

Some valuable gains • Reduced workload on testers. • Testers concentrate on exploratory testing instead of boring repetitive tasks. • A powerful regression test suite exists within. • It is possible to autogenerate requirements documentation from your tests Automated acceptance tests can be costly to maintain. However, by following good practices and using appropriate tools, it is possible to dramatically reduce the cost of creating and maintaining automated acceptance tests to the point where the benefits clearly exceed the costs Continuous Delivery – Functional Acceptance Tests

Slide 63

Slide 63 text

Continuous Delivery – Non-Functional Acceptance Tests “Nonfunctional tests, test all the qualities of a system other than its functionality, such as capacity, availability, security, etc.” • Nonfunctional acceptance criteria should be specified as part of the application’s requirements in exactly the same way as functional acceptance criteria. • Different Tools, special environments. • Usually deferred implementation and run towards the end of the deployment pipeline. • Project time allocated to the research and implementation of Nonfunctional Tests is an investment for mission critical projects.

Slide 64

Slide 64 text

Continuous Delivery – Technology-Facing Tests that… • Unit-Tests A particular piece of the code in isolation, a major part of the regression suite. Doesn’t touch the database! • Component Tests, Testing larger clusters of functionality. Perform more I/O, talking to databases, the filesystem, or other systems. • Deployment Tests Performed whenever the application is deployed, checking that it is correctly installed, correctly configured and responding. Written and maintained exclusively by developers When a bug is identified, instead of fixing the bug, write a test to expose it first, then fix it … support the Development Process

Slide 65

Slide 65 text

Continuous Delivery – Business-Facing Tests that … • Showcases Agile teams perform showcases to users at the end of every iteration to demonstrate the new functionality that they have delivered • Exploratory Testing The tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests. • Usability Testing how easy it is for users to accomplish their goals with your software • Beta Testing Give your application to real users using beta testing programs. Manual tests that verify that the application will in fact deliver to the users the value they are expecting Check that the project meets the correct specifications. … critique the project

Slide 66

Slide 66 text

• Dummy objects Are passed around but never actually used. Usually they are just used to fill parameter lists. • Fake objects Actually have working implementations, but usually take some shortcut that makes them not suitable for production. • Stubs Provide canned answers to the calls made during the test, usually not responding at all to anything outside what’s programmed in for the test. • Spies Are stubs that also record some information based on how they were called. • Mocks Are preprogrammed with expectations that form a specification of the calls they are expected to receive. Continuous Delivery - Test Doubles

Slide 67

Slide 67 text

Continuous Delivery – Integration Tests “ Incorporate activities concerning integration into your release plan ” • Can be written in the same way as you write normal acceptance tests • Run in two contexts, the real external systems/replicas and the test stubs. • Isolate access to the external system in the testing environment with a firewall • Have a configuration setting in the application that makes it talk to a simulated version of the external system • Reused as smoke tests during deployment of the system into production • Reused as diagnostics to monitor the production system • Replicate not only the expected responses to service calls

Slide 68

Slide 68 text

Deployment Pipeline

Slide 69

Slide 69 text

Continuous Delivery – Definition in Concept Definition Deployment Pipeline models the process from check-in to release, which forms a part of the process of getting a feature from the mind of a customer or user into their hands. The entire process as a value stream map

Slide 70

Slide 70 text

Commit Stage Automated Acceptance Testing Exploratory Testing Automated Performance Testing Deployment Continuous Delivery - Anatomy The end-to-end automation, when repeated many times , it uncovered some common abstractions/patterns. Build Deploy Test Release

Slide 71

Slide 71 text

Continuous Delivery - Deployment Pipeline Stages • Asserts that the system works at the technical level. It compiles, passes a suite of (primarily unit-level) automated tests, and runs code analysis. Commit Stage • Asserts that the system works at the functional and nonfunctional level, that behaviorally it meets the needs of its users and the specifications of the customer Automated Acceptance Stage • Asserts that the system is usable and fulfills its requirements, detect any defects not caught by automated tests, and verify that it provides value to its users Manual Tests Stage • Delivers the system to users, either as packaged software or by deploying it into a production or staging environment Release stage

Slide 72

Slide 72 text

Continuous Delivery – The Goal of adopting it The Goal? An end-to-end pull system through which everyone has access to the things they need and visibility into the release process to improve feedback. Testing teams deploy builds into testing environments themselves, at the push of a button Operations can deploy builds into staging and production environments at the push of a button Developers can see which builds have been through which stages in the release process, and what problems were found Managers can watch such key metrics as cycle time, throughput, and code quality

Slide 73

Slide 73 text

Continuous Delivery – A basic deployment pipeline

Slide 74

Slide 74 text

Continuous Delivery – Tradeoffs •Exploratory •User Experience •Look & Feel User acceptance testing •Performance •Security •Auditing Capacity testing Commit Stage • Compile • Unit Test • Analysis • Installers Functional Acceptance Tests • Component Tests • Integration Tests • Behavioral Tests Production Increase confidence in build’s production readiness Environment become more production like Faster Feedback

Slide 75

Slide 75 text

Continuous Delivery – Tradeoffs • Prevented from releasing into production untested builds • No regression bugs, especially from hotfixes • No surprises on the production environment • Rapid, repeatable, and reliable releases = Normal Events • In the worst case of critical bug introduced? revert to an earlier version! Immediate Gains

Slide 76

Slide 76 text

Continuous Delivery – Monitor & Control Through commercial or custom tools Devs, Testers & Ops able to • See the release candidates available to them as well as their status • Press a button to deploy the selected build by running the deployment script in the relevant environment. • See which build is deployed into which environment, and which stages in the pipeline each build has passed

Slide 77

Slide 77 text

Continuous Delivery – Practices Only Build Your Binaries Once • Every time we compile, we run the risk of introducing some difference. I.e. pick a different library with Maven. • Recompiling violates the fast feedback principle • Production binaries should be exactly the same as those that went through the acceptance test process. Use hashes, checksums etc. • Forces us to separate code and configuration 

Slide 78

Slide 78 text

Continuous Delivery – Practices Deploy the same way to every* Environment • Every = a developer workstation, a testing environment, or production. • Don’t use a different deployment script for every environment. • Using the same script to deploy to production that you use to deploy to development environments is a fantastic way to prevent the “it works on my machine” syndrome. • Have a separate properties file for each environment (stored in VCS). The correct one selected either by looking at the hostname of the local server, or (in a multimachine environment) through the use of an environment variable supplied to the deployment script. Or store it in a database and access it through an application like ESCAPE.

Slide 79

Slide 79 text

Continuous Delivery – Practices Smoke-Test Your Deployments • Have an automated script that does a smoke test to make sure that the application is up and running. • Check that any services the application depends on are up and running, such as a database, messaging bus, or external service. • If unsuccessful should give some basic diagnostics about the reason. • It is probably the most important test to write once you have a unit test suite up and running

Slide 80

Slide 80 text

Continuous Delivery – Practices Deploy into a Copy of Production • Do testing and continuous integration on environments that are as similar as possible to your production environment. • Ideally have an exact replica staging environment. • What same means? Infrastructure, such as network topology and firewall configuration Operating system configuration, including patches Application stack Application’s data is in a known, valid state How? With disk imaging and virtualization & tools like Puppet, Chef, CfEngine & InstallShield along with a version control repository.

Slide 81

Slide 81 text

Continuous Delivery – Practices Each Change should Propagate through the Pipeline Instantly The first stage should be triggered upon every check-in, and each stage should trigger the next one immediately upon successful completion.

Slide 82

Slide 82 text

Continuous Delivery – Practices If Any Part of the Pipeline Fails, Stop the Line • The team accepts that every time they check code into version control, it will successfully build and pass every test.* • The whole team owns the failure of a failed deployment • So… whole team stops and fixes it before doing anything else.

Slide 83

Slide 83 text

Continuous Delivery – What’s Next? How to start implementing a Deployment Pipeline What metrics do we use? Build & Deployment Scripting The Commit Stage Automated Functional Acceptance Tests Automated Non-Functional Acceptance Tests Deploying & Releasing Applications

Slide 84

Slide 84 text

Stratos Pavlakis [email protected] This presentation was created as part of the effort to present the basic principles and techniques of achieving continuous delivery in software development as described in Jez Humbles’ and David Farleys’ “Continuous Delivery” published in 2010 from Addison-Wesley, to all SD & IT colleagues at Upstream. [email protected]