TM699 Continuous Integration Defense

Technology Management defense presentation

Transcript

1. Marshall University Technology Management: Information Technology Eric G. Wolfe Adoption of Continuous Integration Practices to Ensure Quality in IT Operations and Application Delivery
2. Continuous Integration • Continuous integration (CI) is a practice in software development, where developers integrate, or build and test, software as incremental changes are made (Fowler, 2006). • Continuous delivery is an extension of CI applying automated deployment, or delivery, of an application.
3. Problems • Chef software configuration management (SCM) –Had grown to 45,000+ lines of source code • Hard to understand • High barrier to entry • Abandoned work-in-progress and unused code –Modules (cookbooks), and programs (recipes) stored together. • Inconsistent quality controls • Tightly coupled components • No dependency management
4. What? 1)Refactored code into loosely coupled components. 2)Tied components together with dependency management. 3)Introduced static analyses, unit, and integration tests. 4)Released only known good software components. 5)Learned from mistakes and introduced regression tests to prevent future failure.
5. How? • Applied lessons learned from: –Lean manufacturing/Toyota Production System • Shewart/Deming Circle (Rother, 2010). • Theory of Constraints (Goldratt, 2004). • Andon cord and Toyota “Best Practices” (Rother, 2010). –Systems Management theory • Promise theory (Bergstra & Burgess, 2008). • The “Three Ways” (Kim et al., 2013). –Agile software development
6. Shewart/Deming Cycle
7. Theory of Constraints
8. Andon cord • The Andon cord is a red length of rope at every worker's station empowering them to stop the line when there is a safety, or quality, problem. • How many times a shift does it get used? • What does it mean to Toyota management when the average rate of Andon cord use drops? (Rother, 2010).
9. Promise Theory • “A promise is an announcement of fact or behaviour by a promiser to a promisee, observed by a number of witnesses, whose outcome has yet to be assessed.” (Bergstra & Burgess, 2008). • When one promises a neighbor to walk their dog it constitutes a promise, but not an obligation. • One has to test, or observe, the promise through another agent to know the outcome.
10. “Three Ways” 1)“Create fast flow of work from development to IT operations, because this is what lies between the product and the customer.” 2)“Shorten and amplify feedback loops.” 3)“Create a culture that fosters experimentation, learning from failure, and understanding that repetition and practice are required for mastery.” (Kim et al., 2013)
11. Why all of the manufacturing theory? • Every organization adapting to the modern marketplace is an IT organization. • There is a large body of published work related to management theory in manufacturing industry. • There is only a moderate amount of theory to be applied from Agile software development, and much of it actually draws from the manufacturing industry.
12. Tools and methods applied • CI service –Jenkins –Build jobs scripted with Rake (Ruby make) • Static analyses/Linting –Foodcritic (Crump, n.d.). –Rubocop (Batsov, n.d.). • Unit testing –Chefspec • Integration testing
16. Staged builds • Gate changes in CI pipeline. –Moderate rate of broken build in CI environment. –Low rate of defect release into production environment. • Fail fast in CI environment. –Those who introduce change are notified when they break the build. –Shortens feedback loop the sooner a broken build fails.
17. Kitchen.CI + Docker container pipeline Recommendations for future work: • Kitchen.CI, or kitchen-docker, API to pass image artifact to downstream dependencies. • Built-in Kitchen.CI build job Rake tasks do not execute concurrently to save build time.
18. Results Nominal reduction of waste 93.23% source lines of code in two in-scope projects. Effective reduction of waste 73.27% source lines of code in monolithic code base, plus in-scope projects. Average nominal failure rate for baseline configuration in CI environment. 15.95% nominal rate of build job failure for static analysis, integration tests, and deploy jobs. Average effective failure rate for baseline configuration in CI environment. 18.37% effective rate of build job failure for static analysis, and integration tests. Baseline changes introduced; rate of defect release; changes per day. 102 changes introduced; 1.96% (2/102) defects released; 1.18 changes per day. Average nominal failure rate for mail configuration in CI environment. 13.96% nominal rate of build job failure for static analysis, integration tests, and deploy jobs. Average effective failure rate for mail configuration in CI evironment. 16.43% effective rate of build job failure for static analysis, and integration tests. Mail changes introduced; rate of defect release; changes per day. 101 changes introduced; 0.99% (1/101) defect released; 1.7 changes per day.
19. Questions?