How Automation Reveals Technical Debt

These slides accompany the webinar examining best practices for implementing traceable, reproducible processes, using automation for on-going visibility into development the development processes, how to construct an implementation plan for build and deployment automation, and more.

Transcript

1. © 2013 IBM Corporation How Automation Reveals Technical Debt
2. © 2013 IBM Corporation Eric’s Bio I’m a DevOps Evangelist at UrbanCode where I helps customers get the most out of their build, deploy and release processes. I have automation experience throughout the application life-cycle in roles as a developer, test automation engineer, and support engineer. For the last 9 years, I’ve been focused on CI, CD and DevOps Eric Minick eric@urbancode.com @EricMinick
3. © 2013 IBM Corporation Technical Debt
4. © 2013 IBM Corporation Why do we accumulate technical debt? § We leverage technical debt to deliver more faster. § De-leveraging is rarely accounted for in project planning. § Green-Shifting* Scope Time Resources * http://www.drdobbs.com/191600661
5. © 2013 IBM Corporation Why do we care? Paying interest 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Now Later Later Still Much Later Debt Paid Interest Paid Value Delivered Debt
6. © 2013 IBM Corporation Why do we care? Or paying our debts 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Now Later Later Still Much Later Debt Paid Interest Paid Value Delivered Debt
7. © 2013 IBM Corporation Pay it now, or pay more later Debt Paid Interest Paid Value Delivered Debt Paid Interest Paid Value Delivered
8. © 2013 IBM Corporation Why should we care? Baggage that slows the team § Lack of automated tests lengthen QA cycles § Fear of merging work § Unrefactored code slow to work with § Slow build / deploy processes delay learning and release pace Quality issues § Lack of tests results in buggier code § Releases are error prone and lead to unnecessary outages
9. © 2013 IBM Corporation The limits of what we know § Known Knowns: Bugs confirmed and tracked § Known Unknowns: Undiscovered bugs § Unknown unknowns: One of our project teams is using a GPL’d library making their product impossible to ship
10. © 2013 IBM Corporation Where Automation Helps § “Testing” for debt: Automated tests, code scans and reports can help identify (and quantify) problems. § Automation as a learning experience: The act of automating brings surprises.
11. © 2013 IBM Corporation Testing for Debt § Continuous Integration (multi-component) § Code Inspection § Watching Trends
12. © 2013 IBM Corporation Testing for Debt: Continuous Integration § On code commit, build and test the software § Roll up changes to build-time or runtime dependencies and test those too to identify API incompatibilities
13. © 2013 IBM Corporation Testing for Debt: Code Inspection Code Reviews – Rapidly detect issues Static Code Analysis – Tool based checks for bugs, code duplication, code theft, & non-compliance with dev standards.
14. © 2013 IBM Corporation Testing for Debt: Code Inspection Code Reviews – Rapidly detect issues Static Code Analysis – Tool based checks for bugs, code duplication, code theft, & non-compliance with dev standards.
15. © 2013 IBM Corporation Testing for Debt: Watching Trends
16. © 2013 IBM Corporation More visualizations: Sonar http://nemo.sonarsource.org/dashboard/index/327690? did=6
17. © 2013 IBM Corporation An example safety net § Continuous build & unit test § Nightly slow tests / code scans § Emails identifying new issues – ideally tied against source code changes § Regular inspection of trends § Bugs / Stories entered around issues
18. © 2013 IBM Corporation Automation as a learning experience Implementing the safety net helps us discover unacknowledged debt
19. © 2013 IBM Corporation Tests? What tests?
20. © 2013 IBM Corporation Automation Examples: The Build § Build time dependencies not understood § Build scripts missing or incomplete § “Magic build server” anti-pattern http://www.urbancode.com/html/resources/webinars/Role_of_Binary_repositories_in_Software_Configuration_Management.html
21. © 2013 IBM Corporation Automation Examples: Deployment § Deployment scripts scarce § “Special Instructions” with most deployments indicate non- repeatable process § Environmental differences handled poorly § Separation of duties less than real
22. © 2013 IBM Corporation Automation Example: Test Automation § Requirements less well understood § Existing tests are few, stale, un-optimized § Application not architected for testability
23. © 2013 IBM Corporation Expect the unexpected when automating § At scale, Green-Shifting, has hidden issues § Include these discoveries in ROI estimations for automation (positively and negatively) § This is a happy side effect
24. © 2013 IBM Corporation Start making decisions
25. © 2013 IBM Corporation Direct and indirect automation benefits § Direct: We tested for problems and found them. § Indirect: In attempting to be more efficient, we automated, and accidently discovered problems.
26. © 2013 IBM Corporation Automating for the team § Provide a “safety net” to detect and recognize issues. § Diagnose and repair lack of repeatability in build-deploy-test § Quantify accumulating debt in support of fighting scope creep § Team level tooling is fine
27. © 2013 IBM Corporation Automating the enterprise § Benefits extend beyond aggregate team level benefits § Central Automation and Reporting gets us: – Identify who can use shared configuration – Who has tests, who doesn’t – Who is using what tools – Build / deploy failure rates
28. © 2013 IBM Corporation Stories from customers
29. © 2013 IBM Corporation Favorite Examples: Deployment Failures § Debt: High rate of deployment failures a problem § Interest: QA productivity is getting hurt & lengthened time to market § Goal: Reduce failure rate from 40% to 5% § Approach: Avoid manually fixing a deployment. Fix the automation and redeploy. § Enforcement: Monthly / weekly spreadsheet of success to CIO with a six month plan.
30. © 2013 IBM Corporation Favorite Examples: “End of Day” Commits § Debt: Developers commit breaking changes at the end of the day § Interest: Code base broken in morning, or other people stay up late to fix it. § Goal: Avoid other devs staying late to clean up § Approach: Report on failures, and react to negative patterns as a team. § Enforcement: Social pressures
31. © 2013 IBM Corporation
32. © 2013 IBM Corporation Favorite Examples: Low numbers of tests § Problem: Unit testing discipline breaking down over time § Goal: Maintain high or improving coverage § Approach: Standard coverage tools and an emphasis on upward trends. § Enforcement: Trending report on monitor over CIO’s door
33. © 2013 IBM Corporation No hiding! No greenshifting
34. © 2013 IBM Corporation Summary § We accumulate technical debt as we race to deliver more, faster. § This causes us to eventually release less, slower, with worse quality § Automation directly and indirectly helps us find issues. § There are benefits at both team and enterprise levels.
35. © 2013 IBM Corporation More References http://urbancode.com/resources § Enterprise CD Maturity Model § Lean Build & Deployment Automation § Managing Release Risks with Metrics Blogs.urbancode.com Twitter.com/UrbanCode Facbebook.com/UrbanCode Slideshare.net/Urbancode
36. © 2013 IBM Corporation Yes, we sell products for this § uBuild – Build management and continuous integration § uDeploy – Deployment and release automation § uRelease – Release management tool for planning and executing big releases ... And IBM’s amazing portfolio of CLM, testing tools, service virtualization, provisioning, etc, etc.
37. © 2013 IBM Corporation Questions? eminick@us.ibm.com @EricMinick, @Urbancode Slideshare.net/Urbancode