DevOpsDays Cuba 2016: DevOps from the cuban trenches

Author: Anesto del Toro Almenares
Summary:
We’ll share experiences during the implementation of our current software delivery system. Our initial goal was to achieve faster development and deployment cycles of quality products, in a reliable and sustainable way. The main challenges we faced and our achievements in terms of product planning, feature development, continuous integration, change and configuration management and continuous delivery will be exposed during the presentation.

Transcript

1. DevOps from Cuban trenches
2. We have challenges
3. … configure to block access from your country
4. Slooooooow internet access
8. What can we achieve despite of this?
9. DevOps from Cuban trenches
10. DevOps from Cuban trenches
11. DevOps from DATYS VC trenches
12. 25 Team Villa Clara Division
13. 25 Team 1 3 Villa Clara Division 1 6 Head of Production Team Lead Team members QA, Docs,… 1 6 1 6 Ops* Dev
14. About Me @anestodta anesto.deltoro@datys.cu https://www.linkedin.com/in/anesto-del-toro • Head of Production, DATYS VC • Computer Sciences • Wearing multiple hats oProject Management o Business Analysis o Architecture o Technical Lead o Software Development (mostly backend) o Lean-Agile enthusiast o Eager learner • 16 years of experience o ~6 Researcher at UCLV (BE, PR & CV) Anesto del Toro Almenares
15. About the TEAM
16. About the TEAM • Disciplined • Collaboration culture • Automation mindset*
17. About the TEAM @MarialinaBall @kikicarbonell @jcgomezcorrea1 @DuvielRodriguez @dayron_aj @christian861220 @genrylg @osmel_mora • Disciplined • Collaboration culture • Automation mindset*
18. Up to 10 deploys/releases* per day
19. Back to 2007 How bad could it be?
21. manual software builDing
22. manual software builDing manual Testing after development
23. manual software builDing no artifacts Management manual Testing after development
24. manual software builDing painful intEgrations no artifacts Management manual Testing after development
25. manual proVisioning manual software builDing painful intEgrations no artifacts Management manual Testing after development
26. manual proVisioning manual software builDing painful intEgrations no artifacts Management manual deplOyment manual Testing after development
27. manual proVisioning poorLy controlled processes manual software builDing painful intEgrations no artifacts Management manual deplOyment manual Testing after development
28. manual proVisioning manual software builDing painful intEgrations no artifacts Management manual deplOyment manual Testing after development infrequent Releases poorLy controlled processes
29. manual proVisioning lOng feedback cycles manual software builDing painful intEgrations no artifacts Management manual deplOyment infrequent Releases manual Testing after development poorLy controlled processes
30. manual proVisioning lOng feedback cycles manual software builDing painful intEgrations no artifacts Management manual deplOyment infrequent Releases manual Testing after development poorLy controlled processes
31. At first sigh it looked scary
32. SCRUM TFS
33. Worsening effect
35. Software Delivery System Achieve faster development and deployment cycles of quality products, in a reliable and sustainable way Enterprise Agility
36. Assessment
37. Jez Humble and David Farley Continuous Delivery A Maturity Assessment Model
38. • Build management and Continuous Integration • Environment and deployments • Testing • Release management and compliance • Configuration management • Data management Continuous Delivery A Maturity Assessment Model
39. "We can't solve problems by using the same kind of thinking we used when we created them." Albert Einstein
41. Principles
42. Principles Practices
43. Principles Practices Tools
44. Principles Practices Tools Order matters
45. Identify Evaluate Adopt / Discard Try Improve
46. Identify Evaluate Adopt / Discard Try Improve Context matters
47. Harry potter entrenando order de fenix Evangelism
48. Don’t give up
49. •Optimize the whole •Eliminate waste •Build Quality-In •Deliver Fast •Focus on Flow LEAN Principles • Defer committment • Amplified learning • Engage Everyone • Continuous Improvement Mary and Tom Poppendieck
50. Removing waste Reducing time to customer feedback Accelerating software delivery Lean guiding transition to Agility (context creation) Lean-Agile
51. Agile Practices focused on iteratively producing functionality Agility Business Value Increments not development cycles
52. Technical and Engineering Practices
54. Focus on optimize value creation Holistic view
55. Value Stream Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version
56. Value Stream and Actors Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Head of Production Team Lead Dev Teams Ops Team QA&Docs Team
57. Value Stream and Actors Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Head of Production Team Lead Dev Teams Ops Team QA&Docs Team • Workflow • Bottlenecks • Teamwork • …
58. A system is never the sum of its parts. It is the product of the interactions of its parts Russell Ackoff
59. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version
60. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Start End Cycle Time
61. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Reducing the Cycle Time requires the optimization of the whole Value Stream Start End Cycle Time
62. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Reducing the Cycle Time requires the optimization of the whole Value Stream Start End Cycle Time Optimize what we do? Remove delays between what we do?
63. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Speed without sacrificing quality Time To Market
64. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy Version Speed without sacrificing quality Explicit policies = Definition of Done + Pull criteria
65. Map the mess Empower the team to fix/improve it
67. Increment Review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
68. Increment Review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
69. Business Value Increments Predictability
71. LEVEL 1 LEVEL 2 LEVEL 3 Milestone 01 PROJECT XYZ Milestone 01 PROJECT XYZ Milestone 01 PROJECT XYZ High Level Goals MBIs Tasks ANALYSIS SPECIFICATION Tasks • Pull-System • Incremental & Iterative • Continuous Delivery Management and Planning
73. • MBIs Mean resolution time: 15 days • 90 % MBIs takes at most 30 days • 98 % MBIs takes at most 34 días • … Predictability Analysis by Class of Service
74. Increment Review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
75. Technical agility Producing functionality
76. Next reléase from this point forward Merge bugfix Start RC 1.0 bugfixes Tag 0.1 Tag 0.2 Tag 1.0 Features for Next release Bug fixed production: hotfix 0.2 Merge bugfixes from RC to develop Future feature Time Features branches Develop RC hotfixes master Practices • Branch per feature (short lived) • Sandboxing • Self-service CI & Deploy • Blessed repositories Development activity
77. Minutes Weeks Production Staging UAT Security Increment deploy Unit Testing Acceptance tests Code Reviews Code metrics (*) Each cycle targets a different concern Feedback loops
78. Ops Team as Enablers
79. Deployment automation Infrastructure as Code Supporting services Monitoring Security … TDI
80. Collaboration materialized Improving efficiency Team happiness
81. Increment Review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
82. Build the thing Right
83. Technical debt
84. Maintainability Testability
85. 99.74% Unit: 5086 Integration: 3485 ~ 25 min
86. Increment Review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
87. Did we built the Right product?
88. Root cause analysis
89. Learn from failures
90. PRODUCTION SYSTEM Increment review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation, Self- services, Code hosting Platform as a Service Artifact and dependencies repositories Stakeholders Internet
91. Docs automation
92. Docs as Code
93. Docs and releases alignment Known workflow
94. PRODUCTION SYSTEM Increment review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation, Self- services, Code hosting Platform as a Service Artifact and dependencies repositories Stakeholders Internet
95. (supporting services + products)
96. Game changer Better resource utilization Simplified provisioning Uniform deployment Failure tolerance Scalability …
97. Bring Ops tools closer to Developers
98. Increment Review User Acceptance Testing Documentation Development & Operations Management & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
99. Monitoring
100. UAT Deploy Production like Deploy Unit tests Integration tests Code & Vulnerabilities Analysis Acceptance tests Load, Smoke, Security tests … Exploratory Production deploy Deployment pipeline Up to 10 deploys/releases* per day
101. Assessment 2016
102. Continuous Delivery: A Maturity Assessment Model
109. Not designed a priori
110. Not designed a priori Result of evolution
111. Context matters
112. Context matters Challenge principles, practices and tools
113. Reality changes
114. Stay constantly learning, adapting and evolving Reality changes
115. Focus
116. Value creation People Culture Automation mindset Focus
117. Results are not the solely point, but the development of People and Systems that allows us to achieve the results we expect
118. Stay alert Endless trip
119. It is supposed to be FUN!!!
120. DevOps from Cuban trenches @anestodta anesto.deltoro@datys.cu https://www.linkedin.com/in/anesto-del-toro Anesto del Toro Almenares Thank you