DevOpsDays Cuba 2016: DevOps from the cuban trenches

DevOpsDays Cuba 2016: DevOps from the cuban trenches

Author: Anesto del Toro Almenares
Summary:
We’ll share experiences during the implementation of our current software delivery system. Our initial goal was to achieve faster development and deployment cycles of quality products, in a reliable and sustainable way. The main challenges we faced and our achievements in terms of product planning, feature development, continuous integration, change and configuration management and continuous delivery will be exposed during the presentation.

D5db2dc3cc883df3479797edb63b581b?s=128

DevOpsDays Cuba

October 19, 2016
Tweet

Transcript

  1. DevOps from Cuban trenches

  2. We have challenges

  3. … configure to block access from your country

  4. Slooooooow internet access

  5. None
  6. None
  7. None
  8. What can we achieve despite of this?

  9. DevOps from Cuban trenches

  10. DevOps from Cuban trenches

  11. DevOps from DATYS VC trenches

  12. 25 Team Villa Clara Division

  13. 25 Team 1 3 Villa Clara Division 1 6 Head

    of Production Team Lead Team members QA, Docs,… 1 6 1 6 Ops* Dev
  14. About Me @anestodta anesto.deltoro@datys.cu https://www.linkedin.com/in/anesto-del-toro • Head of Production, DATYS

    VC • Computer Sciences • Wearing multiple hats oProject Management o Business Analysis o Architecture o Technical Lead o Software Development (mostly backend) o Lean-Agile enthusiast o Eager learner • 16 years of experience o ~6 Researcher at UCLV (BE, PR & CV) Anesto del Toro Almenares
  15. About the TEAM

  16. About the TEAM • Disciplined • Collaboration culture • Automation

    mindset*
  17. About the TEAM @MarialinaBall @kikicarbonell @jcgomezcorrea1 @DuvielRodriguez @dayron_aj @christian861220 @genrylg

    @osmel_mora • Disciplined • Collaboration culture • Automation mindset*
  18. Up to 10 deploys/releases* per day

  19. Back to 2007 How bad could it be?

  20. 2007

  21. manual software builDing

  22. manual software builDing manual Testing after development

  23. manual software builDing no artifacts Management manual Testing after development

  24. manual software builDing painful intEgrations no artifacts Management manual Testing

    after development
  25. manual proVisioning manual software builDing painful intEgrations no artifacts Management

    manual Testing after development
  26. manual proVisioning manual software builDing painful intEgrations no artifacts Management

    manual deplOyment manual Testing after development
  27. manual proVisioning poorLy controlled processes manual software builDing painful intEgrations

    no artifacts Management manual deplOyment manual Testing after development
  28. manual proVisioning manual software builDing painful intEgrations no artifacts Management

    manual deplOyment manual Testing after development infrequent Releases poorLy controlled processes
  29. manual proVisioning lOng feedback cycles manual software builDing painful intEgrations

    no artifacts Management manual deplOyment infrequent Releases manual Testing after development poorLy controlled processes
  30. manual proVisioning lOng feedback cycles manual software builDing painful intEgrations

    no artifacts Management manual deplOyment infrequent Releases manual Testing after development poorLy controlled processes
  31. At first sigh it looked scary

  32. SCRUM TFS

  33. Worsening effect

  34. None
  35. Software Delivery System Achieve faster development and deployment cycles of

    quality products, in a reliable and sustainable way Enterprise Agility
  36. Assessment

  37. Jez Humble and David Farley Continuous Delivery A Maturity Assessment

    Model
  38. • Build management and Continuous Integration • Environment and deployments

    • Testing • Release management and compliance • Configuration management • Data management Continuous Delivery A Maturity Assessment Model
  39. "We can't solve problems by using the same kind of

    thinking we used when we created them." Albert Einstein
  40. None
  41. Principles

  42. Principles Practices

  43. Principles Practices Tools

  44. Principles Practices Tools Order matters

  45. Identify Evaluate Adopt / Discard Try Improve

  46. Identify Evaluate Adopt / Discard Try Improve Context matters

  47. Harry potter entrenando order de fenix Evangelism

  48. Don’t give up

  49. •Optimize the whole •Eliminate waste •Build Quality-In •Deliver Fast •Focus

    on Flow LEAN Principles • Defer committment • Amplified learning • Engage Everyone • Continuous Improvement Mary and Tom Poppendieck
  50. Removing waste Reducing time to customer feedback Accelerating software delivery

    Lean guiding transition to Agility (context creation) Lean-Agile
  51. Agile Practices focused on iteratively producing functionality Agility Business Value

    Increments not development cycles
  52. Technical and Engineering Practices

  53. None
  54. Focus on optimize value creation Holistic view

  55. Value Stream Ideas Materialize MBIs Specify Prioritize Dev Review UAT

    Document Deploy Version
  56. Value Stream and Actors Ideas Materialize MBIs Specify Prioritize Dev

    Review UAT Document Deploy Version Head of Production Team Lead Dev Teams Ops Team QA&Docs Team
  57. Value Stream and Actors Ideas Materialize MBIs Specify Prioritize Dev

    Review UAT Document Deploy Version Head of Production Team Lead Dev Teams Ops Team QA&Docs Team • Workflow • Bottlenecks • Teamwork • …
  58. A system is never the sum of its parts. It

    is the product of the interactions of its parts Russell Ackoff
  59. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy

    Version
  60. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy

    Version Start End Cycle Time
  61. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy

    Version Reducing the Cycle Time requires the optimization of the whole Value Stream Start End Cycle Time
  62. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy

    Version Reducing the Cycle Time requires the optimization of the whole Value Stream Start End Cycle Time Optimize what we do? Remove delays between what we do?
  63. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy

    Version Speed without sacrificing quality Time To Market
  64. Ideas Materialize MBIs Specify Prioritize Dev Review UAT Document Deploy

    Version Speed without sacrificing quality Explicit policies = Definition of Done + Pull criteria
  65. Map the mess Empower the team to fix/improve it

  66. None
  67. Increment Review User Acceptance Testing Documentation Development & Operations Management

    & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
  68. Increment Review User Acceptance Testing Documentation Development & Operations Management

    & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
  69. Business Value Increments Predictability

  70. None
  71. LEVEL 1 LEVEL 2 LEVEL 3 Milestone 01 PROJECT XYZ

    Milestone 01 PROJECT XYZ Milestone 01 PROJECT XYZ High Level Goals MBIs Tasks ANALYSIS SPECIFICATION Tasks • Pull-System • Incremental & Iterative • Continuous Delivery Management and Planning
  72. None
  73. • MBIs Mean resolution time: 15 days • 90 %

    MBIs takes at most 30 days • 98 % MBIs takes at most 34 días • … Predictability Analysis by Class of Service
  74. Increment Review User Acceptance Testing Documentation Development & Operations Management

    & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
  75. Technical agility Producing functionality

  76. Next reléase from this point forward Merge bugfix Start RC

    1.0 bugfixes Tag 0.1 Tag 0.2 Tag 1.0 Features for Next release Bug fixed production: hotfix 0.2 Merge bugfixes from RC to develop Future feature Time Features branches Develop RC hotfixes master Practices • Branch per feature (short lived) • Sandboxing • Self-service CI & Deploy • Blessed repositories Development activity
  77. Minutes Weeks Production Staging UAT Security Increment deploy Unit Testing

    Acceptance tests Code Reviews Code metrics (*) Each cycle targets a different concern Feedback loops
  78. Ops Team as Enablers

  79. Deployment automation Infrastructure as Code Supporting services Monitoring Security …

    TDI
  80. Collaboration materialized Improving efficiency Team happiness

  81. Increment Review User Acceptance Testing Documentation Development & Operations Management

    & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
  82. Build the thing Right

  83. Technical debt

  84. Maintainability Testability

  85. 99.74% Unit: 5086 Integration: 3485 ~ 25 min

  86. Increment Review User Acceptance Testing Documentation Development & Operations Management

    & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
  87. Did we built the Right product?

  88. Root cause analysis

  89. Learn from failures

  90. PRODUCTION SYSTEM Increment review User Acceptance Testing Documentation Development &

    Operations Management & Planning Knowledge sharing/retention Automation, Self- services, Code hosting Platform as a Service Artifact and dependencies repositories Stakeholders Internet
  91. Docs automation

  92. Docs as Code

  93. Docs and releases alignment Known workflow

  94. PRODUCTION SYSTEM Increment review User Acceptance Testing Documentation Development &

    Operations Management & Planning Knowledge sharing/retention Automation, Self- services, Code hosting Platform as a Service Artifact and dependencies repositories Stakeholders Internet
  95. (supporting services + products)

  96. Game changer Better resource utilization Simplified provisioning Uniform deployment Failure

    tolerance Scalability …
  97. Bring Ops tools closer to Developers

  98. Increment Review User Acceptance Testing Documentation Development & Operations Management

    & Planning Knowledge sharing/retention Automation Self-services Code repository Platform as a Service Artifacts and dependencies repositories Stakeholders Internet Production System
  99. Monitoring

  100. UAT Deploy Production like Deploy Unit tests Integration tests Code

    & Vulnerabilities Analysis Acceptance tests Load, Smoke, Security tests … Exploratory Production deploy Deployment pipeline Up to 10 deploys/releases* per day
  101. Assessment 2016

  102. Continuous Delivery: A Maturity Assessment Model

  103. None
  104. None
  105. None
  106. None
  107. None
  108. None
  109. Not designed a priori

  110. Not designed a priori Result of evolution

  111. Context matters

  112. Context matters Challenge principles, practices and tools

  113. Reality changes

  114. Stay constantly learning, adapting and evolving Reality changes

  115. Focus

  116. Value creation People Culture Automation mindset Focus

  117. Results are not the solely point, but the development of

    People and Systems that allows us to achieve the results we expect
  118. Stay alert Endless trip

  119. It is supposed to be FUN!!!

  120. DevOps from Cuban trenches @anestodta anesto.deltoro@datys.cu https://www.linkedin.com/in/anesto-del-toro Anesto del Toro

    Almenares Thank you