Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Increasing Engineering Tempo at Splice

Increasing Engineering Tempo at Splice

A video of this talk can be found at https://codeclimate.wistia.com/medias/jcrn6ahlp1

The Splice Engineering team grew almost ten times in eighteen months. The delivery practices that worked for us when we were five broke way before we got to fifty. We wanted our organization to learn faster than the market; to do this, we had to find a way to unlock our delivery. We used insights found on the State of DevOps Report and Accelerate the book. With this knowledge and the help of metrics to visualize our delivery, we were able to get back on a high-performance track.

This talk tells the story of how we used data to engineer the performance of our engineering organization and gives you practical takeaways on how you can do it too.

Juan Pablo Buriticá

May 22, 2019
Tweet

More Decks by Juan Pablo Buriticá

Other Decks in Technology

Transcript

  1. Remember how we said we weren't planning on growing that

    much? yup... Change of plans, what do you need to build W, X, Y, and Z? more than a few engineers Cool, here's some $$$
  2. @buritica Splice Engineering's Mission: We enable Splice to learn faster

    than the market by delivering production-ready software at an accelerating tempo.
  3. Things feel slow, don't they? yeah... What are we going

    to do about it? I'm not sure yet, but we'll come up with a plan to fix it ok
  4. @buritica team growth had taken the majority of our attention,

    and we let our delivery process stagnate
  5. @buritica given our org size, how we managed change was

    as important as the change desired
  6. @buritica possible metrics Time to Implementation Time to Merge Time

    to Deploy Deploy Confidence Deploy Frequency Support Time / Frequency System Stability Product <> Eng Partnership? Code review and testing? Deployment pipelines state? Test coverage? Deployment pipelines state? Quality? Quality?
  7. @buritica our vision: 
 A future where shipping working code

    is one of the fastest, safest and, most effective ways to learn and to test new ideas.
  8. @buritica a future where: 
 Every engineer is empowered by

    tools and processes, willing and able to take more significant risks.
  9. @buritica a future where: 
 Engineers are confident that the

    code they deliver won't break Splice and when it does, they can fix it quickly.
  10. @buritica a future where: 
 Everyone feels comfortable and welcome

    contributing code to any part of our codebase, including End to End tests.
  11. @buritica a future where: 
 ⚡We continuously expose all Splice

    employees to new functionality and learnings.
  12. @buritica time to merge 
 By the end of Q3,

    we are merging 100% of Pull Requests within 36 hours.
  13. @buritica deploy frequency 
 By the end of Q3, all

    product teams are deploying at least once a day.
  14. @buritica new e2e test coverage 
 By the end of

    Q3, 100% of Engineers have written an End to End Test in an improved testing environment.
  15. @buritica delivery working group 
 three person infra team a

    backend & a frontend engineer a quality engineer management
  16. @buritica mid Q3 report "We have not moved Time to

    Merge in any significant way since Q3 began"
  17. @buritica mid Q3 report "Daily deploys have gone up significantly

    since the introduction of Frontend Preview Branches"
  18. @buritica mid Q3 report "Some Frontend Engineers have begun to

    contribute to E2E tests, we still have significant ground to cover before end of quarter"
  19. @buritica "I wish someone else would tell me the important

    bits about all this data" - @buritica
  20. @buritica Product Design & Development Product Delivery Create new products

    and services that solve customer problems using hypothesis-driven delivery, modern UX, design thinking. Enable fast flow from development to production and reliable releases by standardizing work, and reducing variability and batch sizes. Feature design and implementation may require work that has never been performed before. Integration, test, and deployment must be performed continuously as quickly as possible. Estimates are highly uncertain. Cycle times should be well-known and predictable. Outcomes are highly variable Outcomes should have low variability Novelty implemented Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”
  21. @buritica work starts Lead Time vs Cycle Time cycle time

    work requested work finished lead time
  22. @buritica Delivery Performance Metrics: ⌚ Lead Time (commit to prod)

    Deployment Frequency Mean Time to Restore
 Change Failure Rate Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”
  23. @buritica High Performing Teams: ⌚ Less than one hour Multiple

    deploys per day Less than one hour
 0 - 15% Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”
  24. @buritica Splice Engineering at the time: ⌚ Between one week

    and one month Once per week to several times a month Between one hour to one day
 ??? * Anecdotal experience
  25. @buritica Velocity is designed to be used as a capacity

    planning tool. Using it as a productivity metric has several flaws.
  26. @buritica Velocity is relative and team dependent. When used as

    productivity metric, teams game it, which impacts collaboration.
  27. @buritica Velocity is relative and team dependent. When used as

    productivity metric, teams game it, which impacts collaboration. Using capacity to measure productivity leads to high utilization, which reduces ability to take unplanned work.
  28. @buritica “Queue theory in math tells us that as utilization

    approaches 100%, lead times approach infinity—in other words, once you get to very high levels of utilization, it takes teams exponentially longer to get anything done. ” Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”
  29. @buritica Delivery Matters Whatever the mission, engineering performance can predict

    overall organizational performance Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”
  30. @buritica CONTINUOUS DELIVERY PERFORMANCE ORIENTED CULTURE LEAN PRODUCT MANAGEMENT DELIVERY

    PERFORMANCE ORGANIZATIONAL PERFORMANCE Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.” culture drivers culture outcomes
  31. @buritica To achieve this goal: We relentlessly pursue continuous improvement

    to drive the Tech Team's culture towards high-performance by using a capabilities model proven to predict high performance in organizations.
  32. @buritica Capability Categories Continuous Delivery Architecture Product & Process Lean

    management & monitoring Cultural Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”
  33. @buritica Continuous Delivery Architecture Product & Process Lean Mgt &

    Monitoring Culture Version control for all artifacts Loosely coupled architecture Customer feedback implementation Lightweight change approval Support a generative culture Automated deployment Architect for empowered teams Visibility into flow of work through value stream Infrastructure monitoring Encourage and support learning Continuous integration Small batched work Proactive system health checks Support and facilitate cross-team collaboration Trunk-based development Enable experimentation Work In Progress limits Provide resources and tools that make work meaningful Test automation Visualize work to monitor quality Transformational leadership Test data management Integrated security Continuous delivery Capabilities
  34. @buritica Continuous Delivery Architecture Product & Process Lean Mgt &

    Monitoring Culture Version control for all artifacts Loosely coupled architecture Customer feedback implementation Lightweight change approval Support a generative culture Automated deployment Architect for empowered teams Visibility into flow of work through value stream Infrastructure monitoring Encourage and support learning Continuous integration Small batched work Proactive system health checks Support and facilitate cross-team collaboration Trunk-based development Enable experimentation Work In Progress limits Provide resources and tools that make work meaningful Test automation Visualize work to monitor quality Transformational leadership Test data management Integrated security Continuous delivery Capabilities
  35. @buritica "I wish someone made a tool that helped me

    measure my team easily" - @buritica
  36. @buritica our working group has been successful, we'd like to

    invest more sounds good Gorsuch, you got this reeeeleaaase the
  37. @buritica Production Engineering Tooling, services, and expertise that enables teams

    to deliver and operate high quality, production-level software and services.
 
 Security Engineering
 Site Reliability Engineering
 Quality Reliability Engineering
 Developer Experience (DevX)
  38. @buritica Production Engineering Primary Metric: Cycle Time (commit to production)

    Secondary Metrics: 
 Mean Time to Restore, Deploy Frequency, Time to First Commit
  39. 119 a performance oriented culture We have created a culture

    of continuous systems and process improvement. We also have a language and a framework to measure this change.
  40. @buritica Notable Changes • Quality Expansion & Acceleration • Focused

    Quality Engs • Transferred Quality Ownership to Eng • New & Improved Testing Infra & Tools • Decentralized Staging Environment • Sub 2 min API Deploys • Stabilized Search Infrastructure • Decoupled Desktop UI • Feature Flags