Increasing Engineering Tempo at Splice

Juan Pablo Buritica - @buritica  speakerdeck.com/buritica  Increasing Engineering Tempo at

sounds studio plugins

Empower musicians to realize their creative potential and unleash it
on the world.

@buritica

@buritica I've had to solve two difficult problems in the
pursuit of this mission

@buritica 1. grow the team to adapt to a new
strategy

Remember how we said we weren't planning on growing that
much? yup... Change of plans, what do you need to build W, X, Y, and Z? more than a few engineers Cool, here's some $$$

@buritica ¡ay dios mio!

@buritica and so we grew...

@buritica Splice Engineering Team Size 2014 2015 2016 2017 2018
2019 20 40 60

@buritica

@buritica 2. fix the mess I caused by growing the
team that fast

@buritica Splice Engineering's Mission: We enable Splice to learn faster
than the market by delivering production-ready software at an accelerating tempo.

@buritica we wanted to learn faster than the market but
everything felt slow

@buritica it was only a feeling

Things feel slow, don't they? yeah... What are we going
to do about it? I'm not sure yet, but we'll come up with a plan to fix it ok

¡ay dios mio!

@buritica what felt slow?

@buritica moving completed engineering work out to customers felt like
an uphill battle

@buritica "staging is blocked"

@buritica "can you merge my PR?"

@buritica "how long has this been broken for?"

@buritica team growth had taken the majority of our attention,
and we let our delivery process stagnate

@buritica given our org size, how we managed change was
as important as the change desired

@buritica we needed a strategy

@buritica plan for the plan  define metrics set targets measure
make changes measure again

@buritica we researched

@buritica

@buritica possible metrics Time to Implementation Time to Merge Time
to Deploy Deploy Confidence Deploy Frequency Support Time / Frequency System Stability Product <> Eng Partnership? Code review and testing? Deployment pipelines state? Test coverage? Deployment pipelines state? Quality? Quality?

@buritica delivery bugs

@buritica delivery bugs opportunities

@buritica we researched

@buritica we researched we discussed we disagreed we committed

@buritica and we introduced our first proposal to the team

@buritica our vision:   A future where shipping working code
is one of the fastest, safest and, most effective ways to learn and to test new ideas.

@buritica a future where:   Every engineer is empowered by
tools and processes, willing and able to take more significant risks.

@buritica a future where:   Engineers are confident that the
code they deliver won't break Splice and when it does, they can fix it quickly.

@buritica a future where:   Everyone feels comfortable and welcome
contributing code to any part of our codebase, including End to End tests.

@buritica a future where:   ⚡We continuously expose all Splice
employees to new functionality and learnings.

@buritica timeframe: crawl the first month &walk by end of
Q3 ' run by end of H2

@buritica metrics & targets:

@buritica time to merge   By the end of Q3,
we are merging 100% of Pull Requests within 36 hours.

@buritica deploy frequency   By the end of Q3, all
product teams are deploying at least once a day.

@buritica new e2e test coverage   By the end of
Q3, 100% of Engineers have written an End to End Test in an improved testing environment.

@buritica delivery working group   three person infra team a
backend & a frontend engineer a quality engineer management

@buritica [Read the full RFC here]

@buritica middle of the quarter came along

@buritica mid Q3 report "We have not moved Time to
Merge in any significant way since Q3 began"

@buritica

@buritica mid Q3 report "Daily deploys have gone up significantly
since the introduction of Frontend Preview Branches"

@buritica

@buritica mid Q3 report "Some Frontend Engineers have begun to
contribute to E2E tests, we still have significant ground to cover before end of quarter"

@buritica outlook didn't look that good

i'm v busy, sorry

@buritica our instincts were right, but our story wasn't as
convincing

@buritica I needed better metrics and measurements

@buritica "I wish someone surveyed thousands of organizations about their
delivery practices" - @buritica

@buritica

@buritica "I wish someone else would tell me the important
bits about all this data" - @buritica

@buritica

@buritica Product Design & Development Product Delivery Create new products
and services that solve customer problems using hypothesis-driven delivery, modern UX, design thinking. Enable fast flow from development to production and reliable releases by standardizing work, and reducing variability and batch sizes. Feature design and implementation may require work that has never been performed before. Integration, test, and deployment must be performed continuously as quickly as possible. Estimates are highly uncertain. Cycle times should be well-known and predictable. Outcomes are highly variable Outcomes should have low variability Novelty implemented Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”

@buritica Splice Tech:    Product Management Product Design Engineering

@buritica Splice Tech is responsible for Design & Development

@buritica Splice Engineering is solely responsible for Delivery

@buritica Product Design & Development

@buritica first  commit Product Design & Development

@buritica first  commit product delivery product design Product Design &
Development

@buritica first  commit product delivery product design PR merge Product
Design & Development

@buritica first  commit product delivery product design PR merge deployment
Product Design & Development

@buritica first  commit product delivery product design PR merge deployment
release Product Design & Development

@buritica work starts Lead Time vs Cycle Time cycle time
work requested work finished lead time

@buritica Delivery Performance Metrics: ⌚ Lead Time (commit to prod)
Deployment Frequency Mean Time to Restore  Change Failure Rate Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”

@buritica High Performing Teams: ⌚ Less than one hour Multiple
deploys per day Less than one hour  0 - 15% Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”

@buritica Splice Engineering at the time: ⌚ Between one week
and one month Once per week to several times a month Between one hour to one day  ??? * Anecdotal experience

@buritica What about points & velocity?

@buritica Velocity is designed to be used as a capacity
planning tool.

@buritica Velocity is designed to be used as a capacity
planning tool. Using it as a productivity metric has several flaws.

@buritica Velocity is relative and team dependent.

@buritica Velocity is relative and team dependent. When used as
productivity metric, teams game it, which impacts collaboration.

@buritica Velocity is relative and team dependent. When used as
productivity metric, teams game it, which impacts collaboration. Using capacity to measure productivity leads to high utilization, which reduces ability to take unplanned work.

@buritica “Queue theory in math tells us that as utilization
approaches 100%, lead times approach infinity—in other words, once you get to very high levels of utilization, it takes teams exponentially longer to get anything done. ” Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”

@buritica velocity is good for planning work and bad for
measuring teams

@buritica hey boss, check this out

@buritica Delivery Matters Engineering performance affects an organization's ability to
achieve goals beyond profit and revenue.

@buritica Delivery Matters Whatever the mission, engineering performance can predict
overall organizational performance Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”

@buritica How does Splice Engineering become high- performing?

@buritica By driving the team towards a performance oriented culture

@buritica CONTINUOUS DELIVERY PERFORMANCE ORIENTED CULTURE LEAN PRODUCT MANAGEMENT DELIVERY
PERFORMANCE ORGANIZATIONAL PERFORMANCE Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.” culture drivers culture outcomes

@buritica To achieve this goal: We relentlessly pursue continuous improvement
to drive the Tech Team's culture towards high-performance by using a capabilities model proven to predict high performance in organizations.

@buritica Capability Categories Continuous Delivery Architecture Product & Process Lean
management & monitoring Cultural Nicole Forsgren PhD, Jez Humble & Gene Kim. “Accelerate.”

@buritica Continuous Delivery Architecture Product & Process Lean Mgt &
Monitoring Culture Version control for all artifacts Loosely coupled architecture Customer feedback implementation Lightweight change approval Support a generative culture Automated deployment Architect for empowered teams Visibility into flow of work through value stream Infrastructure monitoring Encourage and support learning Continuous integration Small batched work Proactive system health checks Support and facilitate cross-team collaboration Trunk-based development Enable experimentation Work In Progress limits Provide resources and tools that make work meaningful Test automation Visualize work to monitor quality Transformational leadership Test data management Integrated security Continuous delivery Capabilities

@buritica "I wish someone made a tool that helped me
measure my team easily" - @buritica

@buritica disclaimer: this is not a sponsored talk

@buritica

@buritica our working group has been successful, we'd like to
invest more sounds good Gorsuch, you got this reeeeleaaase the

@buritica Production Engineering Tooling, services, and expertise that enables teams
to deliver and operate high quality, production-level software and services.    Security Engineering  Site Reliability Engineering  Quality Reliability Engineering  Developer Experience (DevX)

@buritica Production Engineering Primary Metric: Cycle Time (commit to production)
Secondary Metrics:   Mean Time to Restore, Deploy Frequency, Time to First Commit

@buritica where are we now?

wtf cycle time

cycle time!

an accelerating organization

118 a steady rhythm

119 a performance oriented culture We have created a culture
of continuous systems and process improvement. We also have a language and a framework to measure this change.

@buritica what changes caused most impact?

@buritica Notable Changes • Quality Expansion & Acceleration • Focused
Quality Engs • Transferred Quality Ownership to Eng • New & Improved Testing Infra & Tools • Decentralized Staging Environment • Sub 2 min API Deploys • Stabilized Search Infrastructure • Decoupled Desktop UI • Feature Flags

@buritica what I learned

@buritica success: trust & change management

@buritica we can take a scientific approach as we build
our teams

@buritica I can use cycle time to strategize about investments

my team is

what's next?

Thank you @buritica

Increasing Engineering Tempo at Splice

Increasing Engineering Tempo at Splice

More Decks by Juan Pablo Buriticá

Other Decks in Technology

Featured

Transcript