Tweet
Tweet

Slide 1

Slide 1 text

How Does Yahoo! JAPAN Balance Development Speed and Service Quality? Kanako Yasunaga / Yahoo! JAPAN

Slide 2

Slide 2 text

Our Vision Software development in the recent years requires a speedy response to business changes. What is of importance is how quickly we can provide a new value to the customers without a drop in the quality of services. In this presentation, we will introduce examples of measurement and visualization of development speed and service quality, as well as improving the two in a well-balanced manner.

Slide 3

Slide 3 text

Self - introduction 2004 Service development and operation Yahoo! Beauty / Yahoo! Ticket / Yahoo! BB Yahoo! Premium 2015 Promotion of platform development Company-wide Jenkins / Screwdriver / Splunk / Integrated console / In-house technical documentation / In-house community launch / In-house technical conference 2020 5FDIOPMPHZ4PMVUJPO%JWJTJPO Productivity visualization PF system development Analysis Kanako Yasunaga

Slide 4

Slide 4 text

Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge

Slide 5

Slide 5 text

No Measurement, No Improvement 〜 William Thomson 〜

Slide 6

Slide 6 text

Measurement [1] DORA's research program, [2] Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ※SSR is an abbreviation for site change success rate, a specification unique to Yahoo! JAPAN. four key metrics Development Speed Service Quality

Slide 7

Slide 7 text

Measurement Change Failure Rate (SSR※) MTTR (Mean Time To Repair ) [1] DORA's research program, [2] Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ※SSR is an abbreviation for site change success rate, a specification unique to Yahoo! JAPAN. Deploy Frequency Change Lead Time four key metrics Development Speed Service Quality

Slide 8

Slide 8 text

How to measure coding build integrate test deploy operate improve Change Lead Time Deploy Frequency

Slide 9

Slide 9 text

How to measure coding build integrate test deploy operate improve Change Lead Time Deploy Frequency Change Failure Rate MTTR

Slide 10

Slide 10 text

Change Lead Time low high early late *Survey results of 371 products (421 systems) Conditions during (October- December 2020)

Slide 11

Slide 11 text

Deploy Frequency low high low high *Survey results of 371 products (421 systems) Conditions during (October- December 2020)

Slide 12

Slide 12 text

MTTR ( Mean Time To Repair ) low high early late *Survey results of 371 products (421 systems) Conditions during (October- December 2020)

Slide 13

Slide 13 text

Change Failure Rate SSR ※ SSR stands for Site change Success Rate low high low high *Survey results of 371 products (421 systems) Conditions during (October- December 2020)

Slide 14

Slide 14 text

Development Speed vs Service Quality Photo: Afro

Slide 15

Slide 15 text

Cluster Deploy Frequency Change Lead Time MTTR Change Failure Rate (SSR ※） 1 month or more 1 month or more 1 week or more 45%〜 Once a week ~ once a month 1 week~ 1 month １day〜 １week 30％〜45% At least once a week Less than a week Less than 1 day 0%〜30% Low Medium High ※ SSR stands for Site change Success Rate [3] Accelerate State of DevOps 2019

Slide 16

Slide 16 text

Take a Balance Development Speed Service Quality High High Low Low Low Medium High *Survey results of 371 products (421 systems) Conditions during (October- December 2020)

Slide 17

Slide 17 text

Photo: Afro Visualize

Slide 18

Slide 18 text

Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge

Slide 19

Slide 19 text

Service Characteristics Billing system Test Automation Photo: Afro

Slide 20

Slide 20 text

Lifecycle Startup Deployment Automation Photo: Afro

Slide 21

Slide 21 text

Variations Photo: Afro

Slide 22

Slide 22 text

Best Practices Master base development Artifact management Deployment automation Test Automation Test data management Loose coupling

Slide 23

Slide 23 text

下位ランク 中位ランク 上位ランク Best Practices Setting Content Master Base Development Is operated based on the best branch model (git flow, GitHub Flow and others) for the team. Code review is implemented for merging to branches (master branch and others) that will be the development base. Tests are implemented before merging to branches (master branch and others) that will be the development base. Branch survivability period is adequately short (generally within two weeks) Uses configuration management tools (Artifactory, Docker Registry, Chef and others) for packages, images, configuration management. Artifact Management Packages use CI/CD tools (Screwdriver.cd and others) Uses the same package, image and configuration management in all environments. Deployment Automation Deployment procedures are defined. Deployment procedures are the same for all environments. Deployment is automated. Deployment to production environment is implemented by triggering a merge to the branch to be the base for development. Deployment failure is automatically rolled back. Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Development Habits Survey クラス毎の統計データ 下位ランク 中位ランク 上位ランク [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Test Data Management Test data are all managed by version. Loose Coupling System coupling relationship is clear. Scope of effects caused by changes is clear. Incorporates initiatives for loose coupling. (Maintains backward compatibility, includes circuit breaker, and others) Adjustments caused by changes are under their own control. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

Slide 24

Slide 24 text

Development Habits Survey クラス毎の統計データ [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes

Slide 25

Slide 25 text

Development Habits Survey クラス毎の統計データ [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes

Slide 26

Slide 26 text

Photo: Afro Basic Practices

Slide 27

Slide 27 text

Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge

Slide 28

Slide 28 text

Four Steps for Improvement Methods to improve development productivity 1. Know your own product numbers 2. Know your own product development habits 3. Comparison with high class 4. Improvements along with usual operations Photo: Afro

Slide 29

Slide 29 text

1.Know your own product numbers Change Failure Rate 0% Deploy Frequency 0.66回/月 MTTR 0% Change Lead Time 213.05 H Low Medium High High Development Speed Service Quality

Slide 30

Slide 30 text

改善前 1.Know your own product numbers Development Speed Service Quality High High Low Low Low

Slide 31

Slide 31 text

2. Know your own product development habits x Master base development Artifact management Deployment automation Test Automation Test data management Loose coupling My product

Slide 32

Slide 32 text

3. Comparison with High class x Master base development Artifact management Deployment automation Test Automation Test data management Loose coupling High product My product

Slide 33

Slide 33 text

4. Improvements along with usual operations Artifact Management Deployment Automation Test Automation Easy Difficult Easy

Slide 34

Slide 34 text

4. Improvements along with usual operations Uses the same package, image and configuration management in all environments.

Slide 35

Slide 35 text

4. Improvements along with usual operations Deployment procedures are the same for all environments. Deployment is automated.

Slide 36

Slide 36 text

4. Improvements along with usual operations Artifact Management Deployment Automation Test Automation Easy Difficult Easy

Slide 37

Slide 37 text

Quantitative Change Change Failure Rate 0% Deploy Frequency 0.66回/月 8回/月 MTTR 0% Change Lead Time 213.05 H 134.4H Low Medium High High High High Development Speed Service Quality

Slide 38

Slide 38 text

Before and After Improvement 改善前 改善後 Development Speed Service Quality High High Low Low High High Low Low Low High 6 months Service Quality Development Speed

Slide 39

Slide 39 text

Looks like a Diet Photo: Afro

Slide 40

Slide 40 text

Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge

Slide 41

Slide 41 text

From Tacit to Explicit Tacit knowledge Explicit knowledge Photo: Afro

Slide 42

Slide 42 text

Knowledge Management Socialization Externalization Internalization Combination Tacit Tacit Explicit Explicit [4] The Knowledge Creating Company ( Knowledge Management (SECI Model) )

Slide 43

Slide 43 text

Socialization Socialization To share with others Visualization Questionnaire

Slide 44

Slide 44 text

Externalization Externalization To transform into a concrete form Case study

Slide 45

Slide 45 text

Combination Combination Guidelines Shaping into final form

Slide 46

Slide 46 text

Internalization Internalization Conference Personal tacit knowledge

Slide 47

Slide 47 text

Team Power Photo: Afro

Slide 48

Slide 48 text

Summary - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge

Slide 49

Slide 49 text

- Quantification of development speed and service quality

Slide 50

Slide 50 text

- Finding effective development habits for numerical improvement

Slide 51

Slide 51 text

- Actions for improvement

Slide 52

Slide 52 text

- From tacit to explicit knowledge

Slide 53

Slide 53 text

Development Speed and Service Quality

Slide 54

Slide 54 text

Thank you