How Does Yahoo! JAPAN Balance Development Speed and Service Quality?

Kanako Yasunaga (Yahoo! JAPAN / Technical Support Group, System Management Group, Technology Group / System Engineer)

https://tech-verse.me/ja/sessions/196
https://tech-verse.me/en/sessions/196
https://tech-verse.me/ko/sessions/196

Transcript

1. How Does Yahoo! JAPAN Balance Development Speed and Service Quality? Kanako Yasunaga / Yahoo! JAPAN
2. Our Vision Software development in the recent years requires a speedy response to business changes. What is of importance is how quickly we can provide a new value to the customers without a drop in the quality of services. In this presentation, we will introduce examples of measurement and visualization of development speed and service quality, as well as improving the two in a well-balanced manner.
3. Self - introduction 2004 Service development and operation Yahoo! Beauty / Yahoo! Ticket / Yahoo! BB Yahoo! Premium 2015 Promotion of platform development Company-wide Jenkins / Screwdriver / Splunk / Integrated console / In-house technical documentation / In-house community launch / In-house technical conference 2020 5FDIOPMPHZ
   
   4PMVUJPO
   %JWJTJPO Productivity visualization PF system development Analysis Kanako Yasunaga
4. Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
5. No Measurement, No Improvement 〜 William Thomson 〜
6. Measurement [1] DORA's research program, [2] Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ※SSR is an abbreviation for site change success rate, a specification unique to Yahoo! JAPAN. four key metrics Development Speed Service Quality
7. Measurement Change Failure Rate (SSR※) MTTR (Mean Time To Repair ) [1] DORA's research program, [2] Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations ※SSR is an abbreviation for site change success rate, a specification unique to Yahoo! JAPAN. Deploy Frequency Change Lead Time four key metrics Development Speed Service Quality
8. How to measure coding build integrate test deploy operate improve Change Lead Time Deploy Frequency
9. How to measure coding build integrate test deploy operate improve Change Lead Time Deploy Frequency Change Failure Rate MTTR
10. Change Lead Time low high early late *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
11. Deploy Frequency low high low high *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
12. MTTR ( Mean Time To Repair ) low high early late *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
13. Change Failure Rate SSR ※ SSR stands for Site change Success Rate low high low high *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
14. Development Speed vs Service Quality Photo: Afro
15. Cluster Deploy Frequency Change Lead Time MTTR Change Failure Rate (SSR ※） 1 month or more 1 month or more 1 week or more 45%〜 Once a week ~ once a month 1 week~ 1 month １day〜 １week 30％〜45% At least once a week Less than a week Less than 1 day 0%〜30% Low Medium High ※ SSR stands for Site change Success Rate [3] Accelerate State of DevOps 2019
16. Take a Balance Development Speed Service Quality High High Low Low Low Medium High *Survey results of 371 products (421 systems) Conditions during (October- December 2020)
17. Photo: Afro Visualize
18. Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
19. Service Characteristics Billing system Test Automation Photo: Afro
20. Lifecycle Startup Deployment Automation Photo: Afro
21. Variations Photo: Afro
22. Best Practices Master base development Artifact management Deployment automation Test Automation Test data management Loose coupling
23. 下位ランク 中位ランク 上位ランク Best Practices Setting Content Master Base Development Is operated based on the best branch model (git flow, GitHub Flow and others) for the team. Code review is implemented for merging to branches (master branch and others) that will be the development base. Tests are implemented before merging to branches (master branch and others) that will be the development base. Branch survivability period is adequately short (generally within two weeks) Uses configuration management tools (Artifactory, Docker Registry, Chef and others) for packages, images, configuration management. Artifact Management Packages use CI/CD tools (Screwdriver.cd and others) Uses the same package, image and configuration management in all environments. Deployment Automation Deployment procedures are defined. Deployment procedures are the same for all environments. Deployment is automated. Deployment to production environment is implemented by triggering a merge to the branch to be the base for development. Deployment failure is automatically rolled back. Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Development Habits Survey クラス毎の統計データ 下位ランク 中位ランク 上位ランク [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Test Data Management Test data are all managed by version. Loose Coupling System coupling relationship is clear. Scope of effects caused by changes is clear. Incorporates initiatives for loose coupling. (Maintains backward compatibility, includes circuit breaker, and others) Adjustments caused by changes are under their own control. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
24. Development Habits Survey クラス毎の統計データ [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes
25. Development Habits Survey クラス毎の統計データ [Survey question contents and cluster classification] *Survey results of 371 products (421 systems) Conditions during (October- December 2020) Test Automation Unit tests are automated. Tests are implemented using the CI/CD tools. Integrated tests are automated in confidence if deployment to production environment is ready. Best Practices Setting Content Low Medium High Yes Yes Yes Yes Yes Yes Yes Yes Yes
26. Photo: Afro Basic Practices
27. Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
28. Four Steps for Improvement Methods to improve development productivity 1. Know your own product numbers 2. Know your own product development habits 3. Comparison with high class 4. Improvements along with usual operations Photo: Afro
29. 1.Know your own product numbers Change Failure Rate 0% Deploy Frequency 0.66回/月 MTTR 0% Change Lead Time 213.05 H Low Medium High High Development Speed Service Quality
30. 改善前 1.Know your own product numbers Development Speed Service Quality High High Low Low Low
31. 2. Know your own product development habits x Master base development Artifact management Deployment automation Test Automation Test data management Loose coupling My product
32. 3. Comparison with High class x Master base development Artifact management Deployment automation Test Automation Test data management Loose coupling High product My product
33. 4. Improvements along with usual operations Artifact Management Deployment Automation Test Automation Easy Difficult Easy
34. 4. Improvements along with usual operations Uses the same package, image and configuration management in all environments.
35. 4. Improvements along with usual operations Deployment procedures are the same for all environments. Deployment is automated.
36. 4. Improvements along with usual operations Artifact Management Deployment Automation Test Automation Easy Difficult Easy
37. Quantitative Change Change Failure Rate 0% Deploy Frequency 0.66回/月 8回/月 MTTR 0% Change Lead Time 213.05 H 134.4H Low Medium High High High High Development Speed Service Quality
38. Before and After Improvement 改善前 改善後 Development Speed Service Quality High High Low Low High High Low Low Low High 6 months Service Quality Development Speed
39. Looks like a Diet Photo: Afro
40. Agenda - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
41. From Tacit to Explicit Tacit knowledge Explicit knowledge Photo: Afro
42. Knowledge Management Socialization Externalization Internalization Combination Tacit Tacit Explicit Explicit [4] The Knowledge Creating Company ( Knowledge Management (SECI Model) )
43. Socialization Socialization To share with others Visualization Questionnaire
44. Externalization Externalization To transform into a concrete form Case study
45. Combination Combination Guidelines Shaping into final form
46. Internalization Internalization Conference Personal tacit knowledge
47. Team Power Photo: Afro
48. Summary - Quantification of development speed and service quality - Finding effective development habits for numerical improvement - Actions for improvement - From tacit to explicit knowledge
49. - Quantification of development speed and service quality
50. - Finding effective development habits for numerical improvement
51. - Actions for improvement
52. - From tacit to explicit knowledge
53. Development Speed and Service Quality
54. Thank you