Mechanisms for Improving Investment Efficiency through Continuous Delivery in Internet Services Masayoshi Oshima Recruit Co., Ltd PICMET2023 Prof. Naoshi Uchihira Japan Advanced Institute of Science and Technology 

Summary 2 • This research deals with Internet services. • This research focuses on the "Continuous Delivery (CD)" cycle. • The CD cycle refers to the cycle of delivering new features and improvements. • The purpose of this research is to identify the benefits and mechanisms of shortening CD cycles. Case Study Calculation model Calculate virtual case Consider the mechanism Impact on shortening CD cycles Priority decision logic Change CD Cycles 

Agenda • Background – Internet Services – Literature Review of Continuous Software Engineering – Literature Review of Continuous Delivery • Research Purpose & Method • Case Study : Process by which requirement are delivered • Calculation Model • Conclusion 3 

Agenda • Background – Internet Services – Literature Review of Continuous Software Engineering – Literature Review of Continuous Delivery • Research Purpose & Method • Case Study : Process by which requirement are delivered • Calculation Model • Conclusion 4 

Lead times continue to shorten. • Internet service providers develop products that are always evolving. • Google,FaceBook,Twitter,Yahoo,Youtube,Bing,etc • They are working to shorten lead times for new features and functional improvements to enhance their competition power. • This shortening trend is accelerating. • Months, weeks, days, hours,,,, • This continued cyclical development is called Contiinuous Delivery(CD). • Shortening lead time means shortening the CD cycle. • Various guides and tools based on agile development are available. • Divide development items to shorten the CD cycle. • Shorten the cycle by breaking up what can be released at once. • However split losses occur in testing, release work, etc. 5 

Software development for internet services. 1. The developed software itself is not provided to customers. • The offer is a service through the software. • The software exists on a server, not distributed physically. • It can be changed and improved at any time, easily. 2. The contract with the customer does not specify the development. • The software will be used if it holds value to the customer, otherwise, it won't be used. • There are no processes like contracts or approvals. • External stakeholders do not constitute a bottleneck in lead time. 3. Business revenue is generated by user actions, not from the sales of developed software. • Revenue is produced when users apply, purchase, or book through the service using the developed features. • If a feature is effective in driving customer actions, it provides semi- permanent profits. 4. The software never reaches a state of completion. • Their software is never completed. • As long as the business service continues, enhancements and improvements are continually made. 6 

Software that will not be completed • Traditional software engineering was about the knowledge necessary to complete software. • Methods to achieve a defined state of completion without failure. The premise is to complete. • Through a case study of Facebook, the software development of internet services was expressed as "never be complete.”. [ Feitelson et al. (2013) ] • There have been many proposals for ”Continuous – " methods, which are collectively named Continuous Software Engineering (SCE). [ Fitzgerald et al. (2015)] • Continuous Integration, Continuous Deploy, etc. • There have been several reports proposing to further expand and systematize the scope of this ”Continuous - " concept. [ Johanssen et al. (2018), Klotins et al. (2022)] • Continuous Security, Architecture, Delivery, Trust, Monitoring, Planning, Compliance, etc. • These are many in what we can call the “Continuous family”. • Traditional software engineering suggests a need for extension, as it lacks the "continuous" perspective. 7 

Continuous Delivery. • Among these "Continuous" Family, there is a domain called Continuous Delivery(CD), which deals with the continuous improvement and market supply of software. • In fact, CD are done in the field of Internet services. • There are guides to automate the series of operations such as build, test, and deploy for shortening the CD cycle.[ Humble et al. (2010)] • There are reports that CD cycles have been shortened to several months, weeks, days, or hours.[ Feitelson et al. (2013)] • Research on the difficulties of implementing CD is ongoing. • There are reports that organizational structure is a factor.[ Leppanen et al. (2015), Leite et al. (2021)] • There are reports that system architecture is a factor.[ Schermann et al. (2016)] • There are also reports that quality is a factor.[ Schermann et al. (2016)] • However, research addressing the benefits of CD is few, vague, and unclear. • Better to provide just-in-time to users. [ Rico et al. (2009)] • Being fast is important from a competitive standpoint. [ Schermann et al. (2016)] 8 

Agenda • Background – Internet Services – Literature Review of Continuous Software Engineering – Literature Review of Continuous Delivery • Research Purpose & Method • Case Study : Process by which requirement are delivered • Calculation Model • Conclusion 9 

Research Purpose and Methodology. • Many companies are working to shorten the CD cycle, and the benefits should exist as a practical reality. • However, it is difficult in terms of organizational structure, architecture, and quality. • Shortening the CD cycle also incurs the overhead of dividing development items. Research Purpose: • Quantitatively express the benefits of CD cycle shortening. • Identify the mechanism by which this benefit occurs. Research Method: • Through case studies, a calculation model will be constructed from the CD process. • From this calculation model, the impact and mechanism of shortening will be clarified. • Are the business benefits just "faster is better"? • Are there really impactful business benefits? 10 

Agenda • Background – Internet Services – Literature Review of Continuous Software Engineering – Literature Review of Continuous Delivery • Research Purpose & Method • Case Study : Process by which requirement are delivered • Calculation Model • Conclusion 11 

Case stutdy: About Recruit Co., Ltd. • A Japanese company founded in 1960. Originally a publisher of job magazines. • Called its business model the ribbon model, it expanded into the matching business for restaurants, hotels, real estate, beauty salon, wedding, and used automobiles etc. • Converted to digital business 20 years ago. Currently, the company is expanding globally with services such as "indeed". • Global sales of $20 Billion per year. • In Japan, they have more than 100 internet services. One of the largest companies in Japan. • Ribbon model, innovation creation, etc., is a case study of Harvard Business School. 12 

The Process by which requests are delivered Five steps to developing requirements: 1. Understand the requirements from Business dept. 2. Divide the requirements into meaningful units. 3. Estimate each divided unit in terms of return and development lead time. 4. Rearrange the priority order based on the “return profit” / ”lead time” calculation. 5. Develop according to priorities. *Steps 1-4 are continuously executed as requirements are dynamically added at any time. 13 

The Process by which requests are delivered • Interestingly, it uses lead time in deciding priorities. • It is a variant of Return Of Investment, but with Investment as time rather than cost. • "Continuous" is a concept of time, and it may be natural that time comes into play in deciding priorities. • The idea of a calculation model that deals with profit and time was inspired. ROI Return Profit / investment Money But Return Profit / investment Time 14 

Agenda • Background – Internet Services – Literature Review of Continuous Software Engineering – Literature Review of Continuous Delivery • Research Purpose & Method • Case Study : Process by which requirement are delivered • Calculation Model • Conclusion 15 

Calculation Model • We set up a very simple model. • One developer is doing the development. • There are a lot of development items that can be completed in a week. • When this item is delivered, it generates one point of profit each week. • Gray cells are in development period. • Blue cells are in profit period. • In this example, • One item is developed in Week 1. • It is delivered before Week 2. • It creates a profit of 1+1+1=3 points in Week2-4. 16 

Calculation Model • Item1 is delivered, item2 is developed. • Item1 is developed in Week1. • Item1 creates a profit of 3points in Week2-4. • Item2 is developed in Week2. • Item2 creates a profit of 2points in Week3-4. • Real-world development is complex. • There are multiple developers. • Multiple items are developed in parallel. 17 

Calculation Model • This is a figure to do for 12 weeks. • The profit to be made in 12 weeks will be 66 points. • This is the basic model concept. • The x-axis is time and the y-axis is development items. • The time developed is the gray cell. • The time created profit is the blue cell. • The profit made is the sum of the area of the blue cells. • It is similar to an integral calculation. 18 

Calculation Model • Change the CD cycle. • One item is developed in one week. • In a 2-week cycle, 2 items are developed each. • After this is delivered, two points of profit are created each week. • A change in the CD cycle changes the delivery timing. • In reality, testing and release operations are more efficient. • There is a divided loss by shortening the CD cycle. • This efficiency increase will be discussed later. 19 

Calculation Model 2 Week Cycle 1 Week Cycle 4 Week Cycle 3 Week Cycle 6 Week Cycle 5 Week Cycle Calculations were made by changing the CD cycle from 1-6 weeks. 20 

Calculation Result Profit Profitability 1Week Cycle 66 100% 2Week Cycle 60 91% 3Week Cycle 54 82% 4Week Cycle 48 73% 5Week Cycle 45 68% 6Week Cycle 36 55% • The total amount of development work is the same in all cases. • The gray cells are 12. • The largest gains are in the 1 Week cycle. • The least profitable is the 6 Week cycle. • The difference is 45% smaller for Week 6 than for Week 1. • The shorter the CD cycle term, the more profitable. • The longer the CD cycle term, the less profit gained. • The 12 development items are completed in all cases. • However, there is a 45% difference in profits earned. 45%Down 21 

Where does the difference in profits earned come from? 2 Week Cycle 1 Week Cycle 4 Week Cycle 3 Week Cycle 6 Week Cycle 5 Week Cycle The difference in profits earned is generated from the red triangle area. The longer the CD cycle, the more and more the red triangle expands. 22 

CD cycle and profits earned mechanism • What does the red triangle mean? • It means lost opportunity. • The size of the red triangle indicates the size of the opportunity loss. 23 • By shortening the CD cycle, "increase in profitability" is obtained. • The correct wording is "reduction of opportunity loss" rather than "increase in profitability". • Opportunity losses are reduced in 1-Day cycles than in 1-Week cycles. • Opportunity losses are reduced in 1-Hour cycles than in 1-Day cycles. 

Productivity Perspective Considerations Profit Profitab ility Productivity (profit/Week) Required Productivity Improvement 1Week Cycle 66 100% 5.5 100% 2Week Cycle 60 91% 5 110% 3Week Cycle 54 82% 4.5 122% 4Week Cycle 48 73% 4 138% 5Week Cycle 45 68% 3.75 147% 6Week Cycle 36 55% 3 183% • This section discusses split losses. • Shortening the CD cycle has the advantage of lowering opportunity losses. • On the other hand, there is a tradeoff in the form of increased testing and release work. • The tradeoff equilibrium point is the appropriate duration of the cycle period. • Needless to say, the result of creating 12 items is the same. • Therefore, what changes with the CD cycle is the Profit Productivity. • The required Profit productivity increase was calculated. • The calculations so far only take into account the reduction in opportunity loss. • Therefore, we calculate the rate of improvement that will result in the same Profit Productivity as 1Week. • For example, a 6-week period would require an 83% reduction in development time. • If 6 items are delivered in 3.3 weeks, the performance will be the same. • This is a very high hurdle. • It is more reasonable to shorten the CD cycle and lower the split loss with tools such as automated testing and automated deployment. 12Week -> 1week 24 +83% 

Agenda • Background – Internet Services – Literature Review of Continuous Software Engineering – Literature Review of Continuous Delivery • Research Purpose & Method • Case Study : Process by which requirement are delivered • Calculation Model • Conclusion 25 

Conclusion • The software development process is recognized as a cost-side issue. • However, the results of this research showed that it is a profit-side parameter. • It was shown to have a significant impact of 45% on profit. • It is mechanism can be explained by the reduction in lost opportunities. • Stakeholder and organizational resistance is reported as a challenge in implementing CD. • We believe that presenting the effects of short-term CD can contribute to facilitating decision-making. • Academic research on "Continuous" is also ongoing. • Conventional research has not been able to demonstrate the benefits quantitatively. • This research quantitatively demonstrates the benefits and proposes a method. • This research models a very simple development organization and process. The reality is complex and not at the level of simulating it. • We hope that the calculation model in this research will be helpful in advancing research from the perspective of "Continuous". 26 

Masayoshi Oshima Recruit Co., Ltd., HR Product development Division Division Officer And Japan Advanced Institute of Science and Technology Doctoral Course