Refactoring Economics

Where: BENEVOL 2009
When: Thursday 17-12-2009
Abstract: We will study the economical aspect of refactoring by investigating what effect refactoring has on the costs of adding new functionality to a system and by looking at the costs of the refactoring activity itself. We expect that the costs of adding functionalities will be lower after the system was refactored. Indeed it is generally accepted that refactoring has a positive influence on the maintainability of a system. However this is purely based on anecdotal evidence and still needs to be reinforced by a thorough scientific and systematic investigation. In our research we will be conducting a series of case studies on some ``real-life'' systems. On these cases we will determine where and when they were refactored and what influence these refactorings had on the development costs of the systems.

Transcript

1. Refactoring €conomic$ An Empirical Study of Refactoring Effect on Development Costs Quinten David Soetens quinten.soetens@ua.ac.be Friday 11 December 2009 Hello my name is ... & I am here to present my position paper on Refactoring Economics. We will study the economical aspect of refactoring by looking at what the costs are of refactoring and what effect refactoring has on the costs of adding new functionality to an existing system.
2. [Lehman & Belady: Program Evolution: processes of software change (1985)] Lehman’s Laws Friday 11 December 2009 The largest part in the life cycle of a software engineering project is the “maintenance” part, of which a large part is the adding of new features. (click) Indeed as Lehmanʼs laws tell us a system must undergo continuing change or it becomes less and less useful. (click) However Lehmanʼs laws also tell us that as a system evolves it will also become more and more complex. So to counter this we need to (click) refactor the system.
3. [Lehman & Belady: Program Evolution: processes of software change (1985)] Lehman’s Laws I. Continuing Change: “A system must be continually adpated or it becomes progressively less satisfactory.” Friday 11 December 2009 The largest part in the life cycle of a software engineering project is the “maintenance” part, of which a large part is the adding of new features. (click) Indeed as Lehmanʼs laws tell us a system must undergo continuing change or it becomes less and less useful. (click) However Lehmanʼs laws also tell us that as a system evolves it will also become more and more complex. So to counter this we need to (click) refactor the system.
4. [Lehman & Belady: Program Evolution: processes of software change (1985)] Lehman’s Laws I. Continuing Change: “A system must be continually adpated or it becomes progressively less satisfactory.” II. Increasing Complexity: “As a system evolves its complexity increses unless work is done to maintain or reduce it.” Friday 11 December 2009 The largest part in the life cycle of a software engineering project is the “maintenance” part, of which a large part is the adding of new features. (click) Indeed as Lehmanʼs laws tell us a system must undergo continuing change or it becomes less and less useful. (click) However Lehmanʼs laws also tell us that as a system evolves it will also become more and more complex. So to counter this we need to (click) refactor the system.
5. [Lehman & Belady: Program Evolution: processes of software change (1985)] Lehman’s Laws I. Continuing Change: “A system must be continually adpated or it becomes progressively less satisfactory.” II. Increasing Complexity: “As a system evolves its complexity increses unless work is done to maintain or reduce it.” Solution = REFACTORING Friday 11 December 2009 The largest part in the life cycle of a software engineering project is the “maintenance” part, of which a large part is the adding of new features. (click) Indeed as Lehmanʼs laws tell us a system must undergo continuing change or it becomes less and less useful. (click) However Lehmanʼs laws also tell us that as a system evolves it will also become more and more complex. So to counter this we need to (click) refactor the system.
6. Problem! Does refactoring improve software maintainability? →Refactoring Economics! Friday 11 December 2009 Now it is generally agreed upon that refactoring improves the maintainability of a software system HOWEVER, this is up until now purely based on anecdotal evidence and still needs to be reenforced by a thorough scientific and systematic research. And we propose to do this by looking at Refactoring Economics.
7. Problem! •What are the costs of the individual refactoring steps? •What effect does refactoring have on the maintenance costs? •When will refactoring pay off its investment? Friday 11 December 2009 To study refactoring economics we ask ourselves these three questions: What effect does refactoring has on the costs of adding new features? What does refactoring itself cost? and if we know those two things we can look at whether or not refactoring can be a cost-effective operation.
8. GOALS: Empirically validate whether refactoring has a positive effect on the maintainability of a software system. $ After Refactoring  $ Before Refactoring Friday 11 December 2009 We have 2 goals that we wish to reach with this research. One is to show that refactoring has a positive effect on the costs of adding new features to a system. To do this we need to look at the costs of adding new features before a period of refactoring and the costs of adding new features after a period of refactoring. We reach this goal if we can show that the latter is substantially lower than the former.
9. GOALS: Examine if and when refactoring will pay off its investment $ Refactoring + $ Features ≤ ′ $ Features Friday 11 December 2009 The second goal we want to reach is to see whether refactoring pays off its investment. A refactoring pays of its investment if the cost of the refactoring + the costs of some features added after this refactoring is lower than the cost of adding those same features without first refactoring the system.
10. Experiments Iterations of case studies, increasing difficulty with each iteration Friday 11 December 2009 So to reach our goals and to answer our questions we will do some empirical experiments in 3 iterations. In the first iteration we will look at some small java cases with known refactorings. (easy cases) In the second iteration we will look at some cases in other programming languages (like C# or C++) and at cases with unknown refactorings. In the last iteration we will also look at some cases of a larger scale.
11. Experiments Cost of new functionalities: 1.Find refactoring period in CVS/ SVN 2.Determine average cost of new features for releases before & after refactoring period. 3.Project how average cost of adding new features changes over time (per release). Friday 11 December 2009 The first experiment deals with the first goal that we want to reach. So here we will see what effect refactoring has on the costs of adding new features. The first step is to find a refactoring period in the history of the system. By mining the software repository. Secondly we need to determine what the costs are of adding new features before and after this refactoring period. And thirdly we can then project how these costs change over time.
12. Time (Releases) Refactoring Period? Cost of adding features Expected Result: Friday 11 December 2009 We expect to see something like this, where the costs of adding features gradually increases over time Indeed Lehman’s second law teaches us that a system increases in complexity as the system evolves, so it seems logical that it becomes harder and thus more costly to add new features. After the refactoring period we expect to see a drop in the costs of new features. Indeed since refactoring is expected to lower the complexity of a system it seems logical that adding new features is easier and thus cheaper.
13. Experiments Payoff of Refactoring: •Will refactoring eventually produce a profit? •How many features need to be added before this profit is apparent? Look at features added after refactoring. Compare its cost estimates to estimated cost without refactoring. Friday 11 December 2009 A second experiment then deals with our second goal where we will investigate whether refactoring is a profitable activity. So here we will look at features that were added after a refactoring period. And then we will need to use some form of cost estimation to estimate what their cost would have been had there not been any refactoring.
14. Risks & Obstacles •Refactoring Detection •Functionality/Feature Location •Costs of Refactoring? •Costs of Feature? Friday 11 December 2009 Now there are still a few obstacles and risks that we need to overcome. Firstly: How do we find when and where refactorings were performed in the history of a system? We are currently working on this by doing a comparative study on some refactoring detection techniques. Secondly: How do we find which features were implemented when and where. Here we will probably need to look at the change-logs or the commit messages in the repositories. Or in some cases we can just ask the developers. And thirdly how do we measure the costs of refactoring and of features? Here we will probably need to use some form of cost model. Or in some cases we can just use the developers own cost measurements. (industrial cases)
15. How to deal with: •Developer Competence/ Background Knowledge •Team Structure •Feature Size/Complexity •Floss Refactoring vs Root canal Refactoring Risks & Obstacles Friday 11 December 2009 Some secondary risks or possible threats to the validity of our research should we choose not to take these into account include these: How do we deal with the fact that developers have different competences and background knowledges? It seems logical that a developer who has been working on the project longer can add new features more easily. (Student Experiment?) We should also take into account that features have different sizes and complexities. To deal with this we should look at some form of average. And lastly we say that we will look for refactoring periods, where refactoring is done in large blocks, however this isn’t usually the case. In most cases refactoring is done with a specific feature in mind (so called floss refactoring) and both these changes (the refactoring and the feature) could be committed to the repository at the same time.
16. Friday 11 December 2009 Yet hope is not lost, despite all these obstacles. If we manage to overcome them our research will be that more satisfactory.
17. In Short: Series of case studies to: (1) study costs of refactoring, (2) study effect refactoring has on costs of new features (3) and determine if refactoring was cost-effective. Friday 11 December 2009 So in a nutshell, what we will do is study the economical aspect of refactorings, by looking at the costs of refactoring themselves, by studying the effect refactoring has on the costs of adding new features to a system and then determine whether refactoring pays back its investment.