Using Software Changes to Understand the Test Driven Development Process Michael Hilton @michaelhilton Nicholas Nelson Hugh McDonald Sean McDonald Ron Metoyer Danny Dig 

My Journey to XP 2 

My Journey to XP 3 

My Journey to XP 4 

My Journey to XP 5 

My Journey to XP 6 

My Journey to XP 7 

My Journey to XP 8 Corvallis OR -> PDX -> IAD -> MAN -> LHR -> EDI 

My Journey to XP 9 Corvallis OR -> PDX -> IAD -> MAN -> LHR -> EDI Was there turbulence? 

There exists many tools to evaluate the of our work, But there are very few tools to evaluate the of our work. 10 product process 

Process Improvement Matters Process improvement has led to: 11 85% defect reduction 50% development cycle time reduction 35% productivity improvement 

TDD is a Software Development Process 12 Rules Of Test Driven Development (TDD): 1. Create a unit test that fails 2. Write production code that makes that test pass 3. refactor Photo: http://blog.cleancoder.com/uncle-bob/2014/12/17/TheCyclesOfTDD.html 

Previous Work 13 Use static analysis at various points throughout the lifecycle of the code to determine characteristics of the process. For example: • Code Coverage • Cyclomatic Complexity • File size • etc. 

Our New Approach 14 Examine the changes between points in time by using the Abstract Syntax Tree (AST) to gain deeper understanding • What type of changes is it? • What can we infer from this type of change? 

Hive Plot 15 

Visualization to Understand TDD 16 

Visualization to Understand TDD 17 Kata Name Aggregate Hive Plot 

Visualization to Understand TDD 18 TDD Heartbeat Cycle Plot 

Visualization to Understand TDD 19 Snapshot Timeline 

Visualization to Understand TDD 20 Single TDD Cycle 

Visualization to Understand TDD 21 Single TDD Phase 

Visualization to Understand TDD 22 Single Snapshot 

Visualization to Understand TDD 23 Code Diffs 

Visualization to Understand TDD 24 

Inference Algorithm 25 

Evaluation 26 RQ1:Identification- Can programmers using TDDViz identify whether the code was developed in conformance with TDD? RQ2:Comprehension- Can programmers using TDDViz identify the reason why code was changed or inserted? RQ3:Comparability- Can programmers using TDDViz determine how much time went into testing vs. development of production code? RQ4: Accuracy- Can an algorithm infer TDD phases accurately? 

Results 27 RQ1: Identification Treatment Correct Not Correct Percent Correct Control (n=18) 37 34 52% Experimental (n=17) 55 13 80% RQ2: Comprehension Treatment Correct Not Correct Percent Correct Control (n=18) 24 48 33% Experimental (n=17) 55 13 62% 

Results 28 RQ3: Comparability Treatment Correct Not Correct Percent Correct Control (n=18) 23 49 32% Experimental (n=17) 33 35 48% 

RQ4: Accuracy 29 Total Corpus: 160 out of 2601 Sessions 2489 snapshots Manual markup of each snapshot in Gold Standard. Evaluate against the Gold Standard: Precision: 81% Recall: 95% Accuracy: 87% 

Contributions 30 • Process Understanding • TDD Visualization Design and Analysis • TDD Inference Algorithm • Implementation and Empirical Evaluation • Demo: github.com/MichaelHilton/TDDViz 

Where do we go from here? 31 Someone should develop a usable tool (spoiler: not me) 

Where do we go from here? 32 • Incorporate process review into existing mechanisms: • Continuous Integration • Code/Process Review Ask me about CI Research! 

There exists many tools to evaluate the of our work, But there are very few tools to evaluate the of our work. 33 product process