TDDViz, understanding development processes

This presentation was given at #XP2016.

Transcript

1. Using Software Changes to Understand the Test Driven Development Process Michael Hilton @michaelhilton Nicholas Nelson Hugh McDonald Sean McDonald Ron Metoyer Danny Dig
2. My Journey to XP 2
3. My Journey to XP 3
4. My Journey to XP 4
5. My Journey to XP 5
6. My Journey to XP 6
7. My Journey to XP 7
8. My Journey to XP 8 Corvallis OR -> PDX -> IAD -> MAN -> LHR -> EDI
9. My Journey to XP 9 Corvallis OR -> PDX -> IAD -> MAN -> LHR -> EDI Was there turbulence?
10. There exists many tools to evaluate the of our work, But there are very few tools to evaluate the of our work. 10 product process
11. Process Improvement Matters Process improvement has led to: 11 85% defect reduction 50% development cycle time reduction 35% productivity improvement
12. TDD is a Software Development Process 12 Rules Of Test Driven Development (TDD): 1. Create a unit test that fails 2. Write production code that makes that test pass 3. refactor Photo: http://blog.cleancoder.com/uncle-bob/2014/12/17/TheCyclesOfTDD.html
13. Previous Work 13 Use static analysis at various points throughout the lifecycle of the code to determine characteristics of the process. For example: • Code Coverage • Cyclomatic Complexity • File size • etc.
14. Our New Approach 14 Examine the changes between points in time by using the Abstract Syntax Tree (AST) to gain deeper understanding • What type of changes is it? • What can we infer from this type of change?
15. Hive Plot 15
16. Visualization to Understand TDD 16
17. Visualization to Understand TDD 17 Kata Name Aggregate Hive Plot
18. Visualization to Understand TDD 18 TDD Heartbeat Cycle Plot
19. Visualization to Understand TDD 19 Snapshot Timeline
20. Visualization to Understand TDD 20 Single TDD Cycle
21. Visualization to Understand TDD 21 Single TDD Phase
22. Visualization to Understand TDD 22 Single Snapshot
23. Visualization to Understand TDD 23 Code Diffs
24. Visualization to Understand TDD 24
25. Inference Algorithm 25
26. Evaluation 26 RQ1:Identification- Can programmers using TDDViz identify whether the code was developed in conformance with TDD? RQ2:Comprehension- Can programmers using TDDViz identify the reason why code was changed or inserted? RQ3:Comparability- Can programmers using TDDViz determine how much time went into testing vs. development of production code? RQ4: Accuracy- Can an algorithm infer TDD phases accurately?
27. Results 27 RQ1: Identification Treatment Correct Not Correct Percent Correct Control (n=18) 37 34 52% Experimental (n=17) 55 13 80% RQ2: Comprehension Treatment Correct Not Correct Percent Correct Control (n=18) 24 48 33% Experimental (n=17) 55 13 62%
28. Results 28 RQ3: Comparability Treatment Correct Not Correct Percent Correct Control (n=18) 23 49 32% Experimental (n=17) 33 35 48%
29. RQ4: Accuracy 29 Total Corpus: 160 out of 2601 Sessions 2489 snapshots Manual markup of each snapshot in Gold Standard. Evaluate against the Gold Standard: Precision: 81% Recall: 95% Accuracy: 87%
30. Contributions 30 • Process Understanding • TDD Visualization Design and Analysis • TDD Inference Algorithm • Implementation and Empirical Evaluation • Demo: github.com/MichaelHilton/TDDViz
31. Where do we go from here? 31 Someone should develop a usable tool (spoiler: not me)
32. Where do we go from here? 32 • Incorporate process review into existing mechanisms: • Continuous Integration • Code/Process Review Ask me about CI Research!
33. There exists many tools to evaluate the of our work, But there are very few tools to evaluate the of our work. 33 product process