Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators

Presentation given at Automated Software Engineering (ASE) International Conference in San Diego, 2019.
I presented a IEEE Transaction on Software Engineering paper titled Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators.

Transcript

1. Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators Giovanni Grano, Fabio Palomba, Harald C. Gall http://tiny.uzh.ch/Zx @giograno90 34th IEEE/ACM International Conference on Automated Software Engineering 2019 (San Diego) software evolu tion & arch itectu re lab
2. effectiveness how do you measure it? detect bugs real-faults unknown ideal code-coverage proxy behavior
3. measuring effectiveness mutation testing if (a && b) c = c + 1; else c = 0; mutation score detected (killed) mutants generated mutants if (a || b) c = c + 1; else c = 0; mutants
4. powerful… …but expensive generation compilation execution do fewer do smarter do faster reduce the cost
5. powerful… …but expensive generation compilation execution do fewer do smarter do faster killed or not reduce the cost PMT
6. our approach estimate the mutation score generation compilation execution mutation score
7. source-code metrics test smells code smells relationship with fault-proneness of production code 67 metrics readability code coverage production/test metrics CK OO
8. feasibility study distribution low mutation score high mutation score 41 metrics
9. machine learning problem binary classification 3 ML algorithms 2 models 0 low score 1 high score random forest k-neighbors support vector machine all features only static
10. approach pairs
14. approach PIT pairs mutants metrics metrics effective 4th not effective 1st score 3 * 2 ML models
15. results dynamic model static model random forest 0.949 AUC 0.864 AUC estimation of the effectiveness without actually run any test
16. important factors line coverage others 0 0.175 0.35 0.525 0.7 dynamic model mean decrease in impurity
17. important factors McCabe prod. RFC prod. WMC test LCOM1 test LCOM1 prod. 0 0.04 0.08 0.12 0.16 static model mean decrease in impurity
18. practical usage analytics dashboards
19. practical usage complementarity pairs prediction oops…not effective fine-grained investigation
20. software evolu tion & arch itectu re lab http://tiny.uzh.ch/Zx