Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators

Transcript

1. Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators Giovanni Grano,

   Fabio Palomba, Harald C. Gall http://tiny.uzh.ch/Zx @giograno90 34th IEEE/ACM International Conference on Automated Software Engineering 2019 (San Diego) software evolu tion & arch itectu re lab

2. effectiveness how do you measure it? detect bugs real-faults unknown

   ideal code-coverage proxy behavior

3. measuring effectiveness mutation testing if (a && b) c =

   c + 1; else c = 0; mutation score detected (killed) mutants generated mutants if (a || b) c = c + 1; else c = 0; mutants

4. powerful… …but expensive generation compilation execution do fewer do smarter

   do faster reduce the cost

5. powerful… …but expensive generation compilation execution do fewer do smarter

   do faster killed or not reduce the cost PMT

6. our approach estimate the mutation score generation compilation execution mutation

   score

7. source-code metrics test smells code smells relationship with fault-proneness of

   production code 67 metrics readability code coverage production/test metrics CK OO

8. feasibility study distribution low mutation score high mutation score 41

   metrics

9. machine learning problem binary classification 3 ML algorithms 2 models

   0 low score 1 high score random forest k-neighbors support vector machine all features only static

10. approach pairs

11. None
12. None
13. None

14. approach PIT pairs mutants metrics metrics effective 4th not effective

    1st score 3 * 2 ML models

15. results dynamic model static model random forest 0.949 AUC 0.864

    AUC estimation of the effectiveness without actually run any test

16. important factors line coverage others 0 0.175 0.35 0.525 0.7

    dynamic model mean decrease in impurity

17. important factors McCabe prod. RFC prod. WMC test LCOM1 test

    LCOM1 prod. 0 0.04 0.08 0.12 0.16 static model mean decrease in impurity

18. practical usage analytics dashboards

19. practical usage complementarity pairs prediction oops…not effective fine-grained investigation

20. software evolu tion & arch itectu re lab http://tiny.uzh.ch/Zx