Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators

Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators

Presentation given at Automated Software Engineering (ASE) International Conference in San Diego, 2019.
I presented a IEEE Transaction on Software Engineering paper titled Lightweight Assessment of Test-Case Effectiveness using Source-Code-Quality Indicators.

Abe475b76ff8b5fe24a0af18f03298f5?s=128

Giovanni Grano

November 12, 2019
Tweet

Transcript

  1. Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators Giovanni Grano,

    Fabio Palomba, Harald C. Gall http://tiny.uzh.ch/Zx @giograno90 34th IEEE/ACM International Conference on Automated Software Engineering 2019 (San Diego) software evolu tion & arch itectu re lab
  2. effectiveness how do you measure it? detect bugs real-faults unknown

    ideal code-coverage proxy behavior
  3. measuring effectiveness mutation testing if (a && b) c =

    c + 1; else c = 0; mutation score detected (killed) mutants generated mutants if (a || b) c = c + 1; else c = 0; mutants
  4. powerful… …but expensive generation compilation execution do fewer do smarter

    do faster reduce the cost
  5. powerful… …but expensive generation compilation execution do fewer do smarter

    do faster killed or not reduce the cost PMT
  6. our approach estimate the mutation score generation compilation execution mutation

    score
  7. source-code metrics test smells code smells relationship with fault-proneness of

    production code 67 metrics readability code coverage production/test metrics CK OO
  8. feasibility study distribution low mutation score high mutation score 41

    metrics
  9. machine learning problem binary classification 3 ML algorithms 2 models

    0 low score 1 high score random forest k-neighbors support vector machine all features only static
  10. approach pairs

  11. None
  12. None
  13. None
  14. approach PIT pairs mutants metrics metrics effective 4th not effective

    1st score 3 * 2 ML models
  15. results dynamic model static model random forest 0.949 AUC 0.864

    AUC estimation of the effectiveness without actually run any test
  16. important factors line coverage others 0 0.175 0.35 0.525 0.7

    dynamic model mean decrease in impurity
  17. important factors McCabe prod. RFC prod. WMC test LCOM1 test

    LCOM1 prod. 0 0.04 0.08 0.12 0.16 static model mean decrease in impurity
  18. practical usage analytics dashboards

  19. practical usage complementarity pairs prediction oops…not effective fine-grained investigation

  20. software evolu tion & arch itectu re lab http://tiny.uzh.ch/Zx