Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation

These are the slides for the talk I gave at ICSE 2020. I presented our IEEE Transaction on Software Engineering journal first paper (preprint https://arxiv.org/abs/1907.08578).

Giovanni Grano

July 08, 2020
Tweet

More Decks by Giovanni Grano

Other Decks in Science

Transcript

  1. Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test

    Case Generation
 
 Giovanni Grano, Christoph Laaber, Annibale Panichella, Sebastiano Panichella IEEE Transaction on Software Engineering JF @ ICSE 2020
  2. Coverage vs non-coverage criteria maximize code coverage criteria other desirable

    qualities
 (readability[1], code quality, 
 execution time, memory usage) code coverage non cov. criteria [1] Daka et.al - Modeling readability to improve unit tests (ESEC/FSE 2015)
  3. Case Study 110 Java classes 27 different projects 3 research

    questions Can we achieve the same 
 branch coverage? Can we achieve the same 
 mutation score? Can we reduce test runtime and 
 heap memory consumption? DynaMOSA[2] [2] Panichella et.al - Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets
  4. RQ1 & RQ2 branch coverage mutation score 72 vs 72

    79 out of 110 32 vs 33 85 out of 110
  5. RQ1 & RQ2 branch coverage mutation score 72 vs 72

    79 out of 110 32 vs 33 85 out of 110 same achieved code coverage and mutation score
  6. RQ3 subjects no difference in coverage rigorous profiling runtime and

    memory consumption ~70% lower runtime and memory consumption
  7. RQ3 subjects no difference in coverage rigorous profiling runtime and

    memory consumption lower resource demand with same coverage ~70% lower runtime and memory consumption
  8. Conclusions aDynaMOSA - adaptive approach focus on tests resource demand

    different secondary objectives @giograno90 https://arxiv.org/abs/1907.08578