Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation

These are the slides for the talk I gave at ICSE 2020. I presented our IEEE Transaction on Software Engineering journal first paper (preprint https://arxiv.org/abs/1907.08578).

Abe475b76ff8b5fe24a0af18f03298f5?s=128

Giovanni Grano

July 08, 2020
Tweet

Transcript

  1. Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test

    Case Generation
 
 Giovanni Grano, Christoph Laaber, Annibale Panichella, Sebastiano Panichella IEEE Transaction on Software Engineering JF @ ICSE 2020
  2. testing activities high quality software Automated Test Case Generation

  3. testing activities high quality software expensive evolutionary search Automated Test

    Case Generation
  4. Coverage vs non-coverage criteria maximize code coverage criteria other desirable

    qualities
 (readability[1], code quality, 
 execution time, memory usage) code coverage non cov. criteria [1] Daka et.al - Modeling readability to improve unit tests (ESEC/FSE 2015)
  5. aDynaMOSA Execution time and heap memory consumption Harm the coverage

    Performance measurement 2 main challenges
  6. Performance Proxies proper measurement 
 unfeasible 7 performance proxies performance

    score 1000 times
  7. Algorithm algorithms population offsprings ranks crowding distance generation

  8. Algorithm algorithms population offsprings ranks crowding distance performance score generation

  9. Case Study 110 Java classes 27 different projects 3 research

    questions Can we achieve the same 
 branch coverage? Can we achieve the same 
 mutation score? Can we reduce test runtime and 
 heap memory consumption? DynaMOSA[2] [2] Panichella et.al - Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets
  10. RQ1 & RQ2 branch coverage mutation score 72 vs 72

    79 out of 110 32 vs 33 85 out of 110
  11. RQ1 & RQ2 branch coverage mutation score 72 vs 72

    79 out of 110 32 vs 33 85 out of 110 same achieved code coverage and mutation score
  12. RQ3 subjects no difference in coverage rigorous profiling runtime and

    memory consumption ~70% lower runtime and memory consumption
  13. RQ3 subjects no difference in coverage rigorous profiling runtime and

    memory consumption lower resource demand with same coverage ~70% lower runtime and memory consumption
  14. Conclusions aDynaMOSA - adaptive approach focus on tests resource demand

    different secondary objectives @giograno90 https://arxiv.org/abs/1907.08578