Better Distributional Fit Does Not Improve Reachable Coverage Estimation in Fuzzing

ISSRE'26

Transcript

1. Better Distributional Fit Does Not Improve Reachable Coverage Estimation in Fuzzing Nelum Attanayake*, Danushka Liyanage, Hoang Nguyen, Suranga Seneviratne, Clement Canonne, Rahul Gopinath
2. What is Fuzzing? Seed Inputs Mutate Inputs Run Program Coverage Coverage feedback Covered blocks Uncovered blocks 2
3. When Should We Stop Fuzzing? Program ? Covered blocks Uncovered reachable blocks 3
4. Statistical View of Coverage Fuzzing Ecology Forest Program Observed species Covered blocks Unseen species Unseen reachable blocks Estimate total species 4 Estimate reachable coverage
5. Real-world Programs vs. Synthetic Parsers Real-world programs Synthetic parsers Realistic Controlled Unknown reachable coverage Known reachable coverage 5
6. Synthetic Parser Programs 6
7. Does a Better-Fitting Distribution Give a Better Estimator? RQ1: What distribution best fits coverage discovery in real programs? RQ2: Do synthetic parsers show similar coverage-discovery behaviour? RQ3: Do the best-fitting parametric models improve reachable-coverage estimation? 7
8. Study Design Subjects Real-world programs (7) Synthetic parser programs (5) Estimators Non-parametric estimators (5) Parametric estimators (6) 8
9. Reachable Coverage Estimators Non- parametric estimators Parametric estimators Poisson Chao2 Exponential Chao2_bc Gamma Jacknife 1 Gamma- Poisson Chiu Negative Binomial LanumteangBohning 9 Zipf- Mandelbrot
10. RQ1: What distribution best fits coverage discovery in real programs? Subject Model Jsoncpp Zipf- Mandelbrot Libtiff Zipf- Mandelbrot Libxml2 Gamma– Poisson Libpcap Zipf- Mandelbrot Jasper Zipf- Mandelbrot Freetype2 Zipf- Mandelbrot Zlib Zipf- Mandelbrot 6 of 7 subjects Zipf - Mandelbrot 10
11. RQ2: Do synthetic parsers show similar coverage-discovery behaviour? Real- world programs Synthetic programs Zipf- Mandelbrot – 6/7 Zipf- Mandelbrot – 2/5 Gamma– Poisson – 1/7 Gamma– Poisson – 3/5 11
12. Coverage Discovery is Highly Uneven 12
13. RQ3: Do the best-fitting parametric models improve reachable-coverage estimation? Parametric estimators Gamma- Poisson Zipf- Mandelbrot Non- parametric estimators ≈ Mean relative error Chao2 Chao2_bc Jacknife 1 Chiu Lanumteang-Bohning Mean relative errors differ by only 0.08 percentage points across all seven estimators 13
14. Better Fit ≠ Better Estimation 14
15. Summary 15
16. RQ3: Do the best-fitting parametric models improve reachable-coverage estimation? 12