Evaluating Impact of Coverage Feedback on Estimators for Maximum Reachability in Fuzzing (Registered Report)

Fuzzing'26

Transcript

1. Evaluating Impact of Coverage Feedback on Estimators for Maximum Reachability in Fuzzing (Registered Report) Authors: Nelum Attanayake*, Danushka Liyanage, Cl´ement Canonne, Suranga Seneviratne, Rahul Gopinath https://www.ndss-symposium.org/wp-content/uploads/fuzzing2026-10.pdf
2. 2 What is Fuzzing? Inputs Program function handle_request(req): # Region A: method check if req.method not in {"GET","POST"}: return # Region B: path check if not req.path.startswith("/api/"): return # Region C: authentication token if not valid_token(req.headers["Auth"]): return # Region D: admin endpoint if req.path == "/api/admin" and req.headers["Role"]=="root": crash() # Region U: unreachable if req.method=="GET" and req.method=="POST": unreachable()
3. 3 function handle_request(req): # Region A: method check if req.method not in {"GET","POST"}: return # Region B: path check if not req.path.startswith("/api/"): return # Region C: authentication token if not valid_token(req.headers["Auth"]): return # Region D: admin endpoint if req.path == "/api/admin" and req.headers["Role"]=="root": crash() # Region U: unreachable if req.method=="GET" and req.method=="POST": unreachable() Stop too early & Stop too Late Why Stopping Matters?
4. 4 Motivation: When to Stop Fuzzing?
5. 5 Assumption in Estimators Independent and Identically Distributed (i.i.d.) samples Draw —> Replace —> Shu ffl e —> Draw
6. 6 Problem: Adaptive Bias in Greybox Fuzzing 6 function handle_request(req): # Region A: method check if req.method not in {"GET","POST"}: return # Region B: path check if not req.path.startswith("/api/"): return # Region C: authentication token if not valid_token(req.headers["Auth"]): return # Region D: admin endpoint if req.path == "/api/admin" and req.headers["Role"]=="root": crash() # Region U: unreachable if req.method=="GET" and req.method=="POST": unreachable() METHOD: GET PATH: /x Auth: invalid_token Role: user Inputs Program
7. 7 Problem: Adaptive Bias in Greybox Fuzzing 7 function handle_request(req): # Region A: method check if req.method not in {"GET","POST"}: return # Region B: path check if not req.path.startswith("/api/"): return # Region C: authentication token if not valid_token(req.headers["Auth"]): return # Region D: admin endpoint if req.path == "/api/admin" and req.headers["Role"]=="root": crash() # Region U: unreachable if req.method=="GET" and req.method=="POST": unreachable() METHOD: GET PATH: /api/x Auth: invalid_token Role: user Inputs Program
8. 8 Problem: Adaptive Bias in Greybox Fuzzing 8 function handle_request(req): # Region A: method check if req.method not in {"GET","POST"}: return # Region B: path check if not req.path.startswith("/api/"): return # Region C: authentication token if not valid_token(req.headers["Auth"]): return # Region D: admin endpoint if req.path == "/api/admin" and req.headers["Role"]=="root": crash() # Region U: unreachable if req.method=="GET" and req.method=="POST": unreachable() METHOD: GET PATH: /api/admin Auth: TOKEN_OK Role: user Inputs Program
9. 9 Statistical View of Coverage Fuzzing Statistical View Coverage element Species Input Sample unit New block discovered New species observed Block A Block B Block C Singleton - f1 = 2 Doubleton - f2 = 1
10. 10 f1-f2 Equilibrium # Coverage Elements Number of Inputs / Sampling Unit (.104) f1 = f2 (Equilibrium) Stopping point
11. 11 f1-f2 equilibrium and Adaptive Bias Does the estimator provide reliable estimates at equilibrium?
12. 12 RQ1: Does the maximum reachable coverage estimates from blackbox and greybox fuzzing campaigns converge at f1f2 equilibrium? # Coverage Elements Number of Inputs / Sampling Unit }Gap at equilibrium
13. 13 RQ2: Are maximum reachable coverage estimators accurate at the f1f2 equilibrium when evaluated against ground truth? # Coverage Elements Number of Inputs / Sampling Unit } Error at equilibrium
14. 14 Experimental Design Experiment 01 (RQ1) Experiment 02 (RQ2) Fuzzer AFL++ (Greybox, Blackbox mode) AFL++ (Greybox, Blackbox mode) Subjects Real-world programs Synthetic programs with known coverage reachability Estimators Standard non-parametric estimators Standard non- parametric estimators
15. 15 Aspect RQ1:Convergence RQ2: Accuracy What are we comparing? Greybox vs Blackbox estimates Estimate vs Ground Truth Main goal Do they agree at the equilibrium point? Is the estimate correct? Metrices 1. Bias 1. Bias
 2. Variance
 3. Con fi dence interval coverage Evaluation Metrics
16. 16 Why is this study important? •Practical guidance on when to stop fuzzing Practical Impact •Empirically evaluates estimator reliability Empirical Evidence
17. 17 Summary