Gazer-Theta: LLVM-based Verifier Portfolio with BMC-CEGAR

Paper presented for the Competition on Software Verification (SV-COMP) at TACAS 2021. DOI: https://doi.org/10.1007/978-3-030-72013-1_27

Gazer-Theta is a software model checking toolchain including various analyses for state reachability. The frontend, namely Gazer, supports C programs through an LLVM-based transformation and optimization pipeline. Gazer includes an integrated bounded model checker (BMC) and can also employ the Theta backend, a generic verification framework based on abstraction-refinement (CEGAR). On SV-COMP 2021, a portfolio of BMC, explicit-value analysis, and predicate abstraction is applied sequentially in this order.

Transcript

1. Gazer-Theta: LLVM-based Verifier Portfolio with BMC/CEGAR (Competition Contribution) TACAS 2021, SV-Comp Zsófia Ádám1, Gyula Sallai2, Ákos Hajdu1 1 Budapest University of Technology and Economics, Budapest, Hungary 2 SonarSource S.A., Geneva, Switzerland This research has received funding from the EU ECSEL JU under the H2020 Framework Programme, JU grant nr. 826452 (Arrowhead Tools project) and from the partners’ national funding authorities.
2. Our verification tools Gazer: • C - frontend • C code to CFA • LLVM based • Backend: ◦ Own BMC engine ◦ Theta TACAS 2021, SV-Comp 1 Theta: • "Modular and configurable model checking framework" • Formal representations supported: STS/XSTS, XTA, XCFA, CFA • CEGAR-based analysis https://github.com/ftsrg/theta/ https://github.com/ftsrg/gazer/
3. TACAS 2021, SV-Comp 2 Verification process C Code Compiler (clang) LLVM passes theta CEGAR LLVM IR trace/witness test harness ✓ / ? / ✗ Z3 gazer BMC PRED EXPL Automata Translation Result interpreter
4. TACAS 2021, SV-Comp 3 Chosen configurations • Fast, solves many simpler tasks • Supports bitvectors, floats, recursive BMC • ECA (based on earlier results*) Theta EXPL • Good in general • For complex tasks (too much for BMC) Theta PRED * https://link.springer.com/content/pdf/10.1007%2Fs10817-019-09535-x.pdf
5. TACAS 2021, SV-Comp 4 The portfolio BMC analysis Explicit analysis Predicate analysis Execute cex Safe Unsafe Inconcl. C program + property 900s 150s 100s 150s Execute cex Execute cex 150s 150s
6. TACAS 2021, SV-Comp 4 The portfolio BMC analysis Explicit analysis Predicate analysis Execute cex Safe Unsafe Inconcl. C program + property 900s 150s 100s 150s Execute cex Execute cex 150s 150s
7. TACAS 2021, SV-Comp 4 The portfolio BMC analysis Explicit analysis Predicate analysis Execute cex Safe Unsafe Inconcl. C program + property 900s 150s 100s 150s Execute cex Execute cex 150s 150s Test harness (by Gazer) + Input file -> Executable Running executable and checking output Filtering false positives
8. TACAS 2021, SV-Comp 4 The portfolio BMC analysis Explicit analysis Predicate analysis Execute cex Safe Unsafe Inconcl. C program + property 900s 150s 100s 150s Execute cex Execute cex 150s 150s
9. TACAS 2021, SV-Comp 5 Results* Category No. of tasks Correct (%) BMC EXPL PRED XCSP 119 82% 97 0 0 Recursive 105 67% 70 0 0 Product Lines 597 92% 451 0 97 Loops 770 48% 333 3 31 Floats 469 59% 275 0 0 ECA 1265 23% 187 100 2 Control Flow 95 40% 38 0 0 Combinations 210 0% 0 0 0 Bitvectors 49 78% 38 0 0 Total 3679 47% 1489 103 130 * https://sv-comp.sosy-lab.org/2021/results/results-verified/gazer-theta.results.SV- COMP21.All.table.html#/
10. TACAS 2021, SV-Comp 5 Results 13 incorrect results in total, the rest are inconclusive: errors/timeouts/etc. Category No. of tasks Correct (%) BMC EXPL PRED XCSP 119 82% 97 0 0 Recursive 105 67% 70 0 0 Product Lines 597 92% 451 0 97 Loops 770 48% 333 3 31 Floats 469 59% 275 0 0 ECA 1265 23% 187 100 2 Control Flow 95 40% 38 0 0 Combinations 210 0% 0 0 0 Bitvectors 49 78% 38 0 0 Total 3679 47% 1489 103 130
11. Category No. of tasks Correct (%) BMC EXPL PRED XCSP 119 82% 97 0 0 Recursive 105 67% 70 0 0 Product Lines 597 92% 451 0 97 Loops 770 48% 333 3 31 Floats 469 59% 275 0 0 ECA 1265 23% 187 100 2 Control Flow 95 40% 38 0 0 Combinations 210 0% 0 0 0 Bitvectors 49 78% 38 0 0 Total 3679 47% 1489 103 130 TACAS 2021, SV-Comp 5 Results BMC did really well! (in unsafe and safe cases as well)
12. Category No. of tasks Correct (%) BMC EXPL PRED XCSP 119 82% 97 0 0 Recursive 105 67% 70 0 0 Product Lines 597 92% 451 0 97 Loops 770 48% 333 3 31 Floats 469 59% 275 0 0 ECA 1265 23% 187 100 2 Control Flow 95 40% 38 0 0 Combinations 210 0% 0 0 0 Bitvectors 49 78% 38 0 0 Total 3679 47% 1489 103 130 TACAS 2021, SV-Comp 5 Results Large CFAs, performance issues
13. Category No. of tasks Correct (%) BMC EXPL PRED XCSP 119 82% 97 0 0 Recursive 105 67% 70 0 0 Product Lines 597 92% 451 0 97 Loops 770 48% 333 3 31 Floats 469 59% 275 0 0 ECA 1265 23% 187 100 2 Control Flow 95 40% 38 0 0 Combinations 210 0% 0 0 0 Bitvectors 49 78% 38 0 0 Total 3679 47% 1489 103 130 TACAS 2021, SV-Comp Results 2nd place in Product Lines
14. TACAS 2021, SV-Comp Summary 5 https://github.com/ftsrg/theta/ https://github.com/ftsrg/gazer/