Gazer-Theta: LLVM-based Verifier Portfolio with BMC-CEGAR

Paper presented for the Competition on Software Verification (SV-COMP) at TACAS 2021. DOI: https://doi.org/10.1007/978-3-030-72013-1_27

Gazer-Theta is a software model checking toolchain including various analyses for state reachability. The frontend, namely Gazer, supports C programs through an LLVM-based transformation and optimization pipeline. Gazer includes an integrated bounded model checker (BMC) and can also employ the Theta backend, a generic verification framework based on abstraction-refinement (CEGAR). On SV-COMP 2021, a portfolio of BMC, explicit-value analysis, and predicate abstraction is applied sequentially in this order.

Transcript

1. Gazer-Theta: LLVM-based Verifier
   Portfolio with BMC/CEGAR
   (Competition Contribution)
   TACAS 2021, SV-Comp
   Zsófia Ádám1, Gyula Sallai2, Ákos Hajdu1
   1 Budapest University of Technology and Economics, Budapest, Hungary
   2 SonarSource S.A., Geneva, Switzerland
   This research has received funding from the EU ECSEL JU under the H2020 Framework Programme, JU grant nr. 826452
   (Arrowhead Tools project) and from the partners’ national funding authorities.
   

   View Slide

2. Our verification tools
   Gazer:
   • C - frontend
   • C code to CFA
   • LLVM based
   • Backend:
   ◦ Own BMC engine
   ◦ Theta
   TACAS 2021, SV-Comp 1
   Theta:
   • "Modular and configurable
   model checking framework"
   • Formal representations
   supported: STS/XSTS, XTA,
   XCFA, CFA
   • CEGAR-based analysis
   https://github.com/ftsrg/theta/
   https://github.com/ftsrg/gazer/
   

   View Slide

3. TACAS 2021, SV-Comp 2
   Verification process
   C
   Code
   Compiler
   (clang)
   LLVM
   passes
   theta
   CEGAR
   LLVM
   IR
   trace/witness
   test harness
   ✓ / ? / ✗
   Z3
   gazer BMC
   PRED EXPL
   Automata
   Translation
   Result interpreter
   

   View Slide

4. TACAS 2021, SV-Comp 3
   Chosen configurations
   • Fast, solves many simpler tasks
   • Supports bitvectors, floats, recursive
   BMC
   • ECA (based on earlier results*)
   Theta EXPL
   • Good in general
   • For complex tasks (too much for BMC)
   Theta PRED
   * https://link.springer.com/content/pdf/10.1007%2Fs10817-019-09535-x.pdf
   

   View Slide

5. TACAS 2021, SV-Comp 4
   The portfolio
   BMC
   analysis
   Explicit
   analysis
   Predicate
   analysis
   Execute
   cex
   Safe
   Unsafe
   Inconcl.
   C program
   + property
   900s
   150s 100s
   150s
   Execute
   cex
   Execute
   cex
   150s 150s
   

   View Slide

6. TACAS 2021, SV-Comp 4
   The portfolio
   BMC
   analysis
   Explicit
   analysis
   Predicate
   analysis
   Execute
   cex
   Safe
   Unsafe
   Inconcl.
   C program
   + property
   900s
   150s 100s
   150s
   Execute
   cex
   Execute
   cex
   150s 150s
   

   View Slide

7. TACAS 2021, SV-Comp 4
   The portfolio
   BMC
   analysis
   Explicit
   analysis
   Predicate
   analysis
   Execute
   cex
   Safe
   Unsafe
   Inconcl.
   C program
   + property
   900s
   150s 100s
   150s
   Execute
   cex
   Execute
   cex
   150s 150s
   Test harness (by Gazer) + Input file -> Executable
   Running executable and checking output
   Filtering false positives
   

   View Slide

8. TACAS 2021, SV-Comp 4
   The portfolio
   BMC
   analysis
   Explicit
   analysis
   Predicate
   analysis
   Execute
   cex
   Safe
   Unsafe
   Inconcl.
   C program
   + property
   900s
   150s 100s
   150s
   Execute
   cex
   Execute
   cex
   150s 150s
   

   View Slide

9. TACAS 2021, SV-Comp 5
   Results*
   Category No. of tasks Correct (%) BMC EXPL PRED
   XCSP 119 82% 97 0 0
   Recursive 105 67% 70 0 0
   Product Lines 597 92% 451 0 97
   Loops 770 48% 333 3 31
   Floats 469 59% 275 0 0
   ECA 1265 23% 187 100 2
   Control Flow 95 40% 38 0 0
   Combinations 210 0% 0 0 0
   Bitvectors 49 78% 38 0 0
   Total 3679 47% 1489 103 130
   * https://sv-comp.sosy-lab.org/2021/results/results-verified/gazer-theta.results.SV-
   COMP21.All.table.html#/
   

   View Slide

10. TACAS 2021, SV-Comp 5
    Results
    13 incorrect results in total, the rest are inconclusive:
    errors/timeouts/etc.
    Category No. of tasks Correct (%) BMC EXPL PRED
    XCSP 119 82% 97 0 0
    Recursive 105 67% 70 0 0
    Product Lines 597 92% 451 0 97
    Loops 770 48% 333 3 31
    Floats 469 59% 275 0 0
    ECA 1265 23% 187 100 2
    Control Flow 95 40% 38 0 0
    Combinations 210 0% 0 0 0
    Bitvectors 49 78% 38 0 0
    Total 3679 47% 1489 103 130
    

    View Slide

11. Category No. of tasks Correct (%) BMC EXPL PRED
    XCSP 119 82% 97 0 0
    Recursive 105 67% 70 0 0
    Product Lines 597 92% 451 0 97
    Loops 770 48% 333 3 31
    Floats 469 59% 275 0 0
    ECA 1265 23% 187 100 2
    Control Flow 95 40% 38 0 0
    Combinations 210 0% 0 0 0
    Bitvectors 49 78% 38 0 0
    Total 3679 47% 1489 103 130
    TACAS 2021, SV-Comp 5
    Results BMC did really well!
    (in unsafe and safe cases as well)
    

    View Slide

12. Category No. of tasks Correct (%) BMC EXPL PRED
    XCSP 119 82% 97 0 0
    Recursive 105 67% 70 0 0
    Product Lines 597 92% 451 0 97
    Loops 770 48% 333 3 31
    Floats 469 59% 275 0 0
    ECA 1265 23% 187 100 2
    Control Flow 95 40% 38 0 0
    Combinations 210 0% 0 0 0
    Bitvectors 49 78% 38 0 0
    Total 3679 47% 1489 103 130
    TACAS 2021, SV-Comp 5
    Results
    Large CFAs, performance issues
    

    View Slide

13. Category No. of tasks Correct (%) BMC EXPL PRED
    XCSP 119 82% 97 0 0
    Recursive 105 67% 70 0 0
    Product Lines 597 92% 451 0 97
    Loops 770 48% 333 3 31
    Floats 469 59% 275 0 0
    ECA 1265 23% 187 100 2
    Control Flow 95 40% 38 0 0
    Combinations 210 0% 0 0 0
    Bitvectors 49 78% 38 0 0
    Total 3679 47% 1489 103 130
    TACAS 2021, SV-Comp
    Results 2nd place in Product Lines
    

    View Slide

14. TACAS 2021, SV-Comp
    Summary
    5
    https://github.com/ftsrg/theta/
    https://github.com/ftsrg/gazer/
    

    View Slide