Interactive Simplifier Tracing and Debugging in Isabelle

Lars Hupel: Interactive Simplifier Tracing and Debugging in Isabelle
Paper: https://lars.hupel.info/pub/simp-trace.pdf
Presented at CICM 2014, Coimbra, Portugal

The Isabelle proof assistant comes equipped with a very powerful tactic for term simplification. While tremendously useful, the results of simplifying a term not always match the user's expectation: sometimes, the resulting term is not in the form the user expected, or the simplifier fails to apply a rule. We describe a new, interactive tracing facility which offers insight into the hierarchical structure of the simplification with user-defined filtering, memoization and search. The new simplifier trace is integrated into the Isabelle/jEdit Prover IDE.

Transcript

1. Interac ve Simplifier Tracing and Debugging in Isabelle
   Lars Hupel
   Technische Universität München
   Chair for Logic and Verifica on
   July 8th, 2014
   

   View Slide

2. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Agenda
   1 State of the Art
   2 Features of the New Simplifier Trace
   3 Challenges & Open Problems
   4 Evalua on
   2 / 24
   

   View Slide

3. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Isabelle
   ▶ interac ve proof assistant
   ▶ powerful automa on
   ▶ classical and equa onal reasoning
   ▶ decision procedures (e.g. linear arithme c)
   ▶ integra on with external automated theorem provers
   ▶ ...
   ▶ IDE with con nuous proof checking based on jEdit
   3 / 24
   

   View Slide

4. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Simplifier
   ▶ one of the core tac cs in Isabelle
   ▶ huge: more than 1800 lines of code
   ▶ applies rewrite rules to terms
   ▶ rules can be condi onal: precondi ons solved recursively
   ▶ rules can be lazy: “simprocs” can generate rules on the fly
   ▶ goals can be condi onal: local assump ons are used
   4 / 24
   

   View Slide

5. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Simplifier
   ▶ one of the core tac cs in Isabelle
   ▶ huge: more than 1800 lines of code
   ▶ applies rewrite rules to terms
   ▶ rules can be condi onal: precondi ons solved recursively
   ▶ rules can be lazy: “simprocs” can generate rules on the fly
   ▶ goals can be condi onal: local assump ons are used
   4 / 24
   

   View Slide

6. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Simplifier
   Example: Condi onal rewrite rules
   x, y ∈ N
   2 · x = x + x (1)
   x < y =⇒ x − y = 0 (2)
   0 < x + 1 (3)
   0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
   5 / 24
   

   View Slide

7. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Simplifier
   Example: Condi onal rewrite rules
   x, y ∈ N
   2 · x = x + x (1)
   x < y =⇒ x − y = 0 (2)
   0 < x + 1 (3)
   0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
   0 − 2 · (x + 1)
   5 / 24
   

   View Slide

8. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Simplifier
   Example: Condi onal rewrite rules
   x, y ∈ N
   2 · x = x + x (1)
   x < y =⇒ x − y = 0 (2)
   0 < x + 1 (3)
   0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
   0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1))
   5 / 24
   

   View Slide

9. λ
   →
   ∀
   =
   Isabelle
   β
   α
   Simplifier
   Example: Condi onal rewrite rules
   x, y ∈ N
   2 · x = x + x (1)
   x < y =⇒ x − y = 0 (2)
   0 < x + 1 (3)
   0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
   0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1))
   5 / 24
   

   View Slide

10. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier
    Example: Condi onal rewrite rules
    x, y ∈ N
    2 · x = x + x (1)
    x < y =⇒ x − y = 0 (2)
    0 < x + 1 (3)
    0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
    0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
    ▶ 0 < ((x + 1) + (x + 1))
    5 / 24
    

    View Slide

11. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier
    Example: Condi onal rewrite rules
    x, y ∈ N
    2 · x = x + x (1)
    x < y =⇒ x − y = 0 (2)
    0 < x + 1 (3)
    0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
    0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
    ▶ 0 < ((x + 1) + (x + 1))
    ▶ 0 < x + 1
    ▶ 0 < x + 1
    5 / 24
    

    View Slide

12. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier
    Example: Condi onal rewrite rules
    x, y ∈ N
    2 · x = x + x (1)
    x < y =⇒ x − y = 0 (2)
    0 < x + 1 (3)
    0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
    0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
    ▶ 0 < ((x + 1) + (x + 1))
    ▶ 0 < x + 1
    ▶ 0 < x + 1
    5 / 24
    

    View Slide

13. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier
    Example: Condi onal rewrite rules
    x, y ∈ N
    2 · x = x + x (1)
    x < y =⇒ x − y = 0 (2)
    0 < x + 1 (3)
    0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
    0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
    ▶ 0 < ((x + 1) + (x + 1))
    ▶ 0 < x + 1
    ▶ 0 < x + 1
    5 / 24
    

    View Slide

14. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier
    Simplifica on might go wrong:
    ▶ no result at all
    ▶ unexpected result
    ▶ non-termina on
    6 / 24
    

    View Slide

15. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier
    Simplifica on might go wrong:
    ▶ no result at all
    ▶ unexpected result
    ▶ non-termina on
    tackled by tracing
    6 / 24
    

    View Slide

16. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Simplifier Trace
    Lists all rewri ng steps, but:
    ▶ poten ally huge
    ▶ can’t be filtered (e.g. “trace only applica ons of X and Y”)
    ▶ offers no hierachical structure
    ▶ problema c with non-termina on
    7 / 24
    

    View Slide

17. λ
    →
    ∀
    =
    Isabelle
    β
    α
    8 / 24
    

    View Slide

18. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Agenda
    1 State of the Art
    2 Features of the New Simplifier Trace
    3 Challenges & Open Problems
    4 Evalua on
    9 / 24
    

    View Slide

19. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Overview
    ▶ interac ve
    ▶ breakpoints on terms and theorems
    ▶ configurable verbosity
    ▶ integrated into Isabelle/jEdit
    10 / 24
    

    View Slide

20. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Demonstra on
    

    View Slide

21. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Related Work
    SWI-Prolog
    ▶ offers interac ve tracing
    ▶ supports breakpoints
    ▶ speciality: marking goals as success
    12 / 24
    

    View Slide

22. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Related Work
    SWI-Prolog
    ▶ offers interac ve tracing
    ▶ supports breakpoints
    ▶ speciality: marking goals as success
    ▶ In Isabelle: difficult because of proof kernel
    12 / 24
    

    View Slide

23. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Related Work
    Maude
    ▶ offers interac ve tracing
    ▶ supports breakpoints
    ▶ speciality: during rewri ng, issue new goal
    13 / 24
    

    View Slide

24. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Related Work
    Maude
    ▶ offers interac ve tracing
    ▶ supports breakpoints
    ▶ speciality: during rewri ng, issue new goal
    ▶ In Isabelle: rarely needed because of parallel processing
    13 / 24
    

    View Slide

25. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Agenda
    1 State of the Art
    2 Features of the New Simplifier Trace
    3 Challenges & Open Problems
    4 Evalua on
    14 / 24
    

    View Slide

26. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Challenges
    Selec ve Memory Clearing
    Scenario
    1. rewrite step fails
    2. user chooses to redo the step
    3. simplifica on starts anew
    4. memoiza on kicks in, step fails again
    15 / 24
    

    View Slide

27. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Challenges
    Selec ve Memory Clearing
    Scenario
    1. rewrite step fails
    2. user chooses to redo the step
    3. simplifica on starts anew
    4. memoiza on kicks in, step fails again
    15 / 24
    

    View Slide

28. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Challenges
    Context Handling
    ▶ simplifica on result depends on local assump ons
    ▶ memoiza on might not make sense across different contexts
    (P =⇒ P) =⇒ (Q =⇒ P) =⇒ R
    16 / 24
    

    View Slide

29. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Challenges
    User Experience
    ▶ user feedback is generally posi ve
    ▶ used for detec ng erra c rules, analyzing simplifier run me, ...
    ▶ very flexible, but: every addi onal op on generates confusion
    17 / 24
    

    View Slide

30. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Agenda
    1 State of the Art
    2 Features of the New Simplifier Trace
    3 Challenges & Open Problems
    4 Evalua on
    18 / 24
    

    View Slide

31. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Performance
    Simplifying 10x · 10y
    Test machine: Core i7, 3.7 GHz
    19 / 24
    

    View Slide

32. λ
    →
    ∀
    =
    Isabelle
    β
    α
    A Parallelized Simplifier?
    ▶ tracing is completely asynchronous
    ▶ supports mul ple ques ons at the same me
    ▶ but: unused by the simplifier
    ▶ proof of concept: development of a ny, parallel simplifier
    20 / 24
    

    View Slide

33. λ
    →
    ∀
    =
    Isabelle
    β
    α
    A Parallelized Simplifier?
    Lessons Learned
    Advantages
    ▶ almost trivial to implement for a toy simplifier
    ▶ GUI part works out of the box
    21 / 24
    

    View Slide

34. λ
    →
    ∀
    =
    Isabelle
    β
    α
    A Parallelized Simplifier?
    Lessons Learned
    Advantages
    ▶ almost trivial to implement for a toy simplifier
    ▶ GUI part works out of the box
    Disadvantages
    ▶ poten ally confusing for users
    ▶ lots of spurious messages
    ▶ be er filtering required?
    ▶ holding back messages required?
    21 / 24
    

    View Slide

35. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Outcomes
    ▶ a generic tracing facility
    ▶ using its interface requires li le changes to a tac c
    ▶ paralleliza on-ready
    ▶ but not 100% there yet
    ▶ first steps towards instrumen ng the simplifier
    ▶ Should all tac cs be wri en in con nua on-passing style?
    22 / 24
    

    View Slide

36. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Future Work
    ▶ support for more tac cs
    ▶ support for other traces (unifier, simp debug, ...)
    ▶ memoiza on: fuzzy matching
    ▶ term provenance (“Where does that ‘5’ come from?”)
    ▶ ghter integra on into Isabelle/jEdit
    23 / 24
    

    View Slide

37. λ
    →
    ∀
    =
    Isabelle
    β
    α
    Q & A
    

    View Slide