Interactive Simplifier Tracing and Debugging in Isabelle

Lars Hupel: Interactive Simplifier Tracing and Debugging in Isabelle
Paper: https://lars.hupel.info/pub/simp-trace.pdf
Presented at CICM 2014, Coimbra, Portugal

The Isabelle proof assistant comes equipped with a very powerful tactic for term simplification. While tremendously useful, the results of simplifying a term not always match the user's expectation: sometimes, the resulting term is not in the form the user expected, or the simplifier fails to apply a rule. We describe a new, interactive tracing facility which offers insight into the hierarchical structure of the simplification with user-defined filtering, memoization and search. The new simplifier trace is integrated into the Isabelle/jEdit Prover IDE.

Transcript

1. Interac ve Simplifier Tracing and Debugging in Isabelle Lars Hupel Technische Universität München Chair for Logic and Verifica on July 8th, 2014
2. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Challenges & Open Problems 4 Evalua on 2 / 24
3. λ → ∀ = Isabelle β α Isabelle ▶ interac ve proof assistant ▶ powerful automa on ▶ classical and equa onal reasoning ▶ decision procedures (e.g. linear arithme c) ▶ integra on with external automated theorem provers ▶ ... ▶ IDE with con nuous proof checking based on jEdit 3 / 24
4. λ → ∀ = Isabelle β α Simplifier ▶ one of the core tac cs in Isabelle ▶ huge: more than 1800 lines of code ▶ applies rewrite rules to terms ▶ rules can be condi onal: precondi ons solved recursively ▶ rules can be lazy: “simprocs” can generate rules on the fly ▶ goals can be condi onal: local assump ons are used 4 / 24
5. λ → ∀ = Isabelle β α Simplifier ▶ one of the core tac cs in Isabelle ▶ huge: more than 1800 lines of code ▶ applies rewrite rules to terms ▶ rules can be condi onal: precondi ons solved recursively ▶ rules can be lazy: “simprocs” can generate rules on the fly ▶ goals can be condi onal: local assump ons are used 4 / 24
6. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 5 / 24
7. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) 5 / 24
8. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) 5 / 24
9. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) 5 / 24
10. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0 ▶ 0 < ((x + 1) + (x + 1)) 5 / 24
11. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0 ▶ 0 < ((x + 1) + (x + 1)) ▶ 0 < x + 1 ▶ 0 < x + 1 5 / 24
12. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0 ▶ 0 < ((x + 1) + (x + 1)) ▶ 0 < x + 1 ▶ 0 < x + 1 5 / 24
13. λ → ∀ = Isabelle β α Simplifier Example: Condi onal rewrite rules x, y ∈ N 2 · x = x + x (1) x < y =⇒ x − y = 0 (2) 0 < x + 1 (3) 0 < x =⇒ 0 < y =⇒ 0 < x + y (4) 0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0 ▶ 0 < ((x + 1) + (x + 1)) ▶ 0 < x + 1 ▶ 0 < x + 1 5 / 24
14. λ → ∀ = Isabelle β α Simplifier Simplifica on might go wrong: ▶ no result at all ▶ unexpected result ▶ non-termina on 6 / 24
15. λ → ∀ = Isabelle β α Simplifier Simplifica on might go wrong: ▶ no result at all ▶ unexpected result ▶ non-termina on tackled by tracing 6 / 24
16. λ → ∀ = Isabelle β α Simplifier Trace Lists all rewri ng steps, but: ▶ poten ally huge ▶ can’t be filtered (e.g. “trace only applica ons of X and Y”) ▶ offers no hierachical structure ▶ problema c with non-termina on 7 / 24
17. λ → ∀ = Isabelle β α 8 / 24
18. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Challenges & Open Problems 4 Evalua on 9 / 24
19. λ → ∀ = Isabelle β α Overview ▶ interac ve ▶ breakpoints on terms and theorems ▶ configurable verbosity ▶ integrated into Isabelle/jEdit 10 / 24
20. λ → ∀ = Isabelle β α Demonstra on
21. λ → ∀ = Isabelle β α Related Work SWI-Prolog ▶ offers interac ve tracing ▶ supports breakpoints ▶ speciality: marking goals as success 12 / 24
22. λ → ∀ = Isabelle β α Related Work SWI-Prolog ▶ offers interac ve tracing ▶ supports breakpoints ▶ speciality: marking goals as success ▶ In Isabelle: difficult because of proof kernel 12 / 24
23. λ → ∀ = Isabelle β α Related Work Maude ▶ offers interac ve tracing ▶ supports breakpoints ▶ speciality: during rewri ng, issue new goal 13 / 24
24. λ → ∀ = Isabelle β α Related Work Maude ▶ offers interac ve tracing ▶ supports breakpoints ▶ speciality: during rewri ng, issue new goal ▶ In Isabelle: rarely needed because of parallel processing 13 / 24
25. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Challenges & Open Problems 4 Evalua on 14 / 24
26. λ → ∀ = Isabelle β α Challenges Selec ve Memory Clearing Scenario 1. rewrite step fails 2. user chooses to redo the step 3. simplifica on starts anew 4. memoiza on kicks in, step fails again 15 / 24
27. λ → ∀ = Isabelle β α Challenges Selec ve Memory Clearing Scenario 1. rewrite step fails 2. user chooses to redo the step 3. simplifica on starts anew 4. memoiza on kicks in, step fails again 15 / 24
28. λ → ∀ = Isabelle β α Challenges Context Handling ▶ simplifica on result depends on local assump ons ▶ memoiza on might not make sense across different contexts (P =⇒ P) =⇒ (Q =⇒ P) =⇒ R 16 / 24
29. λ → ∀ = Isabelle β α Challenges User Experience ▶ user feedback is generally posi ve ▶ used for detec ng erra c rules, analyzing simplifier run me, ... ▶ very flexible, but: every addi onal op on generates confusion 17 / 24
30. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Challenges & Open Problems 4 Evalua on 18 / 24
31. λ → ∀ = Isabelle β α Performance Simplifying 10x · 10y Test machine: Core i7, 3.7 GHz 19 / 24
32. λ → ∀ = Isabelle β α A Parallelized Simplifier? ▶ tracing is completely asynchronous ▶ supports mul ple ques ons at the same me ▶ but: unused by the simplifier ▶ proof of concept: development of a ny, parallel simplifier 20 / 24
33. λ → ∀ = Isabelle β α A Parallelized Simplifier? Lessons Learned Advantages ▶ almost trivial to implement for a toy simplifier ▶ GUI part works out of the box 21 / 24
34. λ → ∀ = Isabelle β α A Parallelized Simplifier? Lessons Learned Advantages ▶ almost trivial to implement for a toy simplifier ▶ GUI part works out of the box Disadvantages ▶ poten ally confusing for users ▶ lots of spurious messages ▶ be er filtering required? ▶ holding back messages required? 21 / 24
35. λ → ∀ = Isabelle β α Outcomes ▶ a generic tracing facility ▶ using its interface requires li le changes to a tac c ▶ paralleliza on-ready ▶ but not 100% there yet ▶ first steps towards instrumen ng the simplifier ▶ Should all tac cs be wri en in con nua on-passing style? 22 / 24
36. λ → ∀ = Isabelle β α Future Work ▶ support for more tac cs ▶ support for other traces (unifier, simp debug, ...) ▶ memoiza on: fuzzy matching ▶ term provenance (“Where does that ‘5’ come from?”) ▶ ghter integra on into Isabelle/jEdit 23 / 24
37. λ → ∀ = Isabelle β α Q & A