Lars Hupel
July 08, 2014
270

# Interactive Simplifier Tracing and Debugging in Isabelle

Lars Hupel: Interactive Simplifier Tracing and Debugging in Isabelle
Paper: https://lars.hupel.info/pub/simp-trace.pdf
Presented at CICM 2014, Coimbra, Portugal

The Isabelle proof assistant comes equipped with a very powerful tactic for term simplification. While tremendously useful, the results of simplifying a term not always match the user's expectation: sometimes, the resulting term is not in the form the user expected, or the simplifier fails to apply a rule. We describe a new, interactive tracing facility which offers insight into the hierarchical structure of the simplification with user-defined filtering, memoization and search. The new simplifier trace is integrated into the Isabelle/jEdit Prover IDE.

July 08, 2014

## Transcript

1. Interac ve Simpliﬁer Tracing and Debugging in Isabelle
Lars Hupel
Technische Universität München
Chair for Logic and Veriﬁca on
July 8th, 2014

2. λ

=
Isabelle
β
α
Agenda
1 State of the Art
2 Features of the New Simpliﬁer Trace
3 Challenges & Open Problems
4 Evalua on
2 / 24

3. λ

=
Isabelle
β
α
Isabelle
▶ interac ve proof assistant
▶ powerful automa on
▶ classical and equa onal reasoning
▶ decision procedures (e.g. linear arithme c)
▶ integra on with external automated theorem provers
▶ ...
▶ IDE with con nuous proof checking based on jEdit
3 / 24

4. λ

=
Isabelle
β
α
Simpliﬁer
▶ one of the core tac cs in Isabelle
▶ huge: more than 1800 lines of code
▶ applies rewrite rules to terms
▶ rules can be condi onal: precondi ons solved recursively
▶ rules can be lazy: “simprocs” can generate rules on the ﬂy
▶ goals can be condi onal: local assump ons are used
4 / 24

5. λ

=
Isabelle
β
α
Simpliﬁer
▶ one of the core tac cs in Isabelle
▶ huge: more than 1800 lines of code
▶ applies rewrite rules to terms
▶ rules can be condi onal: precondi ons solved recursively
▶ rules can be lazy: “simprocs” can generate rules on the ﬂy
▶ goals can be condi onal: local assump ons are used
4 / 24

6. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
5 / 24

7. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1)
5 / 24

8. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1))
5 / 24

9. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1))
5 / 24

10. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
▶ 0 < ((x + 1) + (x + 1))
5 / 24

11. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
▶ 0 < ((x + 1) + (x + 1))
▶ 0 < x + 1
▶ 0 < x + 1
5 / 24

12. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
▶ 0 < ((x + 1) + (x + 1))
▶ 0 < x + 1
▶ 0 < x + 1
5 / 24

13. λ

=
Isabelle
β
α
Simpliﬁer
Example: Condi onal rewrite rules
x, y ∈ N
2 · x = x + x (1)
x < y =⇒ x − y = 0 (2)
0 < x + 1 (3)
0 < x =⇒ 0 < y =⇒ 0 < x + y (4)
0 − 2 · (x + 1) = 0 − ((x + 1) + (x + 1)) = 0
▶ 0 < ((x + 1) + (x + 1))
▶ 0 < x + 1
▶ 0 < x + 1
5 / 24

14. λ

=
Isabelle
β
α
Simpliﬁer
Simpliﬁca on might go wrong:
▶ no result at all
▶ unexpected result
▶ non-termina on
6 / 24

15. λ

=
Isabelle
β
α
Simpliﬁer
Simpliﬁca on might go wrong:
▶ no result at all
▶ unexpected result
▶ non-termina on
tackled by tracing
6 / 24

16. λ

=
Isabelle
β
α
Simpliﬁer Trace
Lists all rewri ng steps, but:
▶ poten ally huge
▶ can’t be ﬁltered (e.g. “trace only applica ons of X and Y”)
▶ oﬀers no hierachical structure
▶ problema c with non-termina on
7 / 24

17. λ

=
Isabelle
β
α
8 / 24

18. λ

=
Isabelle
β
α
Agenda
1 State of the Art
2 Features of the New Simpliﬁer Trace
3 Challenges & Open Problems
4 Evalua on
9 / 24

19. λ

=
Isabelle
β
α
Overview
▶ interac ve
▶ breakpoints on terms and theorems
▶ conﬁgurable verbosity
▶ integrated into Isabelle/jEdit
10 / 24

20. λ

=
Isabelle
β
α
Demonstra on

21. λ

=
Isabelle
β
α
Related Work
SWI-Prolog
▶ oﬀers interac ve tracing
▶ supports breakpoints
▶ speciality: marking goals as success
12 / 24

22. λ

=
Isabelle
β
α
Related Work
SWI-Prolog
▶ oﬀers interac ve tracing
▶ supports breakpoints
▶ speciality: marking goals as success
▶ In Isabelle: diﬃcult because of proof kernel
12 / 24

23. λ

=
Isabelle
β
α
Related Work
Maude
▶ oﬀers interac ve tracing
▶ supports breakpoints
▶ speciality: during rewri ng, issue new goal
13 / 24

24. λ

=
Isabelle
β
α
Related Work
Maude
▶ oﬀers interac ve tracing
▶ supports breakpoints
▶ speciality: during rewri ng, issue new goal
▶ In Isabelle: rarely needed because of parallel processing
13 / 24

25. λ

=
Isabelle
β
α
Agenda
1 State of the Art
2 Features of the New Simpliﬁer Trace
3 Challenges & Open Problems
4 Evalua on
14 / 24

26. λ

=
Isabelle
β
α
Challenges
Selec ve Memory Clearing
Scenario
1. rewrite step fails
2. user chooses to redo the step
3. simpliﬁca on starts anew
4. memoiza on kicks in, step fails again
15 / 24

27. λ

=
Isabelle
β
α
Challenges
Selec ve Memory Clearing
Scenario
1. rewrite step fails
2. user chooses to redo the step
3. simpliﬁca on starts anew
4. memoiza on kicks in, step fails again
15 / 24

28. λ

=
Isabelle
β
α
Challenges
Context Handling
▶ simpliﬁca on result depends on local assump ons
▶ memoiza on might not make sense across diﬀerent contexts
(P =⇒ P) =⇒ (Q =⇒ P) =⇒ R
16 / 24

29. λ

=
Isabelle
β
α
Challenges
User Experience
▶ user feedback is generally posi ve
▶ used for detec ng erra c rules, analyzing simpliﬁer run me, ...
▶ very ﬂexible, but: every addi onal op on generates confusion
17 / 24

30. λ

=
Isabelle
β
α
Agenda
1 State of the Art
2 Features of the New Simpliﬁer Trace
3 Challenges & Open Problems
4 Evalua on
18 / 24

31. λ

=
Isabelle
β
α
Performance
Simplifying 10x · 10y
Test machine: Core i7, 3.7 GHz
19 / 24

32. λ

=
Isabelle
β
α
A Parallelized Simpliﬁer?
▶ tracing is completely asynchronous
▶ supports mul ple ques ons at the same me
▶ but: unused by the simpliﬁer
▶ proof of concept: development of a ny, parallel simpliﬁer
20 / 24

33. λ

=
Isabelle
β
α
A Parallelized Simpliﬁer?
Lessons Learned
▶ almost trivial to implement for a toy simpliﬁer
▶ GUI part works out of the box
21 / 24

34. λ

=
Isabelle
β
α
A Parallelized Simpliﬁer?
Lessons Learned
▶ almost trivial to implement for a toy simpliﬁer
▶ GUI part works out of the box
▶ poten ally confusing for users
▶ lots of spurious messages
▶ be er ﬁltering required?
▶ holding back messages required?
21 / 24

35. λ

=
Isabelle
β
α
Outcomes
▶ a generic tracing facility
▶ using its interface requires li le changes to a tac c
▶ but not 100% there yet
▶ ﬁrst steps towards instrumen ng the simpliﬁer
▶ Should all tac cs be wri en in con nua on-passing style?
22 / 24

36. λ

=
Isabelle
β
α
Future Work
▶ support for more tac cs
▶ support for other traces (uniﬁer, simp debug, ...)
▶ memoiza on: fuzzy matching
▶ term provenance (“Where does that ‘5’ come from?”)
▶ ghter integra on into Isabelle/jEdit
23 / 24

37. λ

=
Isabelle
β
α
Q & A