A Visualization Toolkit for Simplifier Traces in Isabelle/jEdit

The simplifier in Isabelle is a very powerful tactic to discharge large classes of goals automatically. However, the results of simplifying a term not always match the user's expectation: sometimes, the resulting term is not small enough, or the simplifier even failed to apply any rule. For these cases, the simplifier offers a trace which precisely logs all steps which have been made.

These traces can be huge, especially because the library of Isabelle/HOL offers many pre-defined rewriting rules. It is often very difficult for the user to find the necessary piece of information about why and what exactly failed. Furthermore, there is no way to inspect or even influence the system while the simplification is still running. Hence, a simple, linear trace is not sufficient in these situations.

In this talk, we will present a new tracing facility which offers interactivity and a high amount of configurability. It combines successful approaches from other logic languages and adapts them to the Isabelle setup. Furthermore, it fits neatly into the Isabelle/jEdit IDE.

Transcript

1. A Visualization Toolkit for Simplifier Traces in Isabelle/jEdit Lars Hupel TU München July 29th, 2013
2. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Implementation Overview 4 Evaluation 2 / 30
3. λ → ∀ = Isabelle β α Simplifier ▶ ... is one of the core tactics in Isabelle ▶ applies rewrite rules to terms ▶ rules can be conditional: preconditions solved recursively 3 / 30
4. λ → ∀ = Isabelle β α Simplifier Simplification might go wrong: ▶ no result at all ▶ unexpected result ▶ non-termination 4 / 30
5. λ → ∀ = Isabelle β α Simplifier Simplification might go wrong: ▶ no result at all ▶ unexpected result ▶ non-termination tackled by tracing 4 / 30
6. λ → ∀ = Isabelle β α Simplifier Trace Lists all rewriting steps, but: ▶ can be huge ▶ offers no hierachical structure ▶ can’t be searched or filtered ▶ problematic with non-termination 5 / 30
7. λ → ∀ = Isabelle β α 6 / 30
8. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Implementation Overview 4 Evaluation 7 / 30
9. λ → ∀ = Isabelle β α Overview ▶ interactive ▶ breakpoints on terms and theorems ▶ configurable verbosity ▶ integrated into jEdit 8 / 30
10. λ → ∀ = Isabelle β α Interaction 9 / 30
11. λ → ∀ = Isabelle β α Interaction ▶ at certain points, the system displays a question ▶ simplifier is halted ▶ user answers, simplifier continues ▶ user can skip steps → influences the simplification 10 / 30
12. λ → ∀ = Isabelle β α Interaction Problem When should the system display a question? Naïve solution Display a question every time a rewrite rule is applied. 11 / 30
13. λ → ∀ = Isabelle β α Interaction Problem When should the system display a question? Better solution Let the user define breakpoints. 11 / 30
14. λ → ∀ = Isabelle β α Breakpoints ▶ term breakpoints ▶ pattern: ?x < Suc ?y ▶ triggers when pattern matches the term to be rewritten ▶ theorem breakpoint ▶ name: mult_pos_pos ▶ triggers when that rule is applied 12 / 30
15. λ → ∀ = Isabelle β α Demo 13 / 30
16. λ → ∀ = Isabelle β α Redoing steps ▶ system asks when a step failed ▶ user can choose to redo the step (with different settings) 14 / 30
17. λ → ∀ = Isabelle β α Demo 15 / 30
18. λ → ∀ = Isabelle β α Memoization Problem ▶ simplification fails ▶ user enables trace, identifies error ▶ user adds new rewrite rule → simplification starts again 16 / 30
19. λ → ∀ = Isabelle β α Memoization Problem ▶ simplification fails ▶ user enables trace, identifies error ▶ user adds new rewrite rule → simplification starts again Solution Memoize answers 16 / 30
20. λ → ∀ = Isabelle β α Demo 17 / 30
21. λ → ∀ = Isabelle β α Trace Window ▶ shows cumulated trace ▶ hierarchical, foldable, filterable ▶ useful even if interactivity is not used 18 / 30
22. λ → ∀ = Isabelle β α Demo 19 / 30
23. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Implementation Overview 4 Evaluation 20 / 30
24. λ → ∀ = Isabelle β α Implementation ▶ Isabelle/ML ▶ Supertrace module ▶ minor modifications in the simplifier ▶ Isabelle/Scala ▶ Supertrace_Manager actor ▶ Isabelle/jEdit ▶ question panel ▶ trace window 21 / 30
25. λ → ∀ = Isabelle β α System Isabelle/ML val recurse: string -> term -> Proof.context -> Proof.context 22 / 30
26. λ → ∀ = Isabelle β α System Isabelle/ML datatype step_response = Continue of Proof.context | Skip val step: trace_data -> step_response val indicate_success: thm -> Proof.context -> unit 22 / 30
27. λ → ∀ = Isabelle β α System Isabelle/ML datatype fail_response = Exit | Redo val indicate_failure: trace_data -> Proof.context -> fail_response 22 / 30
28. λ → ∀ = Isabelle β α System Isabelle/Scala type Question = (Data, List[String], String) type Context = Map[Long, Question] type Results = Map[Long, XML.Tree] def handle_results(id: Document.Command_ID, results: Results): Context 23 / 30
29. λ → ∀ = Isabelle β α System Isabelle/Scala type Trace = List[Data] def generate_trace(results: Results): Trace 23 / 30
30. λ → ∀ = Isabelle β α Challenges ▶ selective memory clearing ▶ message filtering ▶ tree reconstruction 24 / 30
31. λ → ∀ = Isabelle β α Agenda 1 State of the Art 2 Features of the New Simplifier Trace 3 Implementation Overview 4 Evaluation 25 / 30
32. λ → ∀ = Isabelle β α Performance Simplifying 10x · 10y Test machine: Core i7, 3.5 GHz 26 / 30
33. λ → ∀ = Isabelle β α A Parallelized Simplifier? ▶ tracing is completely asynchronous ▶ supports multiple questions at the same time ▶ but: unused by the simplifier ▶ proof of concept: parallel simplifier 27 / 30
34. λ → ∀ = Isabelle β α Demo 28 / 30
35. λ → ∀ = Isabelle β α Future Work ▶ support for more tactics ▶ support for other traces (unifier, simp debug, ...) ▶ memoization: fuzzy matching ▶ term provenance ▶ tighter integration into jEdit ▶ ... 29 / 30
36. λ → ∀ = Isabelle β α Q & A 30 / 30