When testing just doesn't cut it (Lambda Days edition)

Writing unit tests is pretty much standard practice these days. Otherwise, how would you make sure that your code does what you expect? Yet, some software is mission-critical and merely testing a few examples – or even randomized testing – is not enough. To reach higher levels of assurance, we need proof: mathematical, formal proof. This session will be based on an example from industry, where we successfully verified the core of a financial application. I will describe the core architecture of the system and the mathematical foundations behind the verification, including the classes of problems that we can (or cannot) discover with this approach.

Transcript

1. When testing just doesn't cut it Lars Hupel Lambda Days 2023-06-05
2. Where would this line be used? int mid = (low + high) / 2
3. … and what’s wrong with it? int mid = (low + high) / 2
6. Sorting in Java 6 list.sort((x, y) -> x.beard.compareTo(y.beard) )
7. 7 CAV 2015
8. Programming & Bugs 8
9. Requirements Design/Architecture Implementation Testing Operation
10. Requirements Design/Architecture Implementation Testing Operation “Debugging”
11. 11 OSDI 2014
12. “Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence”
13. Formal Methods 13
14. “Formal Methods refers to mathematically rigorous techniques and tools for the specification, design and verification of software and hardware systems”
15. 15 You have already used Formal Methods! You have already used Formal Methods! … without knowing it … without knowing it
16. ISO 5807 Flowchart 16
17. ISO 5807:1985 17 Syntax Semantics
19. What is verification? Specification Implementation Proof
20. Binary search, again! 20 int mid = (low + high) / 2;
21. Binary search, again! 21 assert low <= high; assert 0 <= low; int mid = (low + high) / 2; assert low <= mid; assert mid <= high; Specification Specification Implementation
22. Binary search, again! 22 ∀𝑙𝑜𝑤, ℎ𝑖𝑔ℎ ∈ 𝐼𝑛𝑡32 . 𝑙𝑜𝑤 ≤ ℎ𝑖𝑔ℎ ⇒ 0 ≤ 𝑙𝑜𝑤 ⇒ 𝑙𝑜𝑤 ≤ 𝑙𝑜𝑤 +32 ℎ𝑖𝑔ℎ 2
23. Formal Methods in practice 23
25. Central Bank Digital Currency 25 CBDC Banknotes Bank deposits and e-money Issued by the central bank Digital money
26. Our customers • central banks • commercial/retail banks • payment service providers 26
28. How money is represented in G+D Filia® 28
32. From specification to implementation Specification Implementation Proof
33. From specification to implementation Abstract specification Implementation Proof Executable specification Proof
34. Isabelle to the rescue! 34
35. “Isabelle/HOL = Functional Programming + Logic”
36. 36 “unspent” “unspent”
37. Example: Money in circulation definition graph_balance :: nat where ‹graph_balance = (∑N ∈ unspent. value N)› 37
38. 38 value_difference = 10 value_difference = 0
39. Example: Money in circulation lemma graph_balance_eq_value_difference: ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦› 39
40. Example: Money in circulation lemma graph_balance_eq_value_difference_pos: shows ‹0 ≤ (∑c ∈ graph. value_difference c)› shows ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦› proof (induction) (* ... *) qed 40 It looks like you are trying to do induction. Do you want me to generate a template?
41. Example: Money in circulation lemma graph_balance_eq_value_difference_pos: shows ‹0 ≤ (∑c ∈ graph. value_difference c)› shows ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦› proof (induction) case empty (* ... *) next (* ... *) qed 41 base case steps
42. It’s not just us 42
43. Proof-Driven Development (PDD) 43
47. Requirements Design/Architecture Implementation Testing Operation “PDD”
48. 48 • Roadmap
49. Questions? Answers! Lars Hupel https://lars.hupel.info lars.hupel@gi-de.com
50. Image sources • Edsger W. Dijskstra: Hamilton Richards, CC-BY-SA 3.0, https://commons.wikimedia.org/w/index.php?title=File:Edsger_Wybe_Dijkstra.jpg&oldid=710250 942 • César A. Muñoz: https://shemesh.larc.nasa.gov/people/cam/ • Type error: Limboer, CC-BY-SA, https://stackoverflow.com/q/60000835