Upgrade to Pro — share decks privately, control downloads, hide ads and more …

When testing just doesn't cut it

When testing just doesn't cut it

Writing unit tests is pretty much standard practice these days. Otherwise, how would you make sure that your code does what you expect? Yet, some software is mission-critical and merely testing a few examples – or even randomized testing – is not enough. To reach higher levels of assurance, we need proof: mathematical, formal proof. This session will be based on an example from industry, where we successfully verified the core of a financial application. I will describe the core architecture of the system and the mathematical foundations behind the verification, including the classes of problems that we can (or cannot) discover with this approach.

Lars Hupel

March 17, 2023

More Decks by Lars Hupel

Other Decks in Programming


  1. 4

  2. 15

  3. “Program testing can be a very effective way to show

    the presence of bugs, but it is hopelessly inadequate for showing their absence”
  4. “Formal Methods refers to mathematically rigorous techniques and tools for

    the specification, design and verification of software and hardware systems” 18
  5. Specification What are Formal Methods? Coverage Rigor Implementation Type system

    First-order logic Model checking State machines Theorem prover Property testing Flowchart
  6. 25

  7. Central Bank Digital Currency 26 CBDC Banknotes Bank deposits and

    e-money Issued by the central bank Digital money
  8. 28

  9. 30

  10. 31

  11. 32

  12. 33

  13. G+D Filia® in Isabelle/HOL • mathematical model of “coins” and

    their evolution • graph-theoretic considerations • high-level correctness properties • reference implementation (executable in Scala)
  14. Example: Money in circulation definition graph_balance :: nat where ‹graph_balance

    = (∑N ∈ unspent. value N)› lemma graph_balance_alt_def: ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦› 37
  15. 40

  16. 41

  17. 42

  18. Designing a new feature • Can the feature work correctly?

    • Are there any undesirable feature interactions? • How can we implement the feature? 43
  19. PDD works for us • we found some flaws in

    our initial design of a feature • … including a feature interaction bug • after iterative improvement, the feature is now better than an alternative design • changed the internal (simpler) data model, but we established a mapping • feature has been shipped to production 45
  20. 47

  21. There’s always more to do … • expanding the scope

    of our formalization • adding model checking to our toolbox • closing the gap between executable specification and implementation 48
  22. Image sources • Edsger W. Dijskstra: Hamilton Richards, CC-BY-SA 3.0,

    https://commons.wikimedia.org/w/index.php?title=File:Edsger_Wybe_Dijkstra.jpg&oldid=710250 942 • César A. Muñoz: https://shemesh.larc.nasa.gov/people/cam/ • BPMN: Mikelo Skarabo, CC-BY-SA 4.0, https://commons.wikimedia.org/w/index.php?title=File:BPMN- AProcessWithNormalFlow.svg&oldid=734511959