$30 off During Our Annual Pro Sale. View Details »

When testing just doesn't cut it (Lambda Days edition)

When testing just doesn't cut it (Lambda Days edition)

Writing unit tests is pretty much standard practice these days. Otherwise, how would you make sure that your code does what you expect? Yet, some software is mission-critical and merely testing a few examples – or even randomized testing – is not enough. To reach higher levels of assurance, we need proof: mathematical, formal proof. This session will be based on an example from industry, where we successfully verified the core of a financial application. I will describe the core architecture of the system and the mathematical foundations behind the verification, including the classes of problems that we can (or cannot) discover with this approach.

Lars Hupel

June 05, 2023
Tweet

More Decks by Lars Hupel

Other Decks in Programming

Transcript

  1. When testing just
    doesn't cut it
    Lars Hupel
    Lambda Days
    2023-06-05

    View Slide

  2. Where would this line be used?
    int mid = (low + high) / 2

    View Slide

  3. … and what’s wrong with it?
    int mid = (low + high) / 2

    View Slide

  4. 4

    View Slide

  5. View Slide

  6. Sorting in Java
    6
    list.sort((x, y) ->
    x.beard.compareTo(y.beard)
    )

    View Slide

  7. 7
    CAV 2015

    View Slide

  8. Programming & Bugs
    8

    View Slide

  9. Requirements
    Design/Architecture
    Implementation
    Testing
    Operation

    View Slide

  10. Requirements
    Design/Architecture
    Implementation
    Testing
    Operation
    “Debugging”

    View Slide

  11. 11
    OSDI 2014

    View Slide

  12. “Program testing can be a
    very effective way to show
    the presence of bugs, but it is
    hopelessly inadequate for
    showing their absence”

    View Slide

  13. Formal Methods
    13

    View Slide

  14. “Formal Methods refers to
    mathematically rigorous
    techniques and tools for the
    specification, design and
    verification of software and
    hardware systems”

    View Slide

  15. 15
    You have already used
    Formal Methods!
    You have already used
    Formal Methods!
    … without knowing it
    … without knowing it

    View Slide

  16. ISO 5807 Flowchart
    16

    View Slide

  17. ISO 5807:1985
    17
    Syntax
    Semantics

    View Slide

  18. 18

    View Slide

  19. What is verification?
    Specification Implementation
    Proof

    View Slide

  20. Binary search, again!
    20
    int mid = (low + high) / 2;

    View Slide

  21. Binary search, again!
    21
    assert low <= high;
    assert 0 <= low;
    int mid = (low + high) / 2;
    assert low <= mid;
    assert mid <= high;
    Specification
    Specification
    Implementation

    View Slide

  22. Binary search, again!
    22
    ∀𝑙𝑜𝑤, ℎ𝑖𝑔ℎ ∈ 𝐼𝑛𝑡32
    .
    𝑙𝑜𝑤 ≤ ℎ𝑖𝑔ℎ ⇒
    0 ≤ 𝑙𝑜𝑤 ⇒
    𝑙𝑜𝑤 ≤
    𝑙𝑜𝑤 +32
    ℎ𝑖𝑔ℎ
    2

    View Slide

  23. Formal Methods in practice
    23

    View Slide

  24. 24

    View Slide

  25. Central Bank Digital Currency
    25
    CBDC
    Banknotes Bank deposits
    and e-money
    Issued by the
    central bank
    Digital money

    View Slide

  26. Our customers
    ● central banks
    ● commercial/retail banks
    ● payment service providers
    26

    View Slide

  27. 27

    View Slide

  28. How money is represented in G+D Filia®
    28

    View Slide

  29. 29

    View Slide

  30. 30

    View Slide

  31. 31

    View Slide

  32. From specification to implementation
    Specification Implementation
    Proof

    View Slide

  33. From specification to implementation
    Abstract
    specification
    Implementation
    Proof
    Executable
    specification
    Proof

    View Slide

  34. Isabelle to the rescue!
    34

    View Slide

  35. “Isabelle/HOL =
    Functional Programming
    + Logic”

    View Slide

  36. 36
    “unspent”
    “unspent”

    View Slide

  37. Example: Money in circulation
    definition graph_balance :: nat where
    ‹graph_balance = (∑N ∈ unspent. value N)›
    37

    View Slide

  38. 38
    value_difference = 10
    value_difference = 0

    View Slide

  39. Example: Money in circulation
    lemma graph_balance_eq_value_difference:
    ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦›
    39

    View Slide

  40. Example: Money in circulation
    lemma graph_balance_eq_value_difference_pos:
    shows ‹0 ≤ (∑c ∈ graph. value_difference c)›
    shows ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦›
    proof (induction)
    (* ... *)
    qed
    40
    It looks like you are
    trying to do induction.
    Do you want me to
    generate a template?

    View Slide

  41. Example: Money in circulation
    lemma graph_balance_eq_value_difference_pos:
    shows ‹0 ≤ (∑c ∈ graph. value_difference c)›
    shows ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦›
    proof (induction)
    case empty
    (* ... *)
    next
    (* ... *)
    qed
    41
    base case
    steps

    View Slide

  42. It’s not just us
    42

    View Slide

  43. Proof-Driven Development (PDD)
    43

    View Slide

  44. 44

    View Slide

  45. 45

    View Slide

  46. 46

    View Slide

  47. Requirements
    Design/Architecture
    Implementation
    Testing
    Operation
    “PDD”

    View Slide

  48. 48
    ● Roadmap

    View Slide

  49. Questions?
    Answers!
    Lars Hupel
    https://lars.hupel.info
    [email protected]

    View Slide

  50. Image sources
    ● Edsger W. Dijskstra: Hamilton Richards, CC-BY-SA 3.0,
    https://commons.wikimedia.org/w/index.php?title=File:Edsger_Wybe_Dijkstra.jpg&oldid=710250
    942
    ● César A. Muñoz: https://shemesh.larc.nasa.gov/people/cam/
    ● Type error: Limboer, CC-BY-SA, https://stackoverflow.com/q/60000835

    View Slide