When testing just doesn't cut it (Lambda Days edition)

Writing unit tests is pretty much standard practice these days. Otherwise, how would you make sure that your code does what you expect? Yet, some software is mission-critical and merely testing a few examples – or even randomized testing – is not enough. To reach higher levels of assurance, we need proof: mathematical, formal proof. This session will be based on an example from industry, where we successfully verified the core of a financial application. I will describe the core architecture of the system and the mathematical foundations behind the verification, including the classes of problems that we can (or cannot) discover with this approach.

Transcript

1. When testing just
   doesn't cut it
   Lars Hupel
   Lambda Days
   2023-06-05
   

   View Slide

2. Where would this line be used?
   int mid = (low + high) / 2
   

   View Slide

3. … and what’s wrong with it?
   int mid = (low + high) / 2
   

   View Slide

4. View Slide

5. View Slide

6. Sorting in Java
   6
   list.sort((x, y) ->
   x.beard.compareTo(y.beard)
   )
   

   View Slide

7. 7
   CAV 2015
   

   View Slide

8. Programming & Bugs
   8
   

   View Slide

9. Requirements
   Design/Architecture
   Implementation
   Testing
   Operation
   

   View Slide

10. Requirements
    Design/Architecture
    Implementation
    Testing
    Operation
    “Debugging”
    

    View Slide

11. 11
    OSDI 2014
    

    View Slide

12. “Program testing can be a
    very effective way to show
    the presence of bugs, but it is
    hopelessly inadequate for
    showing their absence”
    

    View Slide

13. Formal Methods
    13
    

    View Slide

14. “Formal Methods refers to
    mathematically rigorous
    techniques and tools for the
    specification, design and
    verification of software and
    hardware systems”
    

    View Slide

15. 15
    You have already used
    Formal Methods!
    You have already used
    Formal Methods!
    … without knowing it
    … without knowing it
    

    View Slide

16. ISO 5807 Flowchart
    16
    

    View Slide

17. ISO 5807:1985
    17
    Syntax
    Semantics
    

    View Slide

18. View Slide

19. What is verification?
    Specification Implementation
    Proof
    

    View Slide

20. Binary search, again!
    20
    int mid = (low + high) / 2;
    

    View Slide

21. Binary search, again!
    21
    assert low <= high;
    assert 0 <= low;
    int mid = (low + high) / 2;
    assert low <= mid;
    assert mid <= high;
    Specification
    Specification
    Implementation
    

    View Slide

22. Binary search, again!
    22
    ∀𝑙𝑜𝑤, ℎ𝑖𝑔ℎ ∈ 𝐼𝑛𝑡32
    .
    𝑙𝑜𝑤 ≤ ℎ𝑖𝑔ℎ ⇒
    0 ≤ 𝑙𝑜𝑤 ⇒
    𝑙𝑜𝑤 ≤
    𝑙𝑜𝑤 +32
    ℎ𝑖𝑔ℎ
    2
    

    View Slide

23. Formal Methods in practice
    23
    

    View Slide

24. View Slide

25. Central Bank Digital Currency
    25
    CBDC
    Banknotes Bank deposits
    and e-money
    Issued by the
    central bank
    Digital money
    

    View Slide

26. Our customers
    ● central banks
    ● commercial/retail banks
    ● payment service providers
    26
    

    View Slide

27. View Slide

28. How money is represented in G+D Filia®
    28
    

    View Slide

29. View Slide

30. View Slide

31. View Slide

32. From specification to implementation
    Specification Implementation
    Proof
    

    View Slide

33. From specification to implementation
    Abstract
    specification
    Implementation
    Proof
    Executable
    specification
    Proof
    

    View Slide

34. Isabelle to the rescue!
    34
    

    View Slide

35. “Isabelle/HOL =
    Functional Programming
    + Logic”
    

    View Slide

36. 36
    “unspent”
    “unspent”
    

    View Slide

37. Example: Money in circulation
    definition graph_balance :: nat where
    ‹graph_balance = (∑N ∈ unspent. value N)›
    37
    

    View Slide

38. 38
    value_difference = 10
    value_difference = 0
    

    View Slide

39. Example: Money in circulation
    lemma graph_balance_eq_value_difference:
    ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦›
    39
    

    View Slide

40. Example: Money in circulation
    lemma graph_balance_eq_value_difference_pos:
    shows ‹0 ≤ (∑c ∈ graph. value_difference c)›
    shows ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦›
    proof (induction)
    (* ... *)
    qed
    40
    It looks like you are
    trying to do induction.
    Do you want me to
    generate a template?
    

    View Slide

41. Example: Money in circulation
    lemma graph_balance_eq_value_difference_pos:
    shows ‹0 ≤ (∑c ∈ graph. value_difference c)›
    shows ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦›
    proof (induction)
    case empty
    (* ... *)
    next
    (* ... *)
    qed
    41
    base case
    steps
    

    View Slide

42. It’s not just us
    42
    

    View Slide

43. Proof-Driven Development (PDD)
    43
    

    View Slide

44. View Slide

45. View Slide

46. View Slide

47. Requirements
    Design/Architecture
    Implementation
    Testing
    Operation
    “PDD”
    

    View Slide

48. 48
    ● Roadmap
    

    View Slide

49. Questions?
    Answers!
    Lars Hupel
    https://lars.hupel.info
    lars.hupel@gi-de.com
    

    View Slide

50. Image sources
    ● Edsger W. Dijskstra: Hamilton Richards, CC-BY-SA 3.0,
    https://commons.wikimedia.org/w/index.php?title=File:Edsger_Wybe_Dijkstra.jpg&oldid=710250
    942
    ● César A. Muñoz: https://shemesh.larc.nasa.gov/people/cam/
    ● Type error: Limboer, CC-BY-SA, https://stackoverflow.com/q/60000835
    

    View Slide