When testing just doesn't cut it

Slide 1

Slide 1 text

When testing just doesn't cut it Lars Hupel BOB Konferenz 2023-03-17

Slide 2

Slide 2 text

Where would this line be used? int mid = (low + high) / 2

Slide 3

Slide 3 text

… and what’s wrong with it? int mid = (low + high) / 2

Slide 4

Slide 4 text

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Sorting in Java 6 list.sort((x, y) -> x.beard.compareTo(y.beard) )

Slide 7

Slide 7 text

7 CAV 2015

Slide 8

Slide 8 text

Programming & Bugs 8

Slide 9

Slide 9 text

Requirements Design/Architecture Implementation Testing Operation

Slide 10

Slide 10 text

Bugs: We don’t like them Yet, they keep cropping up … 10

Slide 11

Slide 11 text

Requirements Design/Architecture Implementation Testing Operation “Debugging”

Slide 12

Slide 12 text

Debugging is a core skill 12

Slide 13

Slide 13 text

13 OSDI 2014

Slide 14

Slide 14 text

14 Empirical Software Engineering 2015

Slide 15

Slide 15 text

Slide 16

Slide 16 text

“Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence”

Slide 17

Slide 17 text

Formal Methods 17

Slide 18

Slide 18 text

“Formal Methods refers to mathematically rigorous techniques and tools for the specification, design and verification of software and hardware systems” 18

Slide 19

Slide 19 text

Specification What are Formal Methods? Coverage Rigor Implementation Type system First-order logic Model checking State machines Theorem prover Property testing Flowchart

Slide 20

Slide 20 text

ISO 5807 Flowchart 20

Slide 21

Slide 21 text

ISO 5807:1985 21 Syntax Semantics

Slide 22

Slide 22 text

What is verification? Specification Implementation Proof

Slide 23

Slide 23 text

What is verification? Abstract specification Implementation Proof Executable specification Proof

Slide 24

Slide 24 text

Formal Methods in practice 24

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Central Bank Digital Currency 26 CBDC Banknotes Bank deposits and e-money Issued by the central bank Digital money

Slide 27

Slide 27 text

Our customers ● central banks ● commercial/retail banks ● payment service providers 27

Slide 28

Slide 28 text

Slide 29

Slide 29 text

How money is represented in G+D Filia® 29

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Isabelle to the rescue! 34

Slide 35

Slide 35 text

“Isabelle/HOL = Functional Programming + Logic”

Slide 36

Slide 36 text

G+D Filia® in Isabelle/HOL ● mathematical model of “coins” and their evolution ● graph-theoretic considerations ● high-level correctness properties ● reference implementation (executable in Scala)

Slide 37

Slide 37 text

Example: Money in circulation definition graph_balance :: nat where ‹graph_balance = (∑N ∈ unspent. value N)› lemma graph_balance_alt_def: ‹graph_balance = ¦(∑c ∈ graph. value_difference c)¦› 37

Slide 38

Slide 38 text

It’s not just us 38

Slide 39

Slide 39 text

Proof-Driven Development (PDD) 39

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Designing a new feature ● Can the feature work correctly? ● Are there any undesirable feature interactions? ● How can we implement the feature? 43

Slide 44

Slide 44 text

Requirements Design/Architecture Implementation Testing Operation “PDD”

Slide 45

Slide 45 text

PDD works for us ● we found some flaws in our initial design of a feature ● … including a feature interaction bug ● after iterative improvement, the feature is now better than an alternative design ● changed the internal (simpler) data model, but we established a mapping ● feature has been shipped to production 45

Slide 46

Slide 46 text

Roadmap 46

Slide 47

Slide 47 text

Slide 48

Slide 48 text

There’s always more to do … ● expanding the scope of our formalization ● adding model checking to our toolbox ● closing the gap between executable specification and implementation 48

Slide 49

Slide 49 text

Closing the gap Abstract specification Implementation Proof Executable specification Proof

Slide 50

Slide 50 text

Questions? Answers! Lars Hupel https://lars.hupel.info [email protected]

Slide 51

Slide 51 text

Image sources ● Edsger W. Dijskstra: Hamilton Richards, CC-BY-SA 3.0, https://commons.wikimedia.org/w/index.php?title=File:Edsger_Wybe_Dijkstra.jpg&oldid=710250 942 ● César A. Muñoz: https://shemesh.larc.nasa.gov/people/cam/ ● BPMN: Mikelo Skarabo, CC-BY-SA 4.0, https://commons.wikimedia.org/w/index.php?title=File:BPMN- AProcessWithNormalFlow.svg&oldid=734511959