Parametrized Model Checking of Fault Tolerant Distributed Algorithms by Abstraction (part 1)

Slide 1

Slide 1 text

Model Checking of Fault-Tolerant Distributed Algorithms Part III: Parameterized Model Checking of Fault-tolerant Distributed Algorithms by Abstraction Annu Gmeiner Igor Konnov Ulrich Schmid Helmut Veith Josef Widder TMPA 2014, Kostroma, Russia Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 1 / 1

Slide 2

Slide 2 text

Fault-tolerant DAs: Model Checking Challenges unbounded data types counting how many messages have been received parameterization in multiple parameters among n processes f ≤ t are faulty with n > 3t contrast to concurrent programs fault tolerance against adverse environments degrees of concurrency many degrees of partial synchrony continuous time fault-tolerant clock synchronization Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 2 / 1

Slide 3

Slide 3 text

Distributed algorithms: computational model and faults In previous parts, we considered algorithms operating in the classic model by [Fischer, Lynch, Paterson’85] Environment: Asynchronous processes (interleaving semantics) Reliable asynchronous message passing (non-blocking send and receive) Faults: crashes and clean crashes, omission faults, symmetric faults, Byzantine faults Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 3 / 1

Slide 4

Slide 4 text

Model checking problem for fault-tolerant DA algorithms Parameterized model checking problem: given a distributed algorithm and spec. ϕ show for all n, t, and f satisfying n > 3t ∧ t ≥ f ≥ 0 M(n, t, f ) |= ϕ every M(n, t, f ) is a system of n − f correct processes n ? ? ? t n ? ? ? t f Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 4 / 1

Slide 5

Slide 5 text

Model checking problem for fault-tolerant DA algorithms Parameterized model checking problem: given a distributed algorithm and spec. ϕ show for all n, t, and f satisfying resilience condition M(n, t, f ) |= ϕ every M(n, t, f ) is a system of N(n, f ) correct processes n ? ? ? t n ? ? ? t f Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 4 / 1

Slide 6

Slide 6 text

Properties in Linear Temporal Logic Unforgeability (U). If vi = 0 for all correct processes i, then for all correct processes j, acceptj remains 0 forever. G n−f i=1 vi = 0 → G n−f j=1 acceptj = 0 Completeness (C). If vi = 1 for all correct processes i, then there is a correct process j that eventually sets acceptj to 1. G n−f i=1 vi = 1 → F n−f j=1 acceptj = 1 Relay (R). If a correct process i sets accepti to 1, then eventually all correct processes j set acceptj to 1. G n−f i=1 accepti = 1 → F n−f j=1 acceptj = 1 Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 5 / 1

Slide 7

Slide 7 text

Properties in Linear Temporal Logic Unforgeability (U). If vi = 0 for all correct processes i, then for all correct processes j, acceptj remains 0 forever. G n−f i=1 vi = 0 → G n−f j=1 acceptj = 0 Safety Completeness (C). If vi = 1 for all correct processes i, then there is a correct process j that eventually sets acceptj to 1. G n−f i=1 vi = 1 → F n−f j=1 acceptj = 1 Liveness Relay (R). If a correct process i sets accepti to 1, then eventually all correct processes j set acceptj to 1. G n−f i=1 accepti = 1 → F n−f j=1 acceptj = 1 Liveness Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 5 / 1

Slide 8

Slide 8 text

Threshold-guarded fault-tolerant distributed algorithms Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 6 / 1

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Threshold-guarded FTDAs Fault-free construct: quantiﬁed guards (t=f=0) Existential Guard if received m from some process then ... Universal Guard if received m from all processes then ... These guards allow one to treat the processes in a parameterized way what if faults might occur? Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 7 / 1

Slide 11

Slide 11 text

Threshold-guarded FTDAs Fault-free construct: quantiﬁed guards (t=f=0) Existential Guard if received m from some process then ... Universal Guard if received m from all processes then ... These guards allow one to treat the processes in a parameterized way what if faults might occur? Fault-Tolerant Algorithms: n processes, at most t are Byzantine Threshold Guard if received m from n − t processes then ... (the processes cannot refer to f!) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 7 / 1

Slide 12

Slide 12 text

Threshold-based fault-tolerant distributed algorithms The parameters (n, t, f ) are ﬁxed in each run Main loop with the body executed atomically Processes are anonymous (no identiﬁers) Receiving messages, counting them and comparing to thresholds, e.g., if received from t + 1 distinct processes then ... Sending messages to all processes, e.g., send to all Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 8 / 1

Slide 13

Slide 13 text

Control Flow Automata Variables of process i vi : {0 , 1} init with 0 or 1 accepti : {0 , 1} init with 0 An indivisible step: i f vi = 1 then send ( echo ) to all ; i f received (echo) from at l e a s t t + 1 distinct processes and not sent ( echo ) before then send ( echo ) to all ; i f received ( echo ) from at l e a s t n - t distinct processes then accepti := 1; n − f copies of the process qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 q8 qF nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f ) ¬(t + 1 ≤ nrcvd) t + 1 ≤ nrcvd sv = V0 ¬(sv = V0) inc nsnt n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := SE sv := AC Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 9 / 1

Slide 14

Slide 14 text

Counting argument in threshold-guarded algorithms n t f if received m from t + 1 processes then ... t + 1 Correct processes count distinct incoming messages Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 10 / 1

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Counting argument in threshold-guarded algorithms n t f if received m from t + 1 processes then ... t + 1 at least one non-faulty sent the message Correct processes count distinct incoming messages Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 10 / 1

Slide 17

Slide 17 text

qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 q8 qF nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f ) ¬(t + 1 ≤ nrcvd) t + 1 ≤ nrcvd sv = V0 ¬(sv = V0) inc nsnt n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := SE sv := AC concrete values are not important thresholds are essential: 0, 1, t + 1, n − t Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 11 / 1

Slide 18

Slide 18 text

qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 q8 qF nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f ) ¬(t + 1 ≤ nrcvd) t + 1 ≤ nrcvd sv = V0 ¬(sv = V0) inc nsnt n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := SE sv := AC concrete values are not important thresholds are essential: 0, 1, t + 1, n − t intervals with symbolic boundaries: I0 = [0, 1) I1 = [1, t + 1) It+1 = [t + 1, n − t) In−t = [n − t, ∞) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 11 / 1

Slide 19

Slide 19 text

qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 q8 qF nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f ) ¬(t + 1 ≤ nrcvd) t + 1 ≤ nrcvd sv = V0 ¬(sv = V0) inc nsnt n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := SE sv := AC concrete values are not important thresholds are essential: 0, 1, t + 1, n − t intervals with symbolic boundaries: I0 = [0, 1) I1 = [1, t + 1) It+1 = [t + 1, n − t) In−t = [n − t, ∞) Parameteric Interval Abstraction (PIA) Similar to interval abstraction: [t + 1, n − t) rather than [4, 10). Total order: 0 < 1 < t + 1 < n − t for all parameters satisfying RC: n > 3t, t ≥ f ≥ 0. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 11 / 1

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Technical challenges We have to reduce the verification of an infinite number of instances where 1 the process code is parameterized 2 the number of processes is parameterized to one finite state model checking instance We do that by: 1 PIA data abstraction 2 PIA counter abstraction Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 12 / 1

Slide 22

Slide 22 text

Technical challenges We have to reduce the verification of an infinite number of instances where 1 the process code is parameterized 2 the number of processes is parameterized to one finite state model checking instance We do that by: 1 PIA data abstraction 2 PIA counter abstraction abstraction is an over approximation ⇒ possible abstract behavior that does not correspond to a concrete behavior. 3 Refining spurious counter-examples Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 12 / 1

Slide 23

Slide 23 text

Abstraction overview Parameterized family M(n, t, f ) = P(n, t, f ) · · · P(n, t, f ) N(n,t,f ) processes : n > 3t, t ≥ f , f ≥ 0} extract Parametric Interval Domain D parametric interval data abstraction Uniform parameterized family ˆ M(n, t, f ) = ˆ P · · · ˆ P N(n,t,f ) processes : n > 3t, t ≥ f , f ≥ 0} P does not depend on n, t, f P simulates P(n, t, f ) change representation Counter representation parametric interval counter abstraction one abstract system A that simulates for every n, t, f the behavior of M(n, t, f ) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 13 / 1

Slide 24

Slide 24 text

Abstraction overview Parameterized family M(n, t, f ) = P(n, t, f ) · · · P(n, t, f ) N(n,t,f ) processes : n > 3t, t ≥ f , f ≥ 0} extract Parametric Interval Domain D parametric interval data abstraction Uniform parameterized family ˆ M(n, t, f ) = ˆ P · · · ˆ P N(n,t,f ) processes : n > 3t, t ≥ f , f ≥ 0} P does not depend on n, t, f P simulates P(n, t, f ) change representation Counter representation parametric interval counter abstraction one abstract system A that simulates for every n, t, f the behavior of M(n, t, f ) ﬁnite-state model checkin replay the counter-example reﬁne the system Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 13 / 1

Slide 25

Slide 25 text

Data abstraction Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 14 / 1

Slide 26

Slide 26 text

qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 q8 qF nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f ) ¬(t + 1 ≤ nrcvd) t + 1 ≤ nrcvd sv = V0 ¬(sv = V0) inc nsnt n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := SE sv := AC concrete values are not important thresholds are essential: 0, 1, t + 1, n − t intervals with symbolic boundaries: I0 = [0, 1) I1 = [1, t + 1) It+1 = [t + 1, n − t) In−t = [n − t, ∞) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 15 / 1

Slide 27

Slide 27 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 It+1 In−t Concrete t + 1 ≤ x Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 28

Slide 28 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 It+1 In−t Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 29

Slide 29 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 It+1 In−t Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Concrete x = x + 1, Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 30

Slide 30 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 I0 I1 It+1 In−t Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Concrete x = x + 1, is abstracted as: x = I0 ∧ x = I1 . . . Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 31

Slide 31 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 I0 I1 It+1 In−t Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Concrete x = x + 1, is abstracted as: x = I0 ∧ x = I1 ∨x = I1 ∧ (x = I1 ∨ x = It+1) . . . Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 32

Slide 32 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 I0 I1 It+1 In−t Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Concrete x = x + 1, is abstracted as: x = I0 ∧ x = I1 ∨x = I1 ∧ (x = I1 ∨ x = It+1) ∨x = It+1 ∧ (x = It+1 ∨ x = In−t) . . . Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 33

Slide 33 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 I0 I1 It+1 In−t Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Concrete x = x + 1, is abstracted as: x = I0 ∧ x = I1 ∨x = I1 ∧ (x = I1 ∨ x = It+1) ∨x = It+1 ∧ (x = It+1 ∨ x = In−t) ∨x = In−t ∧ x = In−t Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 34

Slide 34 text

Abstract operations Concrete: Abstract: 0 1 t + 1 n − t above · · · I0 I1 Concrete t + 1 ≤ x is abstracted as x = It+1 ∨ x = In−t. Concrete x = x + 1, is abstracted as: x = I0 ∧ x = I1 ∨x = I1 ∧ (x = I1 ∨ x = It+1) ∨x = It+1 ∧ (x = It+1 ∨ x = In−t) ∨x = In−t ∧ x = In−t abstract increase may keep the same value! Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 16 / 1

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Abstract CFA qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 nrcvd := z where (nrcvd ≤ z ∧ z ≤ nsnt + f ) ¬(t + 1 ≤ nrcvd) t + 1 ≤ nrcvd sv = V0 ¬(sv = V0) inc nsnt n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := AC qI q0 q1 q2 q3 sv = V1 ¬(sv = V1) inc nsnt sv := SE q4 q5 q6 q7 nrcvd = I0 ∧ nsnt = I0 ∧ (nrcvd = I0 ∨ nrcvd = I1 ) ∨ . . . ¬(t + 1 ≤ nrcvd) nrcvd = It+1 ∨ nrcvd = In−t sv = V0 ¬(sv = V0) nsnt = I1 ∧ (nsnt = I1 ∨ nsnt = It+1 ) ∨ . . . n − t ≤ nrcvd ¬(n − t ≤ nrcvd) sv := AC Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 17 / 1

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Counter abstraction Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 19 / 1

Slide 39

Slide 39 text

Classic (0, 1, ∞)-counter abstraction Pnueli, Xu, and Zuck (2001) introduced (0, 1, ∞)-counter abstraction: ﬁnitely many local states, e.g., {N, T, C}. based on counter representation: for each local states count how many processes are in it Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 20 / 1

Slide 40

Slide 40 text

Classic (0, 1, ∞)-counter abstraction Pnueli, Xu, and Zuck (2001) introduced (0, 1, ∞)-counter abstraction: ﬁnitely many local states, e.g., {N, T, C}. based on counter representation: for each local states count how many processes are in it abstract the number of processes in every state, e.g., K : C → 0, T → 1, N → “many”. perfectly reﬂects mutual exclusion properties e.g., G (K(C) = 0 ∨ K(C) = 1). Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 20 / 1

Slide 41

Slide 41 text

Limits of (0, 1, ∞)-counter abstraction Our parametric data + counter abstraction: we require ﬁner counting of processes: t + 1 processes in a speciﬁc state can force global progress, t processes cannot mapping t, t + 1, and n − t to “many” is too coarse. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 21 / 1

Slide 42

Slide 42 text

Limits of (0, 1, ∞)-counter abstraction Our parametric data + counter abstraction: we require ﬁner counting of processes: t + 1 processes in a speciﬁc state can force global progress, t processes cannot mapping t, t + 1, and n − t to “many” is too coarse. starting point of our approach... Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 21 / 1

Slide 43

Slide 43 text

Data + counter abstraction over parametric intervals n = 6, t = 1, f = 1 t + 1 = 2, n − t = 5 nr. processes (counters) received received sent accepted • 0 • 0 • 1 • 1 • 2 • 2 • 3 • 3 • 4 • 4 • 5 • 5 • 6 • 6 • 0 • 1 • 2 • 3 • 4 • 5 • 6 • Local state is (sv, nrcvd), where sv ∈ {sent, accepted} and 0 ≤ rcvd ≤ n 3 processes at (sent, received=3) 1 process at (accepted, received=5) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1

Slide 44

Slide 44 text

Data + counter abstraction over parametric intervals n = 6, t = 1, f = 1 t + 1 = 2, n − t = 5 nr. processes (counters) received received sent accepted • 0 • 0 • 1 • 1 • 2 • 2 • 3 • 3 • 4 • 4 • 5 • 5 • 6 • 6 • 0 • 1 • 2 • 3 • 4 • 5 • 6 • Local state is (sv, nrcvd), where sv ∈ {sent, accepted} and 0 ≤ rcvd ≤ n Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1

Slide 45

Slide 45 text

Data + counter abstraction over parametric intervals n = 6, t = 1, f = 1 t + 1 = 2, n − t = 5 nr. processes (counters) received received sent accepted • 0 • 0 • 1 • 1 • 2 • 2 • 3 • 3 • 4 • 4 • 5 • 5 • 6 • 6 • 0 • 1 • 2 • 3 • 4 • 5 • 6 • Local state is (sv, nrcvd), where sv ∈ {sent, accepted} and 0 ≤ rcvd ≤ n Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1

Slide 46

Slide 46 text

Data + counter abstraction over parametric intervals XXXXXX n = 6, XXXXX X t = 1, XXXXXX f = 1 n > 3 · t ∧ t ≥ f Parametricintervals: I0 = [0, 1) I1 = [1, t + 1) It+1 = [t + 1, n − t) In−t = [n − t, ∞) nr. processes (counters) received received sent accepted • • • • I0 I1 It+1 In−t • • • • I0 I1 It+1 In−t • • • • I0 I1 It+1 In−t A local state is (sv, nrcvd), where sv ∈ {sent, accepted} and nrcvd ∈ {I0, I1, It+1, In−t} Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1

Slide 47

Slide 47 text

Data + counter abstraction over parametric intervals n > 3 · t ∧ t ≥ f Parametricintervals: I0 = [0, 1) I1 = [1, t + 1) It+1 = [t + 1, n − t) In−t = [n − t, ∞) nr. processes (counters) received received sent accepted • • • • I0 I1 It+1 In−t • • • • I0 I1 It+1 In−t • • • • I0 I1 It+1 In−t when all correct processes accepted, all non-zero counters are in this area A local state is (sv, nrcvd), where sv ∈ {sent, accepted} and nrcvd ∈ {I0, I1, It+1, In−t} Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 22 / 1

Slide 48

Slide 48 text

Abstraction reﬁnement Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 23 / 1

Slide 49

Slide 49 text

Spurious behavior abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Spurious behavior abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x) ⇒ specs that hold in concrete system may be violated in abstract system spurious counterexamples we have to reduce the behaviors of the abstract system make it more concrete . . . based on the counterexamples = CEGAR Three sources of spurious behavior # processes decreasing or increasing # messages sent = # processes which have sent a message unfair loops Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1

Slide 52

Slide 52 text

Spurious behavior abstraction adds behaviors (e.g., x’=x+1 may lead to x’ being equal to x) ⇒ specs that hold in concrete system may be violated in abstract system spurious counterexamples we have to reduce the behaviors of the abstract system make it more concrete . . . based on the counterexamples = CEGAR Three sources of spurious behavior # processes decreasing or increasing # messages sent = # processes which have sent a message unfair loops . . . and a new abstraction phenomenon Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 24 / 1

Slide 53

Slide 53 text

Parametric abst. reﬁnement — uniformly spurious paths Classic case: Concrete Abstract Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 25 / 1

Slide 54

Slide 54 text

Parametric abst. reﬁnement — uniformly spurious paths Classic case: Concrete Abstract Our case: Concrete n2 , t2 , f2 Concrete n1 , t1 , f1 Abstract · · · · · · Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 25 / 1

Slide 55

Slide 55 text

CEGAR — automated workﬂow Model Checking Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1

Slide 56

Slide 56 text

CEGAR — automated workﬂow Model Checking correct Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1

Slide 57

Slide 57 text

CEGAR — automated workﬂow Model Checking correct Abstraction reﬁnement using SMT counterexample Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1

Slide 58

Slide 58 text

CEGAR — automated workﬂow Model Checking correct Abstraction reﬁnement using SMT counterexample CE feasible: bug Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1

Slide 59

Slide 59 text

CEGAR — automated workflow Model Checking correct Abstraction refinement using SMT counterexample CE feasible: bug CE spurious: refined abstraction Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 26 / 1

Slide 60

Slide 60 text

Slide 61

Slide 61 text

What is SMT? recall SAT: given a Boolean formula, e.g., (¬a ∨ ¬b ∨ c) ∧ (¬a ∨ b ∨ d ∨ e) is there an assignment of true and false to variables a, b, c, d, e such that the formula evaluates to true? Satisﬁability Modulo Theories (SMT) : here just linear arithmetics given a formula, e.g., x = y ∧ y = z ∧ u = x ∧ (x + y ≤ 1 ∧ 2x + y = 1) ∨ 3x + 2y ≥ 3 is there an assignment of values to u, x, y, z such that formula evaluates to true? practically eﬃcient tools: Yices, Z3 Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 27 / 1

Slide 62

Slide 62 text

Counter example: losing processes Output of data abstraction: 16 local states: L = {(sv, ˆ nrcvd) with sv ∈ {v0, v1, sent, accepted} and ˆ rcvd ∈ {I0, I1, It+1, In−t}} An abstract global state is (ˆ k, ˆ nsnt), where ˆ nsnt ∈ {I0, I1, It+1, In−t} and ˆ k : L → {I0, I1, It+1, In−t} Consider an abstract trace: ˆ nsnt1 = I0 ˆ k1 ( ) =      In−t , if = (v1, I0 ) I0, otherwise ˆ nsnt2 = I1 ˆ k2 ( ) =      In−t , if = (v1, I0 ) I1, if = (sent, I0 ) I0, otherwise ˆ nsnt3 = It+1 ˆ k3 ( ) =      In−t , if = (v1, I0 ) It+1, if = (sent, I0 ) I0, otherwise Encode the last state in SMT as a conjunction T of the constraints: resilience condition n > 3t ∧ t ≥ f ∧ f ≥ 0 zero counters (i = 4 ∧ i = 8) → 0 ≤ k3[i] < 1 non-zero counters n − t ≤ k3[4] ∧ t + 1 ≤ k3[8] < n − t system size n − f = k3[0] + k3[1] + · · · + k3[15] UNSAT Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 28 / 1

Slide 63

Slide 63 text

Remove transitions We ask the SMT solver: is there a satisﬁable assignment for T? if yes, then the state is OK, may be part of a real counterexample if not, then the state is spurious remove transitions to that state in the abstract system Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 29 / 1

Slide 64

Slide 64 text

Liveness distributed algorithm requires reliable communication every message sent is eventually received ¬in transit ≡ [∀i. nrcvdi ≥ nsnt] fairness F G ¬in transit necessary to verify liveness, e.g., F G ¬in transit → G ([∀i. svi = v1] → F [∀i. svi = accept]) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 30 / 1

Slide 65

Slide 65 text

Liveness distributed algorithm requires reliable communication every message sent is eventually received ¬in transit ≡ [∀i. nrcvdi ≥ nsnt] fairness F G ¬in transit necessary to verify liveness, e.g., F G ¬in transit → G ([∀i. svi = v1] → F [∀i. svi = accept]) counter example (lasso): ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit s1 ¬in transit s2 sk s3 · · · · · · · · · Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 30 / 1

Slide 66

Slide 66 text

Liveness — fairness suppression ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit s1 ¬in transit s2 sk s3 · · · · · · · · · if there is a spurious sj (all its concretizations violate ¬in transit), then the loop is spurious. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 31 / 1

Slide 67

Slide 67 text

Liveness — fairness suppression ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit s1 ¬in transit s2 sk s3 · · · · · · · · · if there is a spurious sj (all its concretizations violate ¬in transit), then the loop is spurious. reﬁne fairness to F G ¬in transit ∧ G F 1≤j≤k “out of sj Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 31 / 1

Slide 68

Slide 68 text

Liveness — fairness suppression ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit ¬in transit s1 ¬in transit s2 sk s3 · · · · · · · · · if there is a spurious sj (all its concretizations violate ¬in transit), then the loop is spurious. reﬁne fairness to F G ¬in transit ∧ G F 1≤j≤k “out of sj Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 31 / 1

Slide 69

Slide 69 text

experimental evaluation Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 32 / 1

Slide 70

Slide 70 text

Concrete vs. parameterized (Byzantine case) Time to check relay (sec, logscale) Memory to check relay (MB, logscale) Parameterized model checking performs well (the red line). Experiments for ﬁxed parameters quickly degrade (n = 9 runs out of memory). We found counter-examples for the cases n = 3t and f > t, where the resilience condition is violated. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 33 / 1

Slide 71

Slide 71 text

Experimental results at a glance Algorithm Fault Resilience Property Valid? #Reﬁnements Time ST87 Byz n > 3t U 0 4 sec. ST87 Byz n > 3t C 10 32 sec. ST87 Byz n > 3t R 10 24 sec. ST87 Symm n > 2t U 0 1 sec. ST87 Symm n > 2t C 2 3 sec. ST87 Symm n > 2t R 12 16 sec. ST87 Omit n > 2t U 0 1 sec. ST87 Omit n > 2t C 5 6 sec. ST87 Omit n > 2t R 5 10 sec. ST87 Clean n > t U 0 2 sec. ST87 Clean n > t C 4 8 sec. ST87 Clean n > t R 13 31 sec. CT96 Clean n > t U 0 1 sec. CT96 Clean n > t A 0 1 sec. CT96 Clean n > t R 0 1 sec. CT96 Clean n > t C 0 1 sec. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 34 / 1

Slide 72

Slide 72 text

When resilience condition is wrong... Algorithm Fault Resilience Property Valid? #Reﬁnements Time ST87 Byz n > 3t ∧ f ≤ t+1 U 9 56 sec. ST87 Byz n > 3t ∧ f ≤ t+1 C 11 52 sec. ST87 Byz n > 3t ∧ f ≤ t+1 R 10 17 sec. ST87 Byz n ≥ 3t ∧ f ≤ t U 0 5 sec. ST87 Byz n ≥ 3t ∧ f ≤ t C 9 32 sec. ST87 Byz n ≥ 3t ∧ f ≤ t R 30 78 sec. ST87 Symm n > 2t ∧ f ≤ t+1 U 0 2 sec. ST87 Symm n > 2t ∧ f ≤ t+1 C 2 4 sec. ST87 Symm n > 2t ∧ f ≤ t+1 R 8 12 sec. ST87 Omit n ≥ 2t ∧ f ≤ t U 0 1 sec. ST87 Omit n ≥ 2t ∧ f ≤ t C 0 2 sec. ST87 Omit n ≥ 2t ∧ f ≤ t R 0 2 sec. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 35 / 1

Slide 73

Slide 73 text

Summary of results Abstraction tailored for distributed algorithms threshold-based fault-tolerant allows to express diﬀerent fault assumptions Veriﬁcation of threshold-based fault-tolerant algorithms with threshold guards that are widely used Byzantine faults (and other) for all system sizes Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 36 / 1

Slide 74

Slide 74 text

Related work: non-parameterized Model checking of the small size instances: clock synchronization [Steiner, Rushby, Sorea, Pfeifer 2004] consensus [Tsuchiya, Schiper 2011] asynchronous agreement, folklore broadcast, condition-based consensus [John, Konnov, Schmid, Veith, Widder 2013] and more... Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 37 / 1

Slide 75

Slide 75 text

Slide 76

Slide 76 text

Related work: parameterized case Regular model checking of fault-tolerant distributed protocols: [Fisman, Kupferman, Lustig 2008] “First-shot” theoretical framework. No guards like x ≥ t + 1, only x ≥ 1. No implementation. Manual analysis applied to folklore broadcast (crash faults). Backward reachability using SMT with arrays: [Alberti, Ghilardi, Pagani, Ranise, Rossi 2010-2012] Implementation. Experiments on Chandra-Toueg 1990. No resilience conditions like n > 3t. Safety only. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 38 / 1

Slide 77

Slide 77 text

Our current work Discrete synchronous Discrete partially synchronous Discrete asynchronous Continuous synchronous Continuous partially synchronous One instance/ ﬁnite payload Many inst./ ﬁnite payload Many inst./ unbounded payload Messages with reals core of {ST87, BT87, CT96}, MA06 (common), MR04 (binary) one-shot broadcast, c.b.consensus Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 39 / 1

Slide 78

Slide 78 text

Future work: threshold guards + orthogonal features Discrete synchronous Discrete partially synchronous Discrete asynchronous Continuous synchronous Continuous partially synchronous One instance/ ﬁnite payload Many inst./ ﬁnite payload Many inst./ unbounded payload Messages with reals core of {ST87, BT87, CT96}, MA06 (common), MR04 (binary) one-shot broadcast, c.b.consensus DHM12 ST87 AK00 CT96 (failure detector) DLS86, MA06, L98 (Paxos) ST87, BT87, CT96, DAs with failure-detectors DLPSW86 DFLPS13 WS07 ST87 (JACM) FSFK06 WS09 clock sync broadcast approx. agreement Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 39 / 1

Slide 79

Slide 79 text

Thank you! http://forsyte.at/software/bymc Doctoral College: Vienna, Graz, Linz http://logic-cs.at Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 40 / 1

Slide 80

Slide 80 text

the implementation Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 41 / 1

Slide 81

Slide 81 text

Tool Chain: ByMC Parametric Promela code static analysis + Yices Parametric Interval Domain D Parametric data abstraction with Yices Parametric Promela code Parametric counter abstraction with Yices normal Promela code Spin property holds counterexample Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 42 / 1

Slide 82

Slide 82 text

Tool Chain: ByMC Parametric Promela code static analysis + Yices Parametric Interval Domain D Parametric data abstraction with Yices Parametric Promela code Parametric counter abstraction with Yices normal Promela code Spin property holds counterexample Refine Concrete counter representation (VASS) SMT formula Yices counterexample feasible unsat sat Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 42 / 1

Slide 83

Slide 83 text

Tool Chain: ByMC Parametric Promela code static analysis + Yices Parametric Interval Domain D Parametric data abstraction with Yices Parametric Promela code Parametric counter abstraction with Yices normal Promela code Spin property holds counterexample Refine Concrete counter representation (VASS) SMT formula Yices counterexample feasible invariant candidates (by the user) unsat sat Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 42 / 1

Slide 84

Slide 84 text

Experimental setup The tool (source code in OCaml), the code of the distributed algorithms in Parametric Promela, and a virtual machine with full setup are available at: http://forsyte.at/software/bymc Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 43 / 1

Slide 85

Slide 85 text

Running the tool — concrete case user speciﬁes parameter value useful to check whether the code behaves as expected $bymc/verifyco-spin "N=4,T=1,F=1" bcast-byz.pml relay model checking problem in directory “./x/spin-bcast-byz-relay-N=4,T=1,F=1” in concrete.prm parameters are replaced by numbers process prototype is replaced with N − F = 3 active processes Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 44 / 1

Slide 86

Slide 86 text

Running the tool — parameterized model checking PIA data and counter abstraction ﬁnite-state model checking on abstract model $bymc/verifypa-spin bcast-omit.pml relay model checking problem in directory “./x/bcast-byz-relay-yymmdd-HHMM.*” directory contains abs-interval.prm: result of the data abstraction; abs-counter.prm: result of the counter abstraction; abs-vass.prm: auxiliary abstraction for abstraction reﬁnement; mc.out: the last output by Spin; cex.trace: the counterexample (if there is one); yices.log: communication log with Yices. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 45 / 1

Slide 87

Slide 87 text

Fairness, Reﬁnement, and Invariants In the Byzantine case we have in transit : ∀i. (nrcvdi ≥ nsnt) and G F ¬in transit. In this case communication fairness implies computation fairness. But in the abstract version nsnt can deviate from the number of processes who sent the echo message. In this case the user formulates a simple state invariant candidate, e.g., nsnt = K([sv = SE ∨ sv = AC]) (on the level of the original concrete system). The tool checks automatically, whether the candidate is actually a state invariant. After the abstraction the abstract version of the invariant restricts the behavior of the abstract transition system. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 46 / 1

Slide 88

Slide 88 text

Parametric abstraction reﬁnement — justice suppression justice G F ¬in transit necessary to verify liveness Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1

Slide 89

Slide 89 text

Parametric abstraction reﬁnement — justice suppression justice G F ¬in transit necessary to verify liveness counter example: in transit in transit in transit in transit in transit in transit in transit s1 in transit s2 sk s3 · · · · · · · · · if ∀j all concretizations of sj violate ¬in transit, then CE is spurious. Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1

Slide 90

Slide 90 text

Parametric abstraction reﬁnement — justice suppression justice G F ¬in transit necessary to verify liveness counter example: in transit in transit in transit in transit in transit in transit in transit s1 in transit s2 sk s3 · · · · · · · · · if ∀j all concretizations of sj violate ¬in transit, then CE is spurious. reﬁne justice to G F ¬in transit ∧ G F 1≤j≤k ¬at(sj ) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1

Slide 91

Slide 91 text

Parametric abstraction refinement — justice suppression justice G F ¬in transit necessary to verify liveness counter example: in transit in transit in transit in transit in transit in transit in transit s1 in transit s2 sk s3 · · · · · · · · · if ∀j all concretizations of sj violate ¬in transit, then CE is spurious. refine justice to G F ¬in transit ∧ G F 1≤j≤k ¬at(sj ) . . . we use unsat cores to refine several loops at once Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 47 / 1

Slide 92

Slide 92 text

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Parametric abstraction reﬁnement — justice suppression justice G F ¬in transit necessary to verify liveness counter example: in transit in transit in transit in transit in transit in transit in transit s1 in transit s2 sk s3 · · · · · · · · · if ∀j all concretizations of sj violate ¬in transit, then CE is spurious. reﬁne justice to G F ¬in transit ∧ G F 1≤j≤k ¬at(sj ) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 48 / 1

Slide 95

Slide 95 text

Parametric abstraction refinement — justice suppression justice G F ¬in transit necessary to verify liveness counter example: in transit in transit in transit in transit in transit in transit in transit s1 in transit s2 sk s3 · · · · · · · · · if ∀j all concretizations of sj violate ¬in transit, then CE is spurious. refine justice to G F ¬in transit ∧ G F 1≤j≤k ¬at(sj ) . . . we use unsat cores to refine several loops at once Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 48 / 1

Slide 96

Slide 96 text

asynchronous reliable broadcast (srikanth & toueg 1987) the core of the classic broadcast algorithm from the da literature. it solves an agreement problem depending on the inputs vi . Variables of process i vi : {0 , 1} init with 0 or 1 accepti : {0 , 1} init with 0 An indivisible step: i f vi = 1 then send ( echo ) to all ; i f received (echo) from at l e a s t t + 1 distinct processes and not sent ( echo ) before then send ( echo ) to all ; i f received ( echo ) from at l e a s t n - t distinct processes then accepti := 1; Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 49 / 1

Slide 97

Slide 97 text

asynchronous reliable broadcast (srikanth & toueg 1987) the core of the classic broadcast algorithm from the da literature. it solves an agreement problem depending on the inputs vi . Variables of process i vi : {0 , 1} init with 0 or 1 accepti : {0 , 1} init with 0 An indivisible step: i f vi = 1 then send ( echo ) to all ; i f received (echo) from at l e a s t t + 1 distinct processes and not sent ( echo ) before then send ( echo ) to all ; i f received ( echo ) from at l e a s t n - t distinct processes then accepti := 1; asynchronous t byzantine faults correct if n > 3t resilience condition rc parameterized process skeleton p(n, t) Igor Konnov (www.forsyte.at) Checking Fault-Tolerant Distributed Algos TMPA’14, Nov. 2014 49 / 1