Upgrade to Pro — share decks privately, control downloads, hide ads and more …

TLA⁺ and PlusCal

TLA⁺ and PlusCal

Avatar for Torao Takami

Torao Takami

May 11, 2021

Other Decks in Programming

Transcript

  1. Thought about EXTREMELY HARD TO TEST IN DISTRIBUTED SYSTEMS 01

    HOW TO PROVE THE DESIGN IS CORRECT AT FINANCIAL SYSTEMS 02 A FEW CRITICAL PARTS OF AWS AND FF-XV IS TESTED BY TLA⁺[1,2,3] 03
  2. Ideal World Prove the concurrent system is correct by formal

    method. Check each uniprocess functions work well by test framework. Safety Liveness Load & Save Validation Search
  3. TLA+ or PlusCal? Check Similar syntax to temporal logic expressions.

    TLA⁺ (1999) Transpile Similar syntax to general programming languages. PlusCal (2005) ---- MODULE HelloWorld ---- EXTENDS TLC (* --algorithm HelloWorld begin Greeting: print "hello, world"; end algorithm; *) ==== ---- MODULE HelloWorld ---- EXTENDS TLC VARIABLES pc Init == pc = "greeting” Greeting == /¥ pc = "greeting" /¥ PrintT("hello, world") /¥ pc' = "finish” Termination == /¥ pc = "finish” /¥ UNCHANGED pc Next == Greeting ¥/ Termination ==== TLC Too much boilerplate in general program design 🙁 📌 If you see the ¥ (yen) symbol on these pages, all mean ∖ (backslash). 📌
  4. PlusCal coexists with TLA⁺ 1. The PlusCal model is in

    the block comment of TLA⁺ syntax. 2. The EXTENDS likes import or use. It EXTENDS the TLC to use print function in this sample. 3. Step or Action is one or more statements with a label. 4. Then PlusCal translator will insert the TLA⁺ representation and generate model file. ---- MODULE HelloWorld ---- EXTENDS TLC (* --algorithm HelloWorld begin Greeting: print "hello, world"; end algorithm; *) ==== TLA⁺ Syntax Block Comment T4 bacteriophage HelloWorld.tla ---- MODULE HelloWorld ---- EXTENDS TLC (* --algorithm HelloWorld begin Greeting: print "hello, world"; end algorithm; *) ¥* BEGIN TRANSLATION (chksum(pcal) = "8124a86e" /¥ chksum(tla) = "56880286") VARIABLE pc vars == << pc >> Init == /¥ pc = "Greeting" Greeting == /¥ pc = "Greeting" /¥ PrintT("hello, world") /¥ pc' = "Done" (* Allow infinite stuttering to prevent deadlock on termination. *) Terminating == pc = "Done" /¥ UNCHANGED vars Next == Greeting ¥/ Terminating Spec == Init /¥ [][Next]_vars Termination == <>(pc = "Done") ¥* END TRANSLATION ==== 😅 SPECIFICATION Spec ¥* Add statements after this line. HelloWorld.tla HelloWorld.cfg
  5. What is correctness? In concurrent systems, the correctness is Safety

    and Liveness[4]. ü Safety something bad will never happen Ø e.g., A signal for cars and for pedestrians never turn green at the same time. ü Liveness something good will eventually happen Ø e.g., Pedestrians will eventually cross the street.
  6. Temporal Logic assert(x >= y); // present state It’s possible

    to assert only what the state of the “present” is in the test frameworks for general-purpose programming languages. Temporal logic is a logic over tense, dealing with the logic over the present, past, and future. • Globally: □ 𝑥 ≥ 𝑦; It’s satisfied in all the steps. e.g., ∀𝑎 ∈ 𝐴𝑐𝑜𝑢𝑛𝑡𝑠: □𝑎. 𝑏𝑎𝑙𝑎𝑛𝑐𝑒 ≥ 0, 𝑝, 𝑐 ∈ 𝑔𝑟𝑒𝑒𝑛, 𝑟𝑒𝑑 : □¬(𝑝 = 𝑔𝑟𝑒𝑒𝑛 ∧ 𝑐 = 𝑔𝑟𝑒𝑒𝑛) • Eventually: ◇ 𝑥 ≥ 𝑦 ; It’ll be satisfied at some step. e.g., ∃𝑛 ∈ 𝑁𝑜𝑑𝑒𝑠: ◇𝑛. 𝑣𝑜𝑡𝑒𝑠 > " # 𝑁𝑜𝑑𝑒𝑠. 𝑠𝑖𝑧𝑒 • Led to: 𝑥 ≥ 𝑦 ↝ 𝑥 ≥ 𝑧 ; 𝑥 ≥ 𝑧 will be met at some step after 𝑥 ≥ 𝑦 is met. Same as 𝑥 ≥ 𝑦 ⇒ ◇𝑥 ≥ 𝑧. • Or else. 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 ≥ 𝑦 𝑥 < 𝑦 𝑥 ≥ 𝑦 Success Failure 𝑥 < 𝑦 𝑥 < 𝑦 𝑥 < 𝑦 𝑥 ≥ 𝑦 𝑥 < 𝑦 𝑥 < 𝑦 𝑥 < 𝑦 𝑥 < 𝑦 𝑥 < 𝑦 𝑥 < 𝑦 Success Failure How do we express “Never” and “Eventually” as constraints? This is where we need to introduce the notion of temporal logic.
  7. How does TLA⁺ ensure the safety and liveness properties? The

    TLC model checker: 1. considers the model (the specification written in TLA⁺) as a finite state machine. 2. runs the model step-by-step. 3. explores the state transition by reproducing all possible states, step progresses, and non-deterministic behaviors combinations of the system. 4. then checks all the conditions of temporal logic, assertion, termination option, etc. are satisfied for the graph. Informally, TLC internally builds a state diagram like the one in UML. ---- MODULE Steps ---- EXTENDS Integers, TLC (* --algorithm steps variables s = [i ¥in 1..2 |-> 0]; process P ¥in 1..2 begin Step1: s[self] := 1; print s; Step2: s[self] := 2; print s; end process; end algorithm; *) ==== A B C D E F G H I J Z Model (TLA⁺) State Transition Safety • Checks global temporal properties. They must be satisfied for all reachable states. • Detects livelock (closed circuit). Liveness • Checks eventual temporal properties. They must be satisfied at somewhere state, including termination. • Detects deadlock and infinite waiting for events that never happen. The state transition does NOT mean a control flow.
  8. PlusCal’s syntax is very similar to unit-test code, but it

    does NOT test the branches and loops over control flow. How to reproduce “all possible states”? ---- MODULE States ---- EXTENDS Integers, TLC (* --algorithm States variables x ¥in 1..3, y ¥in 1..2; begin Step1: x := x % 2; y := y % 2; Step2: assert x >= 0 /¥ y >= 0; print << x, y >>; end algorithm; *) ==== (1,1) (2,1) (3,1) (1,2) (2,2) (3,2) (0,0) (0,1) (1,0) (1,1) Init Step1 Step2 (𝑥, 𝑦) (0,0) (0,1) (1,0) (1,1) Terminating Number of states resulting by the step (boxes) Number of actions performed on the step (arrows) ▶Result State variables of this system Step; a unit that TLC progresses in one cycle.
  9. How to reproduce “all possible step progress”? ---- MODULE Steps

    ---- EXTENDS Integers, TLC (* --algorithm steps variables s = [i ¥in 1..2 |-> 0]; process P ¥in 1..2 begin Step1: s[self] := 1; print s; Step2: s[self] := 2; print s; end process; end algorithm; *) ==== (1,0) (0,1) Init (0,0) (2,2) Terminating 𝑠 (1,1) (2,1) (1,2) (2,0) (0,2) Step1 Step1 Step2 Step2 Step1 Step1 Step1 Step1 Step2 Step2 Step2 Step2 Process 1 Process 2 ▶Result Assume the state variable 𝑠 = (𝑠! , 𝑠" ), where 𝑠# is the step where process 𝑖 currently in. • There are Step1×6 and Step2×6 actions. • Step1 and Step2 both results 3 distinct states, and 2 states are resulted from any one of them. PlusCal also inspects combinations of progress by representing the progress of each process as state variables. This makes it possible to detect the race-condition problems. In transpiling to TLA⁺, a variable pc, which has the same role as the 𝑠, is implicitly introduced. Two processes
  10. either and with are syntaxes to describe dynamical state transition,

    e.g., user input or server response. ---- MODULE ReadPassword ---- EXTENDS Integers, TLC (* --algorithm ReadPassword variables try = 0, input = ""; begin ReadPassword: while input /= "success!" /¥ try < 3 do either input := "success!"; or input := "failure?"; end either; try := try + 1; end while; Finally: assert input = "success!" ¥/ try = 3; print << try, input >> end algorithm; *) ==== (1,”success!”) (0,””) (1,”failure?”) (2,”success!”) (2,”failure?”) (3,”success!”) (3,”failure?”) For example, Try up to three times until the correct password is entered. either 𝐴; or 𝐵; or ⋯ end either; with 𝑥 ∈ {𝐴, 𝐵, ⋯ } do ⋯ end with; (try,input) These would make it possible to represent Byzantine behavior such as double-signatures and tampering. How to reproduce “non-deterministic behaviors”? Init
  11. The TLC stops when all the paths in the state

    transition have been explored. Note that the exploration stops even for the system doesn’t terminate. ---- MODULE InfiniteLoop ---- EXTENDS TLC (* --algorithm InfiniteLoop variables flag = FALSE; begin Loop: while TRUE do flag := ~flag; end while; end algorithm; *) ==== Termination of Exploring The third and subsequent Loop step with flag=FALSE has already been inspected. FALSE flag Init TRUE Loop Loop
  12. ---- MODULE Crosswalk ---- EXTENDS Integers, TLC (* --algorithm Crosswalk

    variables signal = << "green", "red" >>, define DontGreenAtTheSameTime == [](~(signal[1] = "green" /¥ signal[2] = "green")) Flip(s) == IF s = "green" THEN "red" ELSE "green" end define; begin Switch: while TRUE do S1: signal[1] := Flip(signal[1]); S2: signal[2] := Flip(signal[2]); end while; end algorithm; *) ==== How to ensure the safety property? Define constraints with a [] temporal operator and specify them as PROPERTIES in the model file. This example shows two traffic lights that’s expected to never turn green at the same time, but the accident case does exist. SPECIFICATION Spec ¥* Add statements after this line. PROPERTIES DontGreenAtTheSameTime signal Init S1 S2 S1 Invariant NoAccidents is violated. Not only temporal properties but also assert features can be used.
  13. Practical way to specify safety • Use Global Temporal Properties

    1. Define temporal properties with [] in model. NoOverdrafts == ∀𝑎 ∈ 𝑎𝑐𝑐𝑜𝑢𝑛𝑡𝑠: □𝑎. 𝑏𝑎𝑙𝑎𝑛𝑐𝑒 ≥ 0 2. Specify them in the PROPERTIES of its model file. PROPERTIES NoOverdrafts • Use Invariants 1. Define invariant rules. PMustBePrime == 𝑝 > 1 ∧ ¬(∃𝑥 ∈ 2, . . , 𝑝 − 1 : 𝑝 % 𝑥 = 0) 2. Specify them in the INVARIANTS of its model file. INVARIANTS PMustBePrime • Use Assertions assert x + y > 0 Usually use “globally” temporal properties and ad-hoc assertions. Invariants can also be used for global requirements, but the meaning is a substitute for type restriction. ASSUME is also a constraint but it’s on constants defined in the model file.
  14. Liveness is a little difficult and a large amount of

    computation. How to ensure the liveness property? Stuttering • Stuttering is a cycle in which there are enabled steps, but none of them are executed. • Finite stuttering doesn’t affect the proving, but TLA⁺ allows infinite stuttering. • For concurrent system, infinite stuttering represents a crash-stop failure of process. • In TLA⁺, any default process may do infinite stuttering at any step. Fairness • A process (or algorithm, step) with fairness property avoids infinite stuttering and ensures that enabled steps will eventually proceed. • IOW, there is an implicit assumption that processes with the fair keyword will never crash-stop. • In multi-process, strong fairness fair+ may be required to proceed through the process of repeated enabled and disabled by await. Before explaining the Liveness property, there are two important notions in TLA⁺ that should be understood.
  15. How to ensure the liveness property? Define constraints with a

    <> temporal operator and specify them as PROPERTIES in the model file. This example shows a system consists of a traffic light that switches between green and red and a pedestrian who can cross the road when it turns green. What happens the light doesn’t turn green, or the process lacks fairness? A coordination system with a pedestrian crossing the street and traffic light. ---- MODULE Crosswalk ---- (* --algorithm Crosswalk variables signal ¥in {"green", "red"}, went_through = FALSE; define PedestrianHasCrossed == <>went_through Flip(s) == IF s = "green" THEN "red" ELSE "green" end define; fair+ process Pedestrian = "pedestrian" begin Wait: await signal = "green"; WentThrough: went_through := TRUE; end process; fair process Signal = "signal" begin Switch: while TRUE do signal := Flip(signal); end while; end process; end algorithm; *) ==== SPECIFICATION Spec ¥* Add statements after this line. PROPERTIES PedestrianHasCrossed
  16. Try to violate the liveness property Q3. What happens when

    change the Pedestrian to weak fairness? ¥* fair+ process Pedestrian = "pedestrian" fair process Pedestrian = "pedestrian" A3. There is a possibility of an endless repetition of stuttering as soon as the signal turns green, so the pedestrian cannot cross the road. ¥* fair process Signal = "signal" process Signal = "signal" A2. In case the light turns on red and signal crash-stops (infinite stutters), the pedestrian cannot cross the road. Q2. What happens if erase fair from the Signal? Q1. What happens if the light never turn on green? ¥* Flip(s) == IF s = "green" THEN "red" ELSE "green" Flip(s) == "red" A1. It'll fail because finish the state graph explore without crossing the road.
  17. How to ensure the termination? -termination option can be used

    as an easy way to inspect for system termination. This Termination property is represented like follows PlusCal options (-termination) ---- MODULE InfiniteLoop ---- EXTENDS TLC (* --fair algorithm InfiniteLoop variables flag = FALSE; begin Loop: while TRUE do flag := ~flag; end while; Finish: print "finish" end algorithm; *) ==== The system-wide termination isn’t necessarily required in abstracted system design. You may define and use your own termination as temporal property instead of Termination. SPECIFICATION Spec PROPERTY Termination ¥* Add statements after this line. 𝑇𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑖𝑜𝑛 ≝ ◇(∀𝑠𝑒𝑙𝑓 ∈ 𝑃𝑟𝑜𝑐𝑆𝑒𝑡: 𝑝𝑐 𝑠𝑒𝑙𝑓 = "Done") In this case, all processes must have fair property because infinite stuttering of even only one process will cause the termination violation.
  18. Practical way to specify liveness • Use Eventual Temporal Properties

    1. Define temporal properties with <> in model. TransferCompleteness == ◇∀𝑎 ∈ 𝑎𝑐𝑐𝑜𝑢𝑛𝑡𝑠: □𝑎. 𝑏𝑎𝑙𝑎𝑛𝑐𝑒 ≥ 0 2. Specify them in the PROPERTIES of its model file. PROPERTIES NoOverdrafts • Use –termination option 1. Specify 1. PlusCal options (-termination) in source, 2. -termination on command-line. 2. Specify fair to overall of algorithm or all the processes. --fair algorithm … If you simply want to check the system termination, it’s easy to use the - termination option.
  19. Isn't TLA⁺ prone to a computational explosion? It’s easy to

    cause a combinatorial explosion. The number of actions and states are combinations of: • the domain of each state variables • order of steps in multi-process • non-deterministic cases Abstraction is the most important and difficult[5] task in TLA⁺. variables PublicKey ∈ {1,…,0xFFFFFFFFFFFFFFFF}, Signature ∈ {1,…,0xFFFFFFFFFFFFFFFF}; begin Step1: if Verify(PublicKey, Signature) then …; 2"#×2"# combinated cases will be created. Not practical values, But abstracted meanings either (* if the specified signature is valid *) … or (* or not *) … end either; But there are no individual 2$%×2$% states that make sense, are there? If you really need a test like this one, you should do it in a unit testing framework. What “valid” means should be revealed and should be tested in the implementation. Abstraction is a subjective work. How much do you believe in “it has been proven“? Liveness test is particularly computationally intensive, and in many cases it has to be done with simpler models.
  20. Conclusion • TLA⁺ and PlusCal inspect the system specification and

    explore the state transition in the perspective of a finite state machine (finite automaton). The states also means the progress of each process. • TLA⁺ uses temporal logic operators to verify each states for safety and liveness in abstracted concurrent systems. • Abstraction is the most important but difficult work. It’s a very subjective process so it’s necessary to review carefully what has been adopted and what has been omitted, without blindly believing in “proven”.
  21. References 1. Use of Formal Methods at Amazon Web Services

    https://lamport.azurewebsites.net/tla/formal-methods-amazon.pdf 2. How Amazon Web Services Uses Formal Methods https://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-services-uses-formal-methods/fulltext 3. FINAL FANTASY XV POCKET EDITION を⽀える AWS サーバレス技術 https://d1.awsstatic.com/events/jp/2018/summit/tokyo/customer/37.pdf 4. Proving the Correctness of Multiprocess Programs (1977) L. Lamport https://ieeexplore.ieee.org/document/1702415 5. The TLA+ Video Course (2020) L. Lamport http://lamport.azurewebsites.net/video/videos.html (recommended to watch first) 6. PlusCal / TLA⁺: An Annotated Cheat Sheet (2019) S.Merz https://d3s.mff.cuni.cz/f/teaching/nswi101/pluscal.pdf 7. Summary of TLA⁺ https://lamport.azurewebsites.net/tla/summary-standalone.pdf
  22. Theorem Proving Formal Specification Other Formal Methods(not accurate) Automated Theorem

    Proving (E, SPASS, Otter, etc.) Interactive Proof System (Coq, Alloy, F*, Agda, Isabelle, etc.) Abstractive Logical Mathematical Find a theorem that leads from 𝐴 to 𝑍 in a path-finding manner from given assumptions 𝐴, 𝐵, … , 𝑍. Specification Language (TLA⁺, VDM, Z, etc.) Concrete Implementation-oriented Industrial Verify no inconsistencies or omissions for given coded logic. Test Implementation Engineering
  23. How to check the model of TLA⁺ and PlusCal 1.

    Install Visual Studio Code and add TLA+ extension. 2. Write your model in Module.tla file. Make sure the file name matches module name, ---- MODULE Module ----. 3. For PlusCal, select [View]-[Command Palette...] menu or press [CTRL]+[Shift]+[P], and select “TLA+: Parse module” to transpile. Then TLA⁺ code will be inserted and module.cfg file created. 4. Right click on the file and run TLC with “Check model with TLC”.
  24. Syntax for describing sequential processing if 𝑃 then 𝐴; elsif

    𝐵; elsif ⋯ else ⋯ end if; Label: while 𝑃 do 𝐴; end while; goto Label; IF 𝑃 THEN 𝐴; ELSE 𝐵; END IF; CASE 𝑃 -> 𝐴; [] 𝑄 -> 𝐵; [] 𝑅 -> 𝐶; OTHER -> 𝐷; PlusCal doesn’t have a paradigm that makes it easy to write complex operations like OOP or functional languages. The reason for this is probably that they make the state transitions of finite state machine too complicated. Expression. It doesn’t have “else if”. Expression Statement. TLA⁺ Syntax PlusCal Syntax ”for” is not provided. That’s all?? 😇 See also: [6][7] for other language features.
  25. Multiprocess Messaging • State variables are also used for inter-

    process messaging. • There is a feature called await that will not activate the step until a condition becomes true. variables greeting = ""; process A = "A" begin WaitMessage: await greeting /= ""; Print: print greeting; end process process B = "B" begin Say: greeting = "hello, world"; end process
  26. Operators and Notations TLA⁺ Notations Math Expressions Meanings x :=

    y Assign the value of 𝑦 to 𝑥. x == 10 ≜ Define 𝑥 as 10. x ¥in {A,B,C} 𝑥 ∈ {𝐴, 𝐵, 𝐶} Let 𝑥 be an element of the set {𝐴, 𝐵, 𝐶}. x ¥in 1..9 𝑥 ∈ {1, … , 9} Let 𝑥 be an element of the set {1, … , 9}. P /¥ Q 𝑃 ∧ 𝑄 Logical disjunction of 𝑃 and 𝑄. P ¥/ Q 𝑃 ∨ 𝑄 Logical conjunction of 𝑃 and 𝑄. ~P ¬𝑃 Negation of 𝑃. x = y 𝑥 = 𝑦 𝑥 equals to 𝑦. x /= y 𝑥 ≠ 𝑦 𝑥 doesn’t equal to 𝑦. P ¥equiv Q 𝑃 ≡ 𝑄 𝑃 equivalents to 𝑄. ¥A x ¥in S: P(x) ∀𝑥 ∈ 𝑆: 𝑃(𝑥) For all elements 𝑥 in the set 𝑆, 𝑃(𝑥) is satisfied. ¥E x ¥in S: P(x) ∃𝑥 ∈ 𝑆: 𝑃(𝑥) There exists an element x in the set S that satisfies P(x). +,-,*,/,% Arithmetic operators x ¥o y 𝑥 ∘ 𝑦 Concatenate the tuples or strings. CHOOSE x ¥in S: P(x) Select arbitrarily element 𝑥 from the set 𝑆 that satisfies 𝑃(𝑥). TLA⁺ Nots. Meanings "abc" String literal. <<A,B,C>> Tuple or list (𝐴, 𝐵, 𝐶) [a|->1,b|->2] Struct {a:1, b:2} [i ¥in 1..3|->i*i] Tuple (1, 4, 9) ToString(number) Len(tuple) See also: [6][7] for other language features.