Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bug-Free Concurrency in Rust

mox692
December 16, 2024
3

Bug-Free Concurrency in Rust

A presentation slide from the Tokyo Rust Meetup in October 2024, discussing Rust's loom crate and model checking.

mox692

December 16, 2024
Tweet

Transcript

  1. • Work at Cyberagent, Inc. • Rust Developer 🦀 •

    Used to use Scala • A member of tokio-rs Hi, I’m Motoyuki(@mox692) 👋 2
  2. 4

  3. 5

  4. Goal • Introduction to the loom library • Understand the

    inner workings ◦ High level idea of the model checking ◦ What the Dynamic Partial Order Reduction is ◦ How loom handle memory order 6
  5. 8

  6. 9

  7. Why so hard? • Non-determinism of thread scheduling • There

    are so many cases we have to care about 10
  8. 12

  9. 13

  10. 14

  11. What about Rust? • Rust’s type system can prevent certain

    kinds of multi-threaded problems • But NOT all problems 15
  12. 17

  13. Much effort / research is going on around this area

    • Theorem Proving ◦ Coq, Agda, Lean • Model Checking ◦ TLA+ ◦ Loom 🦀 19
  14. 20

  15. • Loom explores all possible permutations of thread scheduling •

    It uses DPOR, one of the model checking techniques • Used in many places in the tokio codebase 🗼 21
  16. 23 #[test] fn test_buggy_concurrent_logic() { let v1 = Arc::new(AtomicUsize::new(0)); let

    v2 = v1.clone(); thread::spawn(move || v1.store(1, Relaxed)); let v = v2.load(Relaxed); assert_eq!(1, v); }
  17. 24 #[test] fn test_concurrent_logic_loom() { loom::model(|| { let v1 =

    Arc::new(AtomicUsize::new(0)); let v2 = v1.clone(); thread::spawn(move || v1.store(1, Relaxed)); let v = v2.load(Relaxed); assert_eq!(0, v); }); }
  18. 29

  19. 30

  20. How to model program states? • Once away from Rust

    • How to explore all program states? 🤔 32
  21. State and Transition • Program has State S i •

    t i is a Transition that changes the state from S i to S i+1 a = 0 a = 1 a = 2 S 0 S 1 S 2 t 0 t 1 ・・・ 33
  22. So, • All we have to do is just search

    for all permutations • Easy-peasy 😆 38
  23. 39 • 2 thread, 10 lines of code ◦ →

    2^10 = 1024 • 2 thread, 20 lines of code ◦ → 2^20 = 1,048,576 • 5 thread, 20 lines of code ◦ → 5^20 = 9.5367432e+13 …
  24. 40

  25. x = 1 P = 1 x = 1 y

    = 2 x = 1 P = 1 P = 1 x = 1 P = 1 a = 1 x = 1 y = 2 z = 3 x = 1 y = 2 P = 1 x = 1 P = 1 y = 2 x = 1 P = 1 a = 1 P = 1 x = 1 y = 2 P = 1 x = 1 a = 1 P = 1 a = 1 x = 1 P = 1 a = 1 b = 2 41
  26. x = 1 P = 1 x = 1 y

    = 2 x = 1 P = 1 P = 1 x = 1 P = 1 a = 1 x = 1 y = 2 z = 3 x = 1 y = 2 P = 1 x = 1 P = 1 y = 2 x = 1 P = 1 a = 1 P = 1 x = 1 y = 2 P = 1 x = 1 a = 1 P = 1 a = 1 x = 1 P = 1 a = 1 b = 2 42
  27. Partial Order Reduction • Usually, program has several semantically identical

    transitions. • Try to reduce those identical transitions by introducing the dependency concept. 43
  28. Dependency • Independent Operation ◦ If reordering the two operations

    does not change the final state of the program, they are independent ◦ e.g. read from the thread local variables • Dependent Operation ◦ e.g. acquire a lock from the Mutex 44
  29. 46 thread::spawn(move || { println!("thread 2"); v1.store(1, Relaxed); }); println!("thread

    1"); assert_eq!(0, v2.load(Relaxed)); reordering doesn’t change the final state
  30. 47 thread::spawn(move || { println!("thread 2"); v1.store(1, Relaxed); }); println!("thread

    1"); assert_eq!(0, v2.load(Relaxed)); reordering doesn’t change the final state ↓ Independent Operation
  31. println!(“thread 1”) S 0 49 assert_eq() println!(“thread 1”) v1.store println!(“thread

    2”) println!(“thread 2”) ・・・ Interleaving between “independent operations”
  32. println!(“thread 1”) S 0 50 assert_eq() println!(“thread 1”) v1.store println!(“thread

    2”) println!(“thread 2”) ・・・ Based on the POR notion, we can reduce a bunch of execution paths
  33. Dynamic Partial Order Reduction • Historically, POR was a static

    analysis-based method • DPOR checks the state space at runtime • Using backtracking + DFS, search for all permutations of threads 52
  34. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 1st Execution dependent access 54
  35. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 1st Execution Backtracking Point List (Stack) [S 1 ] save as a backtrack point 55
  36. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 S 3 1st Execution Backtracking Point List (Stack) dependent access again!! [S 1 ] 56
  37. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 S 3 S 4 S 5 thread1 thread3 1st Execution [S 3 , S 1 ] Backtracking Point List (Stack) save as another backtrack point push 57
  38. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 S 3 S 4 S 5 thread1 thread3 1st Execution [S 3 , S 1 ] Backtracking Point List (Stack) End push 58
  39. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 S 3 2nd Execution Backtracking Point List (Stack) [S 1 ] S 3 pop 59
  40. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 S 3 S 4 S 5 thread1 thread3 2nd Execution [S 1 ] Backtracking Point List (Stack) S 3 pop 60
  41. S 0 S 1 thread1 3rd Execution [S 1 ]

    Backtracking Point List (Stack) 61
  42. S 0 S 1 S 2 S 6 thread1 thread1

    thread2 S 3 S 4 S 5 thread1 thread3 3rd Execution [] Backtracking Point List (Stack) S 1 pop 62
  43. Dynamic Partial Order Reduction • If we find a state

    that needs to be explored, add it as a backtracking point at runtime • Repeat the program until all backtrack points are explored 63
  44. 66

  45. Memory Ordering • The rules how memory operations are executed

    and observed in concurrent systems • DPOR only takes into account a sequential execution model 67
  46. 71 thread::spawn(move || v1.store(1, Relaxed)); let v = v2.load(Relaxed); assert_eq!(1,

    v); Even after the store, it's possible to see 0 ↓ Algorithm should explore all possible load’s result ➡ // -> 1 or 0
  47. S 0 S 1 thread1 Backtracking Point List (Stack) []

    72 Read involving memory ordering
  48. S 0 S 1 S 2 S 3 thread1 load

    result1 Backtracking Point List (Stack) [S 1 ] save as a backtrack point 73 load result2 Read involving memory orders
  49. • To handle the memory model, we particularly pay attention

    to read operation • In addition to DPOR's thread execution backtrack points, load result selection based on memory models is also added to the backtracking points 74
  50. 75

  51. • Based on previous DPOR paper, discuss how to handle

    C/C++ memory model • Implementation available (GitHub) 78
  52. Need help with • Implementation clean up • Unimplemented types

    • PR reviews • Multi-threading support 80
  53. Unimplemented Feature • SeqCst accesses (e.g. load, store, ..) ◦

    fence(SeqCst) is supported. ◦ issue: #180 81
  54. Appendix: Difference from other tools • Similar tools in Rust

    ◦ https://crates.io/crates/shuttle 85