Exploring the coordination lattice

Transcript

1. Exploring the Coordination Lattice: Selecting Concurrency Control for Distributed Applications Sreeja S. Nair and Marc Shapiro 08/10/2021
2. Centralized Application 2
3. Distributed Application 3
4. Distributed Auction Sreeja Marc Gustavo 4
5. Distributed Auction Sreeja Marc Gustavo 4
6. 100 Distributed Auction Sreeja Marc Gustavo 4 100 100
7. 100 Distributed Auction Sreeja Marc Gustavo 4 100 100
8. 100 Distributed Auction Sreeja Marc Gustavo 4 100 100 105 105
9. 100 Distributed Auction Sreeja Marc Gustavo 4 100 100 105 105
10. 100 Distributed Auction Sreeja Marc Gustavo 4 100 100 105 105
11. 100 Distributed Auction Sreeja Marc Gustavo 4 100 100 105 105
12. Auction : Static Analysis Con fl icts detected 5 • Place bid - close auction auction • Place bid - unregister buyer buyer • Close auction - remove bid auction
13. Auction : Static Analysis Con fl icts detected 5 • Place bid - close auction auction • Place bid - unregister buyer buyer • Close auction - remove bid auction Using Distributed Locks
14. Constraints imposed by the system and application • Coordination to ensure safety • Skewed workload in locality and frequency • Network latency • Cost of locking Variables that can be controlled • Granularity • Type of lock - Mutual exclusion - Shared/exclusive lock • Placement of lock - Distance between client and lock 6 Selecting Ef fi cient Concurrency Control for Safety
15. Coordination needed to ensure safety using locks 7 Safety constraint Place bid Close auction Unregister buyer Remove bid auction auction buyer
16. Skewed in frequency per operation 8 Workload constraint Place bid Close auction Unregiste r buyer Remove bid 90% 1% 7% 2% Frequency of Place bid operation for a single auction Paris Houston Singapore 95% 5% 0% Skewed in frequency per replica
17. Network latency 9 Network constraint Paris Houston Singapore 75 ms 55 ms 125 ms
18. 10 Lock cost constraint Function of network latency and computation Di ff erent for di ff erent placement; same for both lock modes
19. Constraints Safety constraint • Locks to ensure safety Workload constraint • Skewed workload in locality and frequency Network constraint • Latency between replicas Lock cost constraint • Cost of locking as a function of computation and network latency 11
20. 12 Granularity variable Place bid Close auction Unregister buyer Remove bid auction buyer Place bid Close auction Unregister buyer Remove bid auction auction buyer Place bid Close auction Unregister buyer Remove bid auction buyer + auction Place bid Close auction Unregister buyer Remove bid buyer + auction Coarsening Coarsening Coarsening Coarsening
21. Effect of granularity Coarsening Lock acquisition time Contention Place bid Close auction Unregister buyer Remove bid auction auction buyer Place bid Close auction Unregister buyer Remove bid auction buyer + auction Place bid Close auction Unregister buyer Remove bid buyer + auction 2 locks 1 lock 1 lock 13
22. 14 Mode of locking variable Mutual exclusion Shared/exclusive lock Place bid Close auction Remove bid auction Mutex Mutex Mutex Exclusive mode introduces contention Shared mode allows parallel execution Place bid Close auction Remove bid auction Shared Shared Exclusive Place bid Close auction Remove bid auction Exclusive Exclusive Shared
23. 14 Mode of locking variable Mutual exclusion Shared/exclusive lock Place bid Close auction Remove bid auction Mutex Mutex Mutex Exclusive mode introduces contention Shared mode allows parallel execution Place bid Close auction Remove bid auction Shared Shared Exclusive Place bid Close auction Remove bid auction Exclusive Exclusive Shared
24. 15 Lock placement variable Distance between client and lock affects lock acquisition time Paris Houston Singapore 95% 5% 0% Frequency of Place bid operation for a single auction
25. Selecting Coordination Con fi guration Placement - Minimize acquisition cost Mode - Maximize parallelism allowed Granularity - Minimize execution time 16
26. The Coordination Lattice Place bid Close auction Unregister buyer Remove bid a1 a2 b Coordination Element 17 b a1 a2 b a1a2 ba1 a2 ba1a2
27. Coordination Element 18 b a1a2 Mode b unregister buyer place bid MX EX EX SX1 EX SH SX2 SH EX mode placement a1a2 MX SX1 SX2 H ouston Paris Singapore Mode a1a2 place bid close auction remove bid MX EX EX EX SX1 EX SH EX SX2 SH EX SH Coordination con fi guration mode placement b MX SX1 SX2 H ouston Paris Singapore
28. Navigating the coordination lattice CcRepExecTime = ∑ r∈R (CcOpSerial(r) + CcRepSerial(r) + CcOpParallel(r)) |R| Minimize 19 CcOpSerial(r) = Impact of serialization due to exclusive mode of locking CcRepSerial(r) = Impact of serialization due to serialization inside a replica CcOpParallel(r) = Parallelism allowed by shared mode of locking Cost of locking is impacted by lock placement, a component inside all the metrics
29. Two operation con fl ict x y a Placement 1 Houston 2 Paris 3 Singapore Mode x y 1 EX EX 2 EX SH 3 SH EX granularity-mode-placement Execution time (ms) Worklo ad H P S x 0 500 0 y 250 0 250 20
30. Two operation con fl ict x y a Placement 1 Houston 2 Paris 3 Singapore Mode x y 1 EX EX 2 EX SH 3 SH EX granularity-mode-placement Execution time (ms) Worklo ad H P S x 0 500 0 y 250 0 250 20 1. Colocate lock with the workload 2. Choose shared mode for the most frequently distributed operation
31. Three operations con fl ict x y z a1 a2 a1 , a2 a1 a2 Placement a1 a2 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode a1 a2 x y y z 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X Placement abc 1 H 2 P 3 S Mode abc x y z 1 X X X 2 X S X 3 S X S Workload H P S x 0 1 0 y 100 100 100 z 0 1 0 21
32. 22 Execution time (ms) granularity-mode-placement Three operations con fl ict Workl oad H P S x 0 1 0 y 100 100 100 z 0 1 0 Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X Mode abc place close remove 1 X X X 2 X S X 3 S X S Placement abc 1 H 2 P 3 S x y z a1 a2
33. 23 Execution time (ms) granularity-mode-placement Three operations con fl ict Workl oad H P S x 100 100 0 y 0 0 0 z 0 0 50 Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X Mode abc place close remove 1 X X X 2 X S X 3 S X S Placement abc 1 H 2 P 3 S x y z a1 a2
34. 23 Execution time (ms) granularity-mode-placement Three operations con fl ict Workl oad H P S x 100 100 0 y 0 0 0 z 0 0 50 Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X Mode abc place close remove 1 X X X 2 X S X 3 S X S Placement abc 1 H 2 P 3 S Coarsening is a trade-off between contention and acquisition cost x y z a1 a2
35. Future Work Probabilistic constraint model Support dynamic recon fi guration of coordination con fi guration More concurrency control mechanisms 24
36. Conclusion Coordination lattice is a fi rst step to systematically explore performance implications of concurrency control A step towards engineering Just-Right-Consistent distributed applications 25 Selecting optimal distributed lock con fi guration • Coordination lattice • Metrics
37. Back up
38. Auction : Static Analysis Con fl icts detected • Add product - unregister seller • Create auction - unregister seller • Add to lot - add to lot • Add to lot - start auction • Add to lot - remove auction • Add to lot - remove product • Start auction - remove from lot • Place bid - close auction • Place bid - unregister buyer • Close auction - remove bid 27
39. Auction : Synchronisation Condition Generation 28 • Add product - unregister seller seller • Create auction - unregister seller seller • Add to lot - add to lot seller, (product, seller) • Add to lot - start auction auction • Add to lot - remove auction auction • Add to lot - remove product (product, seller) • Start auction - remove from lot auction • Place bid - close auction auction • Place bid - unregister buyer buyer • Close auction - remove bid auction
40. Auction : Grouping 29 Add product Unregister seller Create auction Place bid Close auction Unregister buyer Remove bid Register seller Register buyer Add to lot Start auction Remove from lot Remove auction Remove product
41. Auction : Filtering 30 Add product Unregister seller Create auction Place bid Close auction Unregister buyer Remove bid Register seller Register buyer Add to lot Start auction Remove from lot Remove auction Remove product
42. Auction : Filtering 30 Add product Unregister seller Create auction Place bid Close auction Unregister buyer Remove bid Add to lot Start auction Remove from lot Remove auction Remove product
43. Placement Centralised (P) vs Clustered(P, C, N) vs Distributed
44. Cost of locking
45. Auction Place bid Close auction auction a b lab−auction Placement 1 Houston 2 Paris 3 Singapore Mode place bid close auction 1 X X 2 X S 3 S X granularity-mode-placement Execution time (ms) Work load H P S place 0 1000 0 close 0 0 0 33
46. Auction Place bid Close auction auction a b lab−auction Placement 1 Houston 2 Paris 3 Singapore Mode place bid close auction 1 X X 2 X S 3 S X granularity-mode-placement Execution time (ms) Work load H P S place 333 333 333 close 0 0 0 34
47. Auction place bid close auction remove bid a1 a2 a1 , a2 a1 a2 Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X Placement abc 1 H 2 P 3 S Mode abc place close remove 1 X X X 2 X S X 3 S X S Workload H P S place 455 455 0 close 37 37 0 remove 8 8 0 35
48. Auction Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X g-m-p Execution time (ms) Placement abc 1 H 2 P 3 S Mode abc place close remove 1 X X X 2 X S X 3 S X S Workload H P S place 455 455 0 close 37 37 0 remove 8 8 0 36
49. Auction Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X g-m-p Execution time (ms) Placement abc 1 H 2 P 3 S Mode abc place close remove 1 X X X 2 X S X 3 S X S Workload H P S place 455 455 0 close 37 37 0 remove 8 8 0 36 1. Colocate lock with the workload 2. Choose shared mode for the most frequently distributed operation 3. Coarsening is a trade-off between contention and acquisition cost
50. Auction Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X g-m-p Execution time (ms) Placement abc 1 H 2 P 3 S Mode abc place close remove 1 X X X 2 X S X 3 S X S Workload H P S place 0 909 0 close 0 75 0 remove 0 15 0 37 Granularity 1 - 2 locks Granularity 2 - single lock
51. Auction Placement ab bc 1 H H 2 P H 3 S H 4 H P 5 P P 6 S P 7 H S 8 P S 9 S S Mode ab bc place close close remove 1 X X X X 2 X S X X 3 S X X X 4 X X X S 5 X S X S 6 S X X S 7 X X S X 8 X S S X 9 S X S X g-m-p Execution time (ms) Placement abc 1 H 2 P 3 S Mode abc place close remove 1 X X X 2 X S X 3 S X S Workload H P S place 303 303 303 close 25 25 25 remove 5 5 5 38 Granularity 1 - 2 locks Granularity 2 - single lock