Scale, USL and DDD: A Curiouser World

Transcript

1. Scale, Multiversions and the Curiouser World Shripad Agashe CTO and Co-Founder @Shibika Twitter: ShripadAgashe
2. The changed person It's no use going back to yesterday, because I was a different person then. Source: https://commons.wikimedia.org/wiki/File:John_Tenniel_-_Illustration_from_The_Nursery_Alice_(1890)_-_c03757_07.jpg
3. We like to play tricks Source: https://www.flickr.com/photos/silly_little_man/5132242358/in/photostream/
4. So what is scale ❖ The size of an object in relationship to another object ❖ A way to measure a quantity ❖ To climb up a steep surface, such as a wall or the side of a mountain, often using special equipment
5. Types of Scaling
6. Our mental model is this
7. Time perspective of scale
8. But the world doesn’t work that way and we get that
9. But the world doesn’t work that way and we get that
10. Is there a better explanation for this?
11. Two important phenomenon Contention Coherence
12. Amdahl’s law N - Number of Parallel processes α - Coefficient of Contention i.e. % of path that is serial
13. Amdahl’s law and Contention
14. Law of diminishing returns
15. Speedup Table Number of Parallel Processes α = 0.01 ( 1%) α = 0.1 ( 10%) 1 1 1 5 4.80 3.63 10 9.17 5.5 20 16.8 7.7 40 28.77 10.5
16. Amdahl’s law Graph
17. But what causes degradation
18. Coherence
19. Coherence
20. Universal Scalability Law N - Number of Parallel processes α - Coefficient of Contention i.e. % of path that is serial β - Coefficient of Coherence
21. The USL Graph source: https://speakerdeck.com/drqz/applying-the-universal-scalability-law-to-distributed-systems?slide=49
22. USL: Sirius and Zookeeper source: https://speakerdeck.com/drqz/applying-the-universal-scalability-law-to-distributed-systems?slide=73
23. Different Shades of Coherence • Checkpoint • Safe point ( in garbage collected languages) • Read Locks ( Rate limiters anyone)
24. But it’s not just for computers Source: https://codahale.com/usl4j-and-you/ ttps://codahale.com/usl4j-and-you/
25. Interesting part of Prime directive "Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand." --Norm Kerth, Project Retrospectives: A Handbook for Team Review
26. OTSP Coherence = On the Same Page
27. Coherence can lead to Contention
28. So the solution is
29. This is classic MVCC Stuff Source: ANDERSON, J. C.; LENHARDT, J.; SLATER, N.: CouchDB: The definitive guide. O'Reilly Media Inc., 2010
30. Disruptor works on similar ideas Source: https://dzone.com/articles/dissecting-disruptor-writing
31. This all is fine but ... Everything before the "but" is meant to be ignored by the speaker; and everything after the "but" should be ignored by the listener. - Nassim Taleb
32. The classic case of Customer and Account Customer Account *
33. How Account Information Is Accessed Select account.id, account.balance, customer.name from account account, customer customer where account.customer_id = customer.id
34. Add real world complications Customer Account Create Customer Add KYC Information Add Demographic Information Add Credit check Information *
35. Access Pattern Ratio Customer Account * 1 10
36. How do you improve performance Customer Account * 1 10 Denormalization
37. Denormalization is a variant of replication
38. Data replication solves all your scalability problems
39. But it creates coherence problem Once you have replication you have to choose how to deal with multiple versions of data. But at least you can choose.
40. Once you have replication
41. The consistency wire is long and stretchable Strong Consistency Read My Writes Monotonic Reads Bounded Staleness Consistent Prefix Eventual Consistency
42. Consistency Guarantees and Performance Guarantee Consistency Performance Availability Strong Consistency Excellent Poor Poor Bounded Staleness Good Okay Poor Read My Writes Okay Okay Okay Monotonic Reads Okay Good Good Consistent Prefix Okay Good Excellent Eventual Consistency Poor Excellent Excellent
43. Consistency “A transaction is a correct transformation of the state. The actions taken as a group do not violate any of the integrity constraints associated with the state. This requires that the transaction be a correct program.” - Jim Gray
44. Correctness vs Freshness Correctness is about Invariants holding Freshness is about Time at which the data is created We often conflate the two aspects in the word “Consistency”
45. Decoupling systems in time
46. Consistency in real world is a mix bag Magenta - Strong Consistency Purple - Monotonic Reads Green - Consistent Prefix
47. Domain Driven Design Domain(n): An area of territory owned or controlled by a particular ruler or government In other words, who is responsible for enforcing the law In other words, who is responsible for ensuring invariants hold when data is generated. In other words it is about Conway’s law
48. DDD a different perspective Entity : The fundamental concept of an Entity is an abstract continuity threading through a life cycle and even passing through multiple forms. Entity creation is about enforcing invariants and holding them through the lifecycle of the entity. In other words controlling mutation. Value Object: As the name suggests it is an immutable object required for larger business invariants but the consumer is not bothered about enforcing the invariants.
49. Entity and Value Object
50. DDD Example: User Entitlements
51. So it’s all about USL For want of scalability, coherency is relaxed. For relaxed coherency, singularity is relaxed.
52. That’s All Folks for today.