Reactive Supply To Changing Demand

Reactive applications need to be able to respond to demand, be elastic and ready to scale up, down, in and out—taking full advantage of mobile, multi-core and cloud computing architectures—in real time.

In this talk we will discuss the guiding principles making this possible through the use of share-nothing and non-blocking designs, applied all the way down the stack. We will learn how to deliver systems that provide reactive supply to changing demand.

I gave this talk at React Conf 2014 in London. Recording available here: https://www.youtube.com/watch?v=mBFdj7w4aFA

Transcript

1. Reactive Supply to
   Changing Demand
   Jonas Bonér
   Typesafe
   CTO & co-founder
   @jboner
   Building Elastic Reactive Systems
   

   View Slide

2. Outline
   1. Introduction
   2. Scale Up
   3. Scale Out
   4. Bring It Together
   2
   

   View Slide

3. Outline
   1. Introduction
   2. Scale Up
   3. Scale Out
   4. Bring It Together
   3
   

   View Slide

4. The rules of the game
   have changed
   

   View Slide

5. 5
   Yesterday Today
   

   View Slide

6. 5
   Yesterday Today
   Single machines
   

   View Slide

7. 5
   Yesterday Today
   Single machines Clusters of machines
   

   View Slide

8. 5
   Yesterday Today
   Single machines Clusters of machines
   Single core processors
   

   View Slide

9. 5
   Yesterday Today
   Single machines Clusters of machines
   Single core processors Multicore processors
   

   View Slide

10. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM
    

    View Slide

11. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    

    View Slide

12. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk
    

    View Slide

13. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    

    View Slide

14. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks
    

    View Slide

15. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    

    View Slide

16. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    Few concurrent users
    

    View Slide

17. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    Few concurrent users Lots of concurrent users
    

    View Slide

18. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    Few concurrent users Lots of concurrent users
    Small data sets
    

    View Slide

19. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    Few concurrent users Lots of concurrent users
    Small data sets Large data sets
    

    View Slide

20. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    Few concurrent users Lots of concurrent users
    Small data sets Large data sets
    Latency in seconds
    

    View Slide

21. 5
    Yesterday Today
    Single machines Clusters of machines
    Single core processors Multicore processors
    Expensive RAM Cheap RAM
    Expensive disk Cheap disk
    Slow networks Fast networks
    Few concurrent users Lots of concurrent users
    Small data sets Large data sets
    Latency in seconds Latency in milliseconds
    

    View Slide

22. View Slide

23. Cost Gravity is at Work
    7
    

    View Slide

24. Cost Gravity is at Work
    7
    

    View Slide

25. The Principles of Reactive Systems
    

    View Slide

26. The Principles of Reactive Systems
    

    View Slide

27. Elastic
    “Capable of ready change or easy expansion or contraction”
    - Merriam Webster
    

    View Slide

28. Scale on Demand?
    Why do we need to
    

    View Slide

29. scalability?
    what is
    

    View Slide

30. 12
    “Capable of being easily expanded or upgraded on demand”
    - Merriam Webster Dictionary
    

    View Slide

31. 13
    “The house in which Amdahl wakes up very early each day and
    rules with an iron fist.”
    - Martin Thompson (originally Gil Tene)
    

    View Slide

32. 13
    “The house in which Amdahl wakes up very early each day and
    rules with an iron fist.”
    - Martin Thompson (originally Gil Tene)
    

    View Slide

33. 14
    “A service is said to be scalable if when we increase the
    resources in a system, it results in increased performance
    in a manner proportional to resources added.”
    - Werner Vogels
    

    View Slide

34. vs
    Scalability
    Performance
    

    View Slide

35. Outline
    1. Introduction
    2. Scale Up
    3. Scale Out
    4. Bring It Together
    16
    

    View Slide

36. UP
    Scale
    

    View Slide

37. UP
    Scale
    and down
    

    View Slide

38. 18
    Modern CPU architecture
    

    View Slide

39. 18
    Modern CPU architecture
    

    View Slide

40. 18
    Modern CPU architecture
    

    View Slide

41. 18
    Modern CPU architecture
    

    View Slide

42. The CPU is gambling—taking bets
    19
    

    View Slide

43. Maximize
    Locality of Reference
    

    View Slide

44. Minimize
    Contention
    

    View Slide

45. Common points of
    Application
    Physical
    contention
    

    View Slide

46. 23
    Never
    ever
    

    View Slide

47. 23
    Never
    ever
    

    View Slide

48. Block
    23
    Never
    ever
    

    View Slide

49. 24
    GO
    

    View Slide

50. Async
    24
    GO
    

    View Slide

51. 25
    NOTHING
    share
    

    View Slide

52. Divide & Conquer
    26
    

    View Slide

53. 27
    Single Writer Principle
    

    View Slide

54. 27
    Single Writer Principle
    IO device
    Producers
    SERIAL &
    CONTENDED
    

    View Slide

55. 27
    Single Writer Principle
    IO device
    Producers
    SERIAL &
    CONTENDED
    IO device
    Producers Actor or Queue
    BATCHED &
    UNCONTENDED
    

    View Slide

56. View Slide

57. 29
    Needs to be async and non-blocking
    all the way down
    

    View Slide

58. 29
    Needs to be async and non-blocking
    all the way down
    

    View Slide

59. The Role of Immutable State
    30
    

    View Slide

60. The Role of Immutable State
    30
    

    View Slide

61. The Role of Immutable State
    • Great to represent facts
    • Messages and Events
    • Database snapshots
    • Representing the succession of time
    30
    

    View Slide

62. The Role of Immutable State
    • Great to represent facts
    • Messages and Events
    • Database snapshots
    • Representing the succession of time
    • Mutable State is ok if local and contained
    • Allows Single-threaded processing
    • Allows single writer principle
    • Feels more natural
    • Publish the results to the world as Immutable State
    30
    

    View Slide

63. on Demand
    Scale
    

    View Slide

64. Outline
    1. Introduction
    2. Scale Up
    3. Scale Out
    4. Bring It Together
    32
    

    View Slide

65. OUT
    Scale
    

    View Slide

66. OUT
    Scale
    and in
    

    View Slide

67. • Mobile
    • Cloud Services, REST etc.
    • NOSQL DBs
    • Big Data
    • etc
    34
    Distributed Computing is the
    new normal
    

    View Slide

68. What is the essence of
    distributed computing?
    35
    To try to overcome that
    

    View Slide

69. What is the essence of
    distributed computing?
    1. Information travels at the speed of light
    2. Independent things fail independently
    35
    To try to overcome that
    

    View Slide

70. View Slide

71. Node Node Node
    36
    Node
    

    View Slide

72. Middleware
    Node Node Node
    36
    Node
    

    View Slide

73. Cluster/Rack/Datacenter
    Cluster/Rack/Datacenter
    Cluster/Rack/Datacenter
    Middleware
    Node Node Node
    36
    Node
    

    View Slide

74. 37
    1. The network is reliable
    2. Latency is zero
    3. Bandwidth is infinite
    4. The network is secure
    5. Topology doesn't change
    6. There is one administrator
    7. Transport cost is zero
    8. The network is homogeneous
    Peter Deutsch’s
    8 Fallacies of
    Distributed
    Computing
    

    View Slide

75. The Graveyard of Distributed Systems
    • Distributed Shared Mutable State
    • 㱺 EVIL (where N is number of nodes)
    • Serializable Distributed Transactions
    • Synchronous RPC
    • Guaranteed Delivery
    • Distributed Objects
    • “Sucks like an inverted hurricane” - Martin Fowler
    38
    N
    

    View Slide

76. Maximize
    Locality of Reference
    

    View Slide

77. Strong
    Consistency
    

    View Slide

78. 41
    Linearizability
    “Under linearizable consistency, all operations appear to have
    executed atomically in an order that is consistent with the
    global real-time ordering of operations.”
    - Herlihy & Wing 1991
    

    View Slide

79. Strong Consistency Protocols
    

    View Slide

80. (Coordination in the Cluster)
    Minimize
    Contention
    

    View Slide

81. 44
    CAP
    Theorem
    

    View Slide

82. 44
    CAP
    Theorem
    

    View Slide

83. Consistency
    Eventual
    

    View Slide

84. 46
    CRDT
    CvRDTs/CmRDTs
    

    View Slide

85. 47
    “In general, application developers simply do not
    implement large scalable applications assuming
    distributed transactions.”
    -­‐  Pat Helland
    Life beyond Distributed Transactions:
    an Apostate’s Opinion
    

    View Slide

86. 48
    “State transitions are an important part of our problem
    space and should be modeled within our domain.”  
    - Greg Young
    Domain Events
    

    View Slide

87. 49
    "The database is a cache of a subset of the log.”
    - Pat Helland
    The Event Log
    

    View Slide

88. The Event Log
    • Append-Only Logging
    • Database of Facts
    • Two models:
    • One single Event Log
    • Strong Consistency
    • Multiple sharded Event Logs
    • Strong + Eventual Consistency
    50
    

    View Slide

89. Outline
    1. Introduction
    2. Scale Up
    3. Scale Out
    4. Bring It Together
    51
    

    View Slide

90. NOTHING
    52
    share
    

    View Slide

91. TRANSPARENCY
    53
    location
    

    View Slide

92. View Slide

93. View Slide

94. View Slide

95. Data
    Center
    55
    Data
    Center
    Cluster
    Cluster
    Machine
    Machine
    JVM
    JVM
    Node
    Node
    Thread
    Thread
    CPU
    CPU
    CPU
    Socket
    CPU
    Socket
    CPU
    Core
    CPU
    Core
    CPU
    L1/L2
    Cache
    CPU
    L1/L2
    Cache
    

    View Slide

96. 56
    Scaling Up and Out is essentially
    the same thing
    

    View Slide

97. 56
    Scaling Up and Out is essentially
    the same thing
    

    View Slide

98. View Slide

99. Elasticity requires a
    Message-Driven
    Architecture
    

    View Slide

100. Questions?
     

     View Slide

101. Reactive Supply to
     Changing Demand
     Jonas Bonér
     Typesafe
     CTO & co-founder
     @jboner
     Building Elastic Reactive Systems
     

     View Slide