From 1000 to 10k users per server - Concurrency rediscovered with Akka.

Concurrency is hard, asynchronous programming is hard. This is why we still design systems using thread-safe mentality. The consequences are harsh. We are blocking threads. Our services spend most of the time waiting for IO. We are wasting resources and generating unnecessary costs.

Not anymore! With introduction of Akka we get a very powerful toolkit which makes asynchronous programming much easier.

I am going to give you an introduction to Akka by showing how we've managed to improve our services at Zeebox. We increased the capacity from 1000 users per server to 10k users by using Akka and embracing the reactive programming.

The source code for this presentation is available at github:
https://github.com/piotrga/asyncer

Transcript

1. From 1000 to 10k users per server Concurrency rediscovered with Akka Piotr Gabryanczyk - @piotrga - peter@scala-experts.com
2. 2x PERFORMANCE! PARALLEL but SIMPLE
3. 10x THROUGHPUT ! with ASYNC IO 2x PERFORMANCE! PARALLEL but SIMPLE
4. 10x THROUGHPUT ! with ASYNC IO 2x PERFORMANCE! PARALLEL but SIMPLE ACTORS! PRACTICAL example
5. Zeebox friends Twitter
6. The blocking way
7. The blocking way
8. The blocking way
9. The blocking way
10. The blocking way
11. The blocking way
12. Exceptions
13. HTTP thread is BLOCKED HTTP! Thread blocking call
14. Duration Total: 6 sec 1000 threads => 166 req/sec
15. Common operations CPU instruction 1 nanosecond reading 1MB from RAM 250 000 nanoseconds TCP roundtrip US <-> EU 150 000 000 nanosecods Peter Norvig - norvig.org
16. 1 nano => 1 second Erik Meijer - coursera.org CPU instruction 1 second reading 1MB from RAM 3 days TCP roundtrip US <-> EU 5 years
17. Duration in human terms Total: 170 years, 6 months and 6 hours
18. Future Future
19. Future Future () onComplete( ) ()
20. Future Future () onComplete( ) () map( ) Future
21. Future
22. Future
23. Future
24. Future
25. Future
26. Let’s start parallelising
27. map
28. Continuation
29. HttpResponse
30. Async response
31. Nested futures
32. flatMap Future flatMap( ) Future Future
33. flatMap Future flatMap( ) Future Future Future{ fetchUser() } flatMap ( u => Future{ getAccountBalance(u.account) }) : Future[Double]
34. flatMap Future flatMap( ) Future Future Future{ fetchUser() } flatMap ( u => Future{ getAccountBalance(u.account) }) : Future[Double]
35. flatMap Future flatMap( ) Future Future Future{ fetchUser() } flatMap ( u => Future{ getAccountBalance(u.account) }) : Future[Double]
36. flatMap
37. flatMap
38. flatMap
39. flatMap
40. flatMap
41. More than one future
42. Nested future Future[Int]
43. flatMap
44. For comprehension
45. For comprehension
46. For comprehension
47. For comprehension
48. For comprehension
49. All in for comprehension
50. All in for comprehension
51. flatMap is sequential could be ! parallel could be ! parallel
52. Zip
53. Durations revisited 2 seconds 1 second 0.02 ms Total: 3 seconds was: 6 seconds
54. M! E! R! G! E Thread Thread Thread 500! threads 500! requests Twitter Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads fetchFriends fetchFriends added removed
55. M! E! R! G! E Thread Thread Thread 500! threads 500! requests Twitter Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads fetchFriends fetchFriends added removed
56. M! E! R! G! E Thread Thread Thread 500! threads 500! requests Twitter Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads fetchFriends fetchFriends added removed
57. M! E! R! G! E Thread Thread Thread 500! threads 500! requests Twitter Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads fetchFriends fetchFriends added removed
58. M! E! R! G! E Thread Thread Thread 500! threads 500! requests Twitter Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads Thread Thread Thread 500! threads fetchFriends fetchFriends added removed 500 req/3 sec =>166 req/sec
59. Sync IO Serialize request 3 hours De-serialize request 3 hours Wait for IO 55 years time ! human terms
60. Async IO Serialize request 3 hours De-serialize request 3 hours Handle 100 000! other requests! time ! human terms request async write and read response received!
61. Fully Async! 6 hours 3x6 hours = 18h 51x6 hours ! ~= 13 days
62. Fully Async! 6 hours Total: 14 days ~> 1.2 ms => 830 req/sec ! or 1660 req/sec on 2 cores was 170y, then 85y 166 req/seq 3x6 hours = 18h 51x6 hours ! ~= 13 days
63. Phone Actor System Actors Phone Connection Speaker Microphone Radio Keyboard
64. Actors are NOT threads! Phone Connection Actor Speaker Actor Microphone Actor Radio Actor Keyboard Actor Dispatcher Thread Thread Thread
65. Simple Actor
66. How to create an actor?
67. How to get response?
68. Async DB
69. Async DB
70. Connection Actor Async DB Connection Actor Connection Actor
71. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor
72. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor
73. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
74. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
75. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
76. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
77. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
78. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
79. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
80. Connection Actor Async DB Connection Actor Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
81. Connection Actor Async DB Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor
82. Connection Actor Async DB Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor Connection Actor
83. Connection Actor Async DB Connection Actor Facade! Actor Router Load! Balancer Supervisor Monitor! Actor Connection Actor
84. Command
85. Command
86. Connection Actor Connection Actor
87. Connection Actor Connection Actor
88. Connection Actor Connection Actor
89. Connection Actor Connection Actor
90. Facade Actor Facade! Actor
91. Facade Actor State Machine
92. Facade Actor State Machine
93. Facade Actor State Machine
94. Facade Actor State Machine
95. Facade Actor State Machine
96. Monitor Actor Inactive Active(k) Inactive for long time Facade! Actor ConnectionUp ConnectionDown ConnectionUp InactiveForLong ResourceDown ResourceUp ConnectionUp Connection Actor Connection Actor Connection Actor ConnectionUp ConnectionDown ConnectionUp ConnectionDown
97. Questions? Piotr Gabryanczyk! @piotrga! peter@scala-experts.com