Building the Real Time Performance Panel

Transcript

1. Building the Real Time Performance Panel An Engineering Journey through Constraints and Portability
6. Highest Granularity: 10 seconds
7. Refresh Rate: 1 minute
8. A Better Experience
9. A Better Experience • New Tool • 10x Higher Granularity • Live Experience • Compromise: Ephemeral Data
10. A Better Experience • New Tool • 10x Higher Granularity • Live Experience • Compromise: Ephemeral Data
11. A Better Experience • New Tool • 10x Higher Granularity • Live Experience • Compromise: Ephemeral Data
12. A Better Experience • New Tool • 10x Higher Granularity • Live Experience • Compromise: Ephemeral Data
13. ACT I ARCHITECTURE
14. Understanding our Architecture Client
15. Understanding our Architecture Client
16. Understanding our Architecture L O A D B A L A N C E R Client
17. Understanding our Architecture L O A D B A L A N C E R Application Server Application Server Client
18. Understanding our Architecture L O A D B A L A N C E R Application Server Application Server DB Client
19. Understanding our Architecture L O A D B A L A N C E R Application Server Application Server DB DB DB Client
20. Understanding our Architecture L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network
21. Understanding our Architecture L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent
22. Understanding our Architecture L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent
23. Ideal Real Time Setup L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent
24. Ideal Real Time Setup L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent I have data!
25. Ideal Real Time Setup L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent I have data! Pub Sub
26. Ideal Real Time Setup L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent I have data! Pub Sub
27. Ideal Real Time Setup L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent Pub Sub I have data!
28. Ideal Real Time Setup L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent Pub Sub I have data!
29. HTTP vs. Websockets HTTP 1.1 Websockets Request/Response Request from Client Client can push, Server cannot Stateless Two way connection Initial Request from Client Server and Client can push Stateful
30. Pub Sub WebSockets with Load Balancer L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent
31. Pub Sub WebSockets with Load Balancer L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent
32. Pub Sub WebSockets with Load Balancer L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent
33. Pub Sub WebSockets with Load Balancer L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent
34. Pub Sub Ops Manager Process Location L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent
35. Pub Sub Ops Manager Process Location L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent L O A D B A L A N C E R
36. Pub Sub Ops Manager Process Location L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent L O A D B A L A N C E R
37. Solution: Polling L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent
38. Long Polling Server Client Agent
39. Long Polling Server Client Agent Got my data?
40. Long Polling Server Client Agent Got my data? It’s not here. Nap time!
41. Long Polling Server Client Agent Got my data? It’s not here. Nap time! Still not here. Nap time!
42. Long Polling Server Client Agent Got my data? Hey, here’s the data! It’s not here. Nap time! Still not here. Nap time!
43. Long Polling Server Client Agent Got my data? Hey, here’s the data! It’s not here. Nap time! Still not here. Nap time! Oh, yeah! Here it is!
44. Long Polling Server Client Agent Got my data? Hey, here’s the data! It’s not here. Nap time! Still not here. Nap time! Oh, yeah! Here it is!
45. Solution: Long Polling L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent
46. ACT II OPTIMIZATION
47. How do we make this... L O A D B A L A N C E R Application Server Application Server DB DB DB Client Customer Network Agent
48. Pub Sub ...behave like this? L O A D B A L A N C E R Application Server Application Server DB DB Client Customer Network Agent
49. How to make "Real Time" mean "Real Time" Respond ASAP
50. Respond ASAP Server Client Agent t = 0 t = 2 t = 1
51. Respond ASAP Server Client Agent t = 0 t = 2 t = 1
52. Source: https://www.reddit.com/r/gifs/comments/3qhlhr/ready_for_a_nap/
53. Respond ASAP Server Client Agent t = 0 t = 2 t = 1
54. Respond ASAP Server Client Agent t = 0 t = 2 t = 1
55. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000
56. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000
57. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020
58. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020
59. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020
60. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 tReceived = 1020
61. Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 tReceived = 1020
62. Last Point was at t = 1020 Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 tReceived = 1020
63. Last Point was at t = 1020 Add 1000ms for agent Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 tReceived = 1020
64. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 tReceived = 1020
65. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 tReceived = 1020
66. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… tReceived = 1020
67. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… 2120 -1600 520 tReceived = 1020
68. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… 2120 -1600 520 Sleep for 520ms tReceived = 1020
69. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… 2120 -1600 520 Sleep for 520ms tReceived = 1020
70. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… 2120 -1600 520 Sleep for 520ms tReceived = 1020
71. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… 2120 -1600 520 Sleep for 520ms Nailed it! tReceived = 1020
72. Last Point was at t = 1020 Add 1000ms for agent Eh, 100ms for db time sounds good Respond ASAP Server Client Agent t = 0 t = 3000 t = 2000 t = 1000 tReceived = 1020 1020 1000 + 100 2120 What time is it? Oh, 1600 so… 2120 -1600 520 Sleep for 520ms Nailed it! tReceived = 1020
73. How to make "Real Time" mean "Real Time" Respond ASAP Smooth Charts
75. Source: http://www.the-nextlevel.com/reviews/ps2/ddrmax/
76. Timing Our "Swing" • Ball = Data • Bat = Animation
77. TOO FAST
78. TOO SLOW
79. Solution?
80. Solution? • Rolling Average • Assume 1000ms • Track time • Average actual time
81. Solution? • Rolling Average • Assume 1000ms • Track time • Average actual time
82. Solution? • Rolling Average • Assume 1000ms • Track time • Average actual time
83. Solution? • Rolling Average • Assume 1000ms • Track time • Average actual time
85. Source: http://kotaku.com/american-takes-first-place-at-ddr-world-champsionship-1792258552
86. JUST RIGHT
87. How to make "Real Time" mean "Real Time" Respond ASAP Smooth Charts Final Jank Removal
89. Performance Problem
90. Performance Problem
91. Performance Problem
92. Performance Problem
93. Performance Problem
94. A Single Long Frame
95. A Single Long Frame
96. A Single Long Frame
97. A Single Long Frame
98. A Single Long Frame
99. Why?
100. Why?
101. Why? • Marionette • Remove all old views and add new child views • Backbone with Handlebars • Wholesale view replacement on every render
102. Why? • Marionette • Remove all old views and add new child views • Backbone with Handlebars • Wholesale view replacement on every render
103. Why? • Marionette • Remove all old views and add new child views • Backbone with Handlebars • Wholesale view replacement on every render
104. Why? • Marionette • Remove all old views and add new child views • Backbone with Handlebars • Wholesale view replacement on every render
105. Solution: Only Update What’s Changed!
106. Solution: Only Update What’s Changed! • React and the Virtual DOM!
107. Solution: Only Update What’s Changed!
108. Solution: Only Update What’s Changed! • D3 • We already have it as a dependency • Our team is already familiar with it • Very intense control over what changes
109. Solution: Only Update What’s Changed! • D3 • We already have it as a dependency • Our team is already familiar with it • Very intense control over what changes
110. Solution: Only Update What’s Changed! • D3 • We already have it as a dependency • Our team is already familiar with it • Very intense control over what changes
111. Solution: Only Update What’s Changed! • D3 • We already have it as a dependency • Our team is already familiar with it • Very intense control over what changes
112. How D3 Thinks About Things Source: https://bost.ocks.org/mike/join/
113. Three Steps
114. Three Steps 1. Insert new DOM elements 2. Remove unnecessary DOM elements 3. Set attributes on existing DOM elements
115. Three Steps 1. Insert new DOM elements 2. Remove unnecessary DOM elements 3. Set attributes on existing DOM elements
116. Three Steps 1. Insert new DOM elements 2. Remove unnecessary DOM elements 3. Set attributes on existing DOM elements
117. Fixed!
118. Fixed!
119. JUST RIGHT
120. Cloud Software Engineer, MongoDB JONATHAN BALSANO @jrbalsano