MongooseIM: Testing Massively Concurrent System

Distribution to multiple virtual machines in the cloud or servers is the way to scale horizontally when there is no more room or resources to grow vertically. In this talk we will discuss how to load test an XMPP server and distribute load generation using Erlang/OTP as a platform. Experiences from testing production ready MongooseIM systems will be presented.

Transcript

1. MONGOOSEIM Testing Massively Concurrent System ! ! Michał Ślaski @michalslaski
2. WEB OF THINGS video by Ericsson - The Social Web of Things
3. WEB OF THINGS video by Ericsson - The Social Web of Things
4. WEB OF THINGS video by Ericsson - The Social Web of Things
5. WEB OF THINGS video by Ericsson - The Social Web of Things
6. WEB OF THINGS video by Ericsson - The Social Web of Things
7. WEB OF THINGS video by Ericsson - The Social Web of Things
8. WEB OF THINGS video by Ericsson - The Social Web of Things
9. WEB OF THINGS video by Ericsson - The Social Web of Things
10. WEB OF THINGS video by Ericsson - The Social Web of Things
11. WEB OF THINGS video by Ericsson - The Social Web of Things
12. WEB OF THINGS video by Ericsson - The Social Web of Things
13. WEB OF THINGS
14. WEB OF THINGS • XMPP
15. WEB OF THINGS • XMPP • IP networks (HTTP, WebSockets)
16. WEB OF THINGS • XMPP • IP networks (HTTP, WebSockets) • wireless (WiFi, Beacon, NFC)
17. XMPP
18. XMPP • Open standard
19. XMPP • Open standard • Secure
20. XMPP • Open standard • Secure • Flexible
21. XMPP • Open standard • Secure • Flexible • Decentralised
22. XMPP • Open standard • Secure • Flexible • Decentralised • Proven
23. MONGOOSEIM
24. MONGOOSEIM • Instant messaging for Social Media, Gaming, etc.
25. MONGOOSEIM • Instant messaging for Social Media, Gaming, etc. • Solution designed for high volume
26. MONGOOSEIM • Instant messaging for Social Media, Gaming, etc. • Solution designed for high volume • Easily scalable distributed system
27. MONGOOSEIM • Instant messaging for Social Media, Gaming, etc. • Solution designed for high volume • Easily scalable distributed system • Meeting challenges of mobile
28. ESCALUS
29. ESCALUS • A library for acceptance testing XMPP servers
30. ESCALUS • A library for acceptance testing XMPP servers • Complicated scenarios
 - several users,
 - several resources per user,
 - setting up & tearing down the state.
31. ESCALUS
32. ESCALUS • Continuous integration
33. ESCALUS • Continuous integration • TDD
34. ESCALUS • Continuous integration • TDD • Load Testing ?
35. ESCALUS
36. ESCALUS % TEST CASE

37. ESCALUS % TEST CASE
 % Alice sends a chat message to Bob’s bare JID
38. ESCALUS % TEST CASE
 % Alice sends a chat message to Bob’s bare JID % Bob gets the message on both resources
39. ESCALUS % TEST CASE
 % Alice sends a chat message to Bob’s bare JID % Bob gets the message on both resources % Bob replies to one of Alice’s resources
40. ESCALUS % TEST CASE
 % Alice sends a chat message to Bob’s bare JID % Bob gets the message on both resources % Bob replies to one of Alice’s resources % Alice receives the reply only on that resource
41. courtesy of Krzysztof Goj
42. COMMON TEST
43. LOAD TESTING
44. LOAD TESTING • Need many escalus scenarios
45. LOAD TESTING $ ./run.sh my_scenario 1 100000
46. LOAD TESTING • Compile all the scenarios $ ./run.sh my_scenario 1 100000
47. LOAD TESTING • Compile all the scenarios • Start my_scenario $ ./run.sh my_scenario 1 100000
48. LOAD TESTING • Compile all the scenarios • Start my_scenario • Users with ids from 1 to 100000 $ ./run.sh my_scenario 1 100000
49. LOAD TESTING
50. LOAD TESTING http://www.danhudson.ca/winter-crows/
51. NEW PROJECT
52. NEW PROJECT • Time spent thinking up cool project name
 vs
53. NEW PROJECT • Time spent thinking up cool project name
 vs • Time spent writing code
54. NEW PROJECT • Time spent thinking up cool project name
 vs • Time spent writing code
55. A Murder Of Crows
56. AMOC SCENARIO
57. • Scenario = Erlang module AMOC SCENARIO
58. • Scenario = Erlang module • two exported functions: init/0 and start/1 AMOC SCENARIO
59. • Scenario = Erlang module • two exported functions: init/0 and start/1 AMOC SCENARIO -module(my_scenario). -export([start/1]). -export([init/0]).
60. INIT/0
61. • called once at the very beginning INIT/0
62. • called once at the very beginning • used to initialize state INIT/0
63. • called once at the very beginning • used to initialize state INIT/0 init() -> exometer:new(?MESSAGES_CT, spiral), exometer_report:subscribe( exometer_report_graphite, ?MESSAGES_CT, [one, count], 10000), ok.
64. • called once at the very beginning • used to initialize state • return state (coming soon™) INIT/0 init() -> exometer:new(?MESSAGES_CT, spiral), exometer_report:subscribe( exometer_report_graphite, ?MESSAGES_CT, [one, count], 10000), ok.
65. START/1
66. • describes the scenario for one user START/1
67. • describes the scenario for one user • argument is an unique integer START/1
68. • describes the scenario for one user • argument is an unique integer START/1 start(MyId) -> % connect user (using escalus) % fetch history % send some messages
 % wait
 % send some messages again
69. • describes the scenario for one user • argument is an unique integer • function is called START/1 start(MyId) -> % connect user (using escalus) % fetch history % send some messages
 % wait
 % send some messages again • again, and again, and again, and again, and again, and again, and again, ...
70. METRICS
71. METRICS
72. METRICS
73. TIME TO DELIVER
74. ERLANG THE MOVIE
75. ERLANG THE MOVIE
76. ERLANG THE MOVIE II
77. ERLANG THE MOVIE II
78. MONGOOSEIM
79. MONGOOSEIM • github.com/esl/MongooseIM
80. MONGOOSEIM • github.com/esl/MongooseIM • github.com/esl/escalus
81. MONGOOSEIM • github.com/esl/MongooseIM • github.com/esl/escalus • github.com/esl/amoc