Performance & Stability testing \w Gatling

Transcript

1. With Gatling Performance & Stability Testing
2. Development Testing
4. @Me Dmitry Vrublevsky, Software developer @ /in/dmitryvrublevsky
5. Roadmap - Mission overview - Gatling introduction - Some usage example
6. Mission
7. Given: Graph database (Neo4j) Challenge: execute query as FAST as we can
8. Neo4j - Written in Java - Native graph storage - Extendable
10. So? - Default API is too slow - Implement custom extension - Control all the things
11. 1. Developed 2. Deployed 3. ??? 4. Success!
13. Performance
14. Specification Do “X” operations in “Y” seconds
15. Specification Do “10000” query operations in “10” seconds
16. POST /extension/query Body: “MATCH (root)-[:HAS*]->(child)” Response: { // JSON data }
17. Client LB DB1 DB2 DB3 - our extension
18. Problems?
19. Client LB DB1 DB2 DB3 - our extension - possible problem
20. Gatling http://gatling.io
21. Load testing framework
22. Free & Open source
23. High performance
25. Friendly DSL Nice html reports
26. Coding ?
27. Bundle 1. Download bundle 2. Extract 3. Ready http://gatling.io/#/download
28. Optional - Maven integration - SBT plugin - Gradle plugin
29. package com.neueda.example
 
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import scala.concurrent.duration._
 
 class Example extends Simulation {
 val httpConf = http.baseURL("http://www.example.com")
 
 val example = scenario("Example")
 .exec(
 http("get_example_index")
 .get("/")
 .check(status.is(200))
 )
 
 setUp(
 example.inject(rampUsers(10) over (10 seconds))
 ).protocols(httpConf)
 }
30. package com.neueda.example
 
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import scala.concurrent.duration._

31. class Example extends Simulation {
 
 }
32. val httpConf = http.baseURL("http://www.example.com")
33. val example = scenario("Example")
34. http("get_example_index")
 .get("/")
 .check(status.is(200))
35. setUp(
 ).protocols(httpConf)
36. example.inject( rampUsers(10) over (10 seconds) )
37. package com.neueda.example
 
 import io.gatling.core.Predef._
 import io.gatling.http.Predef._
 import scala.concurrent.duration._
 
 class Example extends Simulation {
 val httpConf = http.baseURL("http://www.example.com")
 
 val example = scenario("Example")
 .exec(
 http("get_example_index")
 .get("/")
 .check(status.is(200))
 )
 
 setUp(
 example.inject(rampUsers(10) over (10 seconds))
 ).protocols(httpConf)
 }
38. http://gatling.io/docs/2.1.4/cheat-sheet.html
39. ================================================================================ 2015-04-07 23:02:54 5s elapsed ---- Example ------------------------------------------------------------------- [##################################### ] 50% waiting: 5 / running: 0 / done:5 ---- Requests ------------------------------------------------------------------ > Global (OK=5 KO=0 ) > get_example_index (OK=5 KO=0 ) ================================================================================ ================================================================================ ---- Global Information -------------------------------------------------------- > request count 10 (OK=10 KO=0 ) > min response time 255 (OK=255 KO=- ) > max response time 302 (OK=302 KO=- ) > mean response time 274 (OK=274 KO=- ) > std deviation 13 (OK=13 KO=- ) > response time 95th percentile 293 (OK=293 KO=- ) > response time 99th percentile 300 (OK=300 KO=- ) > mean requests/sec 1.08 (OK=1.08 KO=- ) ---- Response Time Distribution ------------------------------------------------ > t < 800 ms 10 (100%) > 800 ms < t < 1200 ms 0 ( 0%) > t > 1200 ms 0 ( 0%) > failed 0 ( 0%) ================================================================================
40. ================================================================================ 2015-04-07 23:02:54 5s elapsed ---- Example ------------------------------------------------------------------- [##################################### ] 50% waiting: 5 / running: 0 / done:5 ---- Requests ------------------------------------------------------------------ > Global (OK=5 KO=0 ) > get_example_index (OK=5 KO=0 ) ================================================================================ ================================================================================ ---- Global Information -------------------------------------------------------- > request count 10 (OK=10 KO=0 ) > min response time 255 (OK=255 KO=- ) > max response time 302 (OK=302 KO=- ) > mean response time 274 (OK=274 KO=- ) > std deviation 13 (OK=13 KO=- ) > response time 95th percentile 293 (OK=293 KO=- ) > response time 99th percentile 300 (OK=300 KO=- ) > mean requests/sec 1.08 (OK=1.08 KO=- ) ---- Response Time Distribution ------------------------------------------------ > t < 800 ms 10 (100%) > 800 ms < t < 1200 ms 0 ( 0%) > t > 1200 ms 0 ( 0%) > failed 0 ( 0%) ================================================================================
41. Reports \o/
46. class Test extends Simulation {
 val httpConf = http.baseURL("http://neo-database:7474")
 
 val test = scenario("GetGraph")
 .exec(
 http("execute_query")
 .post("/extension/query/execute")
 .body(StringBody("MATCH (root)-[:HAS*]->(child)"))
 .check(status.is(200))
 .check(jsonPath("$.results").count.is(100))
 )
 
 setUp(
 test.inject(
 rampUsersPerSec(0) to (1000) during (60 seconds)
 )
 ).protocols(httpConf)
 }
47. http("execute_query")
 .post( "/extension/query/execute" )
 .body(StringBody( "MATCH (root)-[:HAS*]->(child)" ))
 .check( status.is(200) )
 .check( jsonPath("$.results").count.is(100) ) 1 2 3 4
48. rampUsersPerSec(0) to (1000) during (60 seconds)
49. Requests / Second 0 150 300 450 600 Seconds 0 10 20 30 40 50 60
51. When request/second goes up And some threshold reached Then we receive Timeout error
52. Everything is ok? Maybe we can do higher load? How to spot the problem?
53. Drop all non-essential parts
54. Client LB DB1 DB2 DB3 - our extension
55. 0 250 500 750 1000 0 10 20 30 40 50 60
57. Client - Server communication problems
58. • Incorrect server setup • Unconfigured networking • Low max open connection limit
59. Learned - Performance testing should be done - Test results should be interpreted properly - Test results should be verified in different environments
60. We need to test different queries Feeders!
61. val feeder = Array(
 Map("query" -> "..."),
 Map("query" -> "..."),
 Map("query" -> "...")
 ) .queue .random .circular
62. 
 val test = scenario(“GetGraph") .feed(feeder)
 .exec(http("execute_query")
 .post("/extension/query/execute")
 .body(StringBody("${query}"))
 .check(status.is(200))
 .check(jsonPath("$.results").count.is(100))
 )
63. CSV JSON JDBC Sitemap Redis Custom http://gatling.io/docs/2.1.4/session/feeder.html
64. csv("data.csv").random
65. Our own feeder - Custom feeder - Lazy loads data - Performant
66. Dynamic user load? http://gatling.io/docs/2.1.4/general/simulation_setup.html
67. test.inject(
 nothingFor(5 seconds), atOnceUsers(10),
 rampUsersPerSec(10) to (100) during (20 seconds),
 constantUsersPerSec(100) during (40 seconds),
 rampUsersPerSec(100) to (500) during (20 second),
 constantUsersPerSec(100) during (60 seconds)
 )
68. Checks http://gatling.io/docs/2.1.4/http/http_check.html
69. - status - currentLocation - header - regex - xpath - jsonPath
70. regex("Invalid Page title").notExists
 status.is(200)
 jsonPath("$.posts").exists
 regex("""<div class="article">""").count.is(10)
71. Realtime monitoring http://gatling.io/docs/2.1.4/realtime_monitoring/index.html Execution progress visual feedback
72. Graphite (InfluxDB) + Grafana
74. Recorder http://gatling.io/docs/2.1.4/http/recorder.html
76. HTTP http://gatling.io/docs/2.1.4/http/recorder.html - HTTP Protocol - SSL - WebSocket - SSE
77. Jenkins integration https://github.com/jenkinsci/gatling-plugin
78. - Any IDE with Scala support is OK - Intellij IDEA - Eclipse (Scala IDE) - Netbeans
79. Our use cases
80. GET /api/node/1 POST /api/node UPDATE /api/node Basic API tests
81. - Generate background load - Make stability tests - Cluster communications - Load balancer setup - Spikes Load generator
82. - Simulate significant write load - Check how database reacts Data import
83. Easy Fun Performant