Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Performance & Stability testing \w Gatling

Performance & Stability testing \w Gatling

Dmitrijs Vrublevskis

April 09, 2015
Tweet

More Decks by Dmitrijs Vrublevskis

Other Decks in Programming

Transcript

  1. With Gatling
    Performance & Stability Testing

    View full-size slide

  2. Development Testing

    View full-size slide

  3. @Me
    Dmitry Vrublevsky,
    Software developer @
    /in/dmitryvrublevsky

    View full-size slide

  4. Roadmap
    - Mission overview
    - Gatling introduction
    - Some usage example

    View full-size slide

  5. Given: Graph database (Neo4j)
    Challenge: execute query as FAST as we can

    View full-size slide

  6. Neo4j
    - Written in Java
    - Native graph storage
    - Extendable

    View full-size slide

  7. So?
    - Default API is too slow
    - Implement custom extension
    - Control all the things

    View full-size slide

  8. 1. Developed
    2. Deployed
    3. ???
    4. Success!

    View full-size slide

  9. Specification
    Do “X” operations
    in “Y” seconds

    View full-size slide

  10. Specification
    Do “10000” query operations
    in “10” seconds

    View full-size slide

  11. POST /extension/query
    Body: “MATCH (root)-[:HAS*]->(child)”
    Response:
    {
    // JSON data
    }

    View full-size slide

  12. Client
    LB
    DB1 DB2 DB3
    - our extension

    View full-size slide

  13. Client
    LB
    DB1 DB2 DB3
    - our extension
    - possible problem

    View full-size slide

  14. Gatling
    http://gatling.io

    View full-size slide

  15. Load testing framework

    View full-size slide

  16. Free & Open source

    View full-size slide

  17. High performance

    View full-size slide

  18. Friendly DSL
    Nice html reports

    View full-size slide

  19. Bundle
    1. Download bundle
    2. Extract
    3. Ready
    http://gatling.io/#/download

    View full-size slide

  20. Optional
    - Maven integration
    - SBT plugin
    - Gradle plugin

    View full-size slide

  21. package com.neueda.example


    import io.gatling.core.Predef._

    import io.gatling.http.Predef._

    import scala.concurrent.duration._


    class Example extends Simulation {

    val httpConf = http.baseURL("http://www.example.com")


    val example = scenario("Example")

    .exec(

    http("get_example_index")

    .get("/")

    .check(status.is(200))

    )


    setUp(

    example.inject(rampUsers(10) over (10 seconds))

    ).protocols(httpConf)

    }

    View full-size slide

  22. package com.neueda.example


    import io.gatling.core.Predef._

    import io.gatling.http.Predef._

    import scala.concurrent.duration._


    View full-size slide

  23. class Example extends Simulation {


    }

    View full-size slide

  24. val httpConf =
    http.baseURL("http://www.example.com")

    View full-size slide

  25. val example = scenario("Example")

    View full-size slide

  26. http("get_example_index")

    .get("/")

    .check(status.is(200))

    View full-size slide

  27. setUp(

    ).protocols(httpConf)

    View full-size slide

  28. example.inject(
    rampUsers(10) over (10 seconds)
    )

    View full-size slide

  29. package com.neueda.example


    import io.gatling.core.Predef._

    import io.gatling.http.Predef._

    import scala.concurrent.duration._


    class Example extends Simulation {

    val httpConf = http.baseURL("http://www.example.com")


    val example = scenario("Example")

    .exec(

    http("get_example_index")

    .get("/")

    .check(status.is(200))

    )


    setUp(

    example.inject(rampUsers(10) over (10 seconds))

    ).protocols(httpConf)

    }

    View full-size slide

  30. http://gatling.io/docs/2.1.4/cheat-sheet.html

    View full-size slide

  31. ================================================================================
    2015-04-07 23:02:54 5s elapsed
    ---- Example -------------------------------------------------------------------
    [##################################### ] 50%
    waiting: 5 / running: 0 / done:5
    ---- Requests ------------------------------------------------------------------
    > Global (OK=5 KO=0 )
    > get_example_index (OK=5 KO=0 )
    ================================================================================
    ================================================================================
    ---- Global Information --------------------------------------------------------
    > request count 10 (OK=10 KO=0 )
    > min response time 255 (OK=255 KO=- )
    > max response time 302 (OK=302 KO=- )
    > mean response time 274 (OK=274 KO=- )
    > std deviation 13 (OK=13 KO=- )
    > response time 95th percentile 293 (OK=293 KO=- )
    > response time 99th percentile 300 (OK=300 KO=- )
    > mean requests/sec 1.08 (OK=1.08 KO=- )
    ---- Response Time Distribution ------------------------------------------------
    > t < 800 ms 10 (100%)
    > 800 ms < t < 1200 ms 0 ( 0%)
    > t > 1200 ms 0 ( 0%)
    > failed 0 ( 0%)
    ================================================================================

    View full-size slide

  32. ================================================================================
    2015-04-07 23:02:54 5s elapsed
    ---- Example -------------------------------------------------------------------
    [##################################### ] 50%
    waiting: 5 / running: 0 / done:5
    ---- Requests ------------------------------------------------------------------
    > Global (OK=5 KO=0 )
    > get_example_index (OK=5 KO=0 )
    ================================================================================
    ================================================================================
    ---- Global Information --------------------------------------------------------
    > request count 10 (OK=10 KO=0 )
    > min response time 255 (OK=255 KO=- )
    > max response time 302 (OK=302 KO=- )
    > mean response time 274 (OK=274 KO=- )
    > std deviation 13 (OK=13 KO=- )
    > response time 95th percentile 293 (OK=293 KO=- )
    > response time 99th percentile 300 (OK=300 KO=- )
    > mean requests/sec 1.08 (OK=1.08 KO=- )
    ---- Response Time Distribution ------------------------------------------------
    > t < 800 ms 10 (100%)
    > 800 ms < t < 1200 ms 0 ( 0%)
    > t > 1200 ms 0 ( 0%)
    > failed 0 ( 0%)
    ================================================================================

    View full-size slide

  33. class Test extends Simulation {

    val httpConf = http.baseURL("http://neo-database:7474")


    val test = scenario("GetGraph")

    .exec(

    http("execute_query")

    .post("/extension/query/execute")

    .body(StringBody("MATCH (root)-[:HAS*]->(child)"))

    .check(status.is(200))

    .check(jsonPath("$.results").count.is(100))

    )


    setUp(

    test.inject(

    rampUsersPerSec(0) to (1000) during (60 seconds)

    )

    ).protocols(httpConf)

    }

    View full-size slide

  34. http("execute_query")

    .post(
    "/extension/query/execute"
    )

    .body(StringBody(
    "MATCH (root)-[:HAS*]->(child)"
    ))

    .check(
    status.is(200)
    )

    .check(
    jsonPath("$.results").count.is(100)
    )
    1
    2
    3
    4

    View full-size slide

  35. rampUsersPerSec(0) to (1000) during (60 seconds)

    View full-size slide

  36. Requests / Second
    0
    150
    300
    450
    600
    Seconds
    0 10 20 30 40 50 60

    View full-size slide

  37. When request/second goes up
    And some threshold reached
    Then we receive Timeout error

    View full-size slide

  38. Everything is ok?
    Maybe we can do higher load?
    How to spot the problem?

    View full-size slide

  39. Drop all
    non-essential
    parts

    View full-size slide

  40. Client
    LB
    DB1 DB2 DB3
    - our extension

    View full-size slide

  41. 0
    250
    500
    750
    1000
    0 10 20 30 40 50 60

    View full-size slide

  42. Client - Server
    communication
    problems

    View full-size slide

  43. • Incorrect server setup
    • Unconfigured networking
    • Low max open connection limit

    View full-size slide

  44. Learned
    - Performance testing should be done
    - Test results should be interpreted
    properly
    - Test results should be verified in
    different environments

    View full-size slide

  45. We need to test
    different queries
    Feeders!

    View full-size slide

  46. val feeder = Array(

    Map("query" -> "..."),

    Map("query" -> "..."),

    Map("query" -> "...")

    )
    .queue
    .random
    .circular

    View full-size slide


  47. val test = scenario(“GetGraph")
    .feed(feeder)

    .exec(http("execute_query")

    .post("/extension/query/execute")

    .body(StringBody("${query}"))

    .check(status.is(200))

    .check(jsonPath("$.results").count.is(100))

    )

    View full-size slide

  48. CSV
    JSON
    JDBC
    Sitemap
    Redis
    Custom
    http://gatling.io/docs/2.1.4/session/feeder.html

    View full-size slide

  49. csv("data.csv").random

    View full-size slide

  50. Our own feeder
    - Custom feeder
    - Lazy loads data
    - Performant

    View full-size slide

  51. Dynamic user load?
    http://gatling.io/docs/2.1.4/general/simulation_setup.html

    View full-size slide

  52. test.inject(

    nothingFor(5 seconds),
    atOnceUsers(10),

    rampUsersPerSec(10) to (100) during (20 seconds),

    constantUsersPerSec(100) during (40 seconds),

    rampUsersPerSec(100) to (500) during (20 second),

    constantUsersPerSec(100) during (60 seconds)

    )

    View full-size slide

  53. Checks
    http://gatling.io/docs/2.1.4/http/http_check.html

    View full-size slide

  54. - status
    - currentLocation
    - header
    - regex
    - xpath
    - jsonPath

    View full-size slide

  55. regex("Invalid Page title").notExists

    status.is(200)

    jsonPath("$.posts").exists

    regex("""""").count.is(10)

    View full-size slide

  56. Realtime monitoring
    http://gatling.io/docs/2.1.4/realtime_monitoring/index.html
    Execution progress visual feedback

    View full-size slide

  57. Graphite (InfluxDB)
    +
    Grafana

    View full-size slide

  58. Recorder
    http://gatling.io/docs/2.1.4/http/recorder.html

    View full-size slide

  59. HTTP
    http://gatling.io/docs/2.1.4/http/recorder.html
    - HTTP Protocol
    - SSL
    - WebSocket
    - SSE

    View full-size slide

  60. Jenkins integration
    https://github.com/jenkinsci/gatling-plugin

    View full-size slide

  61. - Any IDE with Scala support is OK
    - Intellij IDEA
    - Eclipse (Scala IDE)
    - Netbeans

    View full-size slide

  62. Our use cases

    View full-size slide

  63. GET /api/node/1
    POST /api/node
    UPDATE /api/node
    Basic API tests

    View full-size slide

  64. - Generate background load
    - Make stability tests
    - Cluster communications
    - Load balancer setup
    - Spikes
    Load generator

    View full-size slide

  65. - Simulate significant write load
    - Check how database reacts
    Data import

    View full-size slide

  66. Easy
    Fun
    Performant

    View full-size slide