Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Performance Testing From The Ground Up

Performance Testing From The Ground Up

(Talk given at https://devopsconference.de)

When systems are comprised of many distributed components, each with its own performance and reliability characteristics, and when a misconfiguration that happens to cause a cascading failure under load can be automatically deployed across environments all the way from dev to production in a matter of hours, you really need to make sure that a rigorous, well-understood, and easy-to-follow performance testing process is in place. This talk looks at what performance testing is and how it fits into a delivery process, and how an effective performance testing process can be implemented from the ground up with the open-source Artillery.io toolkit.

991f66ae71008743666ede9d1397fd9b?s=128

hassy veldstra

December 05, 2018
Tweet

Transcript

  1. Effective Performance Testing From The Ground Up When performance matters

    Hassy Veldstra h@artillery.io - @hveldstra DevOpsCon, Dec 2018, Munich
  2. Hello @hveldstra

  3. Open source backend/API testing toolkit Load testing & functional testing

    @hveldstra
  4. What this talk is not about @hveldstra

  5. What this talk is not about • Writing performant code

    (algorithms, optimization techniques) @hveldstra
  6. What this talk is not about • Writing performant code

    (algorithms, optimization techniques) • Profiling code to make it run faster, use less memory etc @hveldstra
  7. What this talk is not about • Writing performant code

    (algorithms, optimization techniques) • Profiling code to make it run faster, use less memory etc • Benchmarking code @hveldstra
  8. What this talk is about @hveldstra

  9. What this talk is about • Understanding where performance testing

    fits into delivery process @hveldstra
  10. What this talk is about • Understanding where performance testing

    fits into delivery process • The people/roles involved @hveldstra
  11. What this talk is about • Understanding where performance testing

    fits into delivery process • The people/roles involved • Understanding the various pieces that make up a performance testing strategy @hveldstra
  12. What this talk is about • Understanding where performance testing

    fits into delivery process • The people/roles involved • Understanding the various pieces that make up a performance testing strategy • Understanding of what an effective performance testing approach may look like @hveldstra
  13. Background @hveldstra

  14. @hveldstra

  15. @hveldstra

  16. @hveldstra

  17. @hveldstra

  18. @hveldstra

  19. Most importantly Performance is an key requirement @hveldstra

  20. Ecommerce @hveldstra

  21. IOT @hveldstra

  22. Events / ticketing @hveldstra

  23. • Chat • Games • Web APIs • Payments APIs

    @hveldstra
  24. How do we meet our performance goals? @hveldstra

  25. Questions, questions @hveldstra

  26. Questions, questions Where does performance testing fit into the development,

    testing, and delivery process? @hveldstra
  27. Questions, questions How do we run performance tests in our

    CI/CD pipeline? @hveldstra
  28. Questions, questions How do we define performance goals & SLOs*

    for our services? Can we have those checked automatically in CI? @hveldstra
  29. SLOs Service Level Objectives, e.g.: • Support up to 2000

    TPS • 99% of all requests should be served in under 200ms • No more than 0.1% of requests should return a 5×x response https://landing.google.com/sre/sre-book/chapters/service-level-objectives/ For a detailed discussion, see Google SRE book: @hveldstra
  30. Questions Questions What types of performance tests are there? Which

    ones do we need for our services? @hveldstra
  31. Questions Questions How do we organize our test suites? What

    are the best practices around structuring those tests? @hveldstra
  32. Questions Questions How do we pick which services to test?

    Do we need to test all of our microservices? What about testing different environments and configurations easily? @hveldstra
  33. Questions Questions How do we encourage collaboration on performance testing

    and involve everyone: developers, testers, SREs and product managers? (performance is everyone’s responsibility!) @hveldstra
  34. Questions Questions What’s the best way to get started with

    all of this? @hveldstra
  35. Maybe some answers @hveldstra

  36. Maybe some answers Understanding the place of performance testing in

    our delivery process @hveldstra
  37. Maybe some answers How to go from zero to a

    performance testing suite that’s: @hveldstra
  38. Maybe some answers How to go from zero to a

    performance testing suite that’s: • Extensible & maintainable @hveldstra
  39. Maybe some answers How to go from zero to a

    performance testing suite that’s: • Extensible & maintainable • Integrates into your CI/CD pipelines @hveldstra
  40. Maybe some answers How to go from zero to a

    performance testing suite that’s: • Extensible & maintainable • Integrates into your CI/CD pipelines • Helps verify SLOs automatically @hveldstra
  41. Maybe some answers How to go from zero to a

    performance testing suite that’s: • Extensible & maintainable • Integrates into your CI/CD pipelines • Helps verify SLOs automatically • Works for developers, testers and SREs @hveldstra
  42. Maybe some answers How to go from zero to a

    performance testing suite that’s: • Extensible & maintainable • Integrates into your CI/CD pipelines • Helps verify SLOs automatically • Works for developers, testers and SREs • Integrates with monitoring & reporting tools @hveldstra
  43. Maybe some answers @hveldstra • Based on projects & teams

    I’ve seen and questions typically raised • YMVV
  44. Definitions @hveldstra

  45. Definitions • Testing can suffer from lack of precision in

    terminology @hveldstra
  46. Definitions • Testing can suffer from lack of precision in

    terminology • What is “testing”? Several different dimensions to consider @hveldstra
  47. Type of thing under test @hveldstra

  48. When a test takes place https://medium.com/@copyconstruct/testing-microservices-the-sane- way-9bb31d158c16 @hveldstra

  49. https://medium.com/@copyconstruct/testing-in-production-the-safe-way-18ca102d0ef1 @hveldstra

  50. What the test is trying to prevent @hveldstra

  51. What is a test? @hveldstra

  52. What is a test? An activity, automated or manual, which:

    @hveldstra
  53. What is a test? An activity, automated or manual, which:

    1. Increases your confidence in some Thing @hveldstra
  54. What is a test? An activity, automated or manual, which:

    1. Increases your confidence in some Thing 2. Confirms that your understanding of a Thing or its behavior is still correct @hveldstra
  55. What is a test? An activity, automated or manual, which:

    1. Increases your confidence in some Thing 2. Confirms that your understanding of a Thing or its behavior is still correct 3. Increases your understanding of a Thing or its properties @hveldstra
  56. Examples @hveldstra

  57. Increase confidence • A/B tests • Canarying • Traffic replay

    • Load test to add traffic above base level @hveldstra
  58. Confirm understanding • Unit tests - known good/bad input, known

    good/bad output • Contract-based / property-based testing • Load test on a known configuration that verifies some metrics afterwards (max response time <500ms) @hveldstra
  59. Increase understanding • Sprint 100m and take a heart rate

    reading • Try opening 10k concurrent connections and see what happens • Exploratory testing of all kinds • Chaos testing @hveldstra
  60. Performance testing? any test that tests some performance-related property of

    a Thing. Often used interchangeably with “load testing”. @hveldstra
  61. Load testing @hveldstra

  62. @hveldstra

  63. Geofiltering • Give an IP address + country code •

    Returns yes/no @hveldstra
  64. Auth Token Service • Give username/password • Returns signed JWT

    token @hveldstra
  65. Or a combined API • Authenticate and get a token

    • Use the token to get geofiltering info @hveldstra
  66. What types of performance tests are there? @hveldstra

  67. What types of performance tests are there? • Load test

    in CI/CD to continuously verify SLOs (service or composite API) — pre-prod @hveldstra
  68. What types of performance tests are there? • Load test

    in CI/CD to continuously verify SLOs (service or composite API) — pre-prod • Load testing to help capacity planning — pre-prod, manual @hveldstra
  69. Capacity Planning • Identify amount of resources an instance of

    a service needs @hveldstra
  70. Capacity Planning • Identify amount of resources an instance of

    a service needs • Identify the number of instances needed to meet a performance target @hveldstra
  71. Capacity Planning • Identify amount of resources an instance of

    a service needs • Identify the number of instances needed to meet a performance target • Identify whether current capacity is sufficient or some limits will need to be raised pre-prod @hveldstra
  72. What types of performance tests are there? • Load test

    in CI/CD to continuously verify SLOs (service or composite API) — pre-prod • Load testing to help capacity planning — pre-prod, manual • Load test a service to better understand its scaling properties and tune configs — pre-prod, manual @hveldstra
  73. What types of performance tests are there? • Load test

    in CI/CD to continuously verify SLOs (service or composite API) — pre-prod • Load testing to help capacity planning — pre-prod, manual • Load test a service to better understand its scaling properties and tune configs — pre-prod, manual • Stress test a service to find its limits & understand how it degrades — pre-prod, manual @hveldstra
  74. What types of performance tests are there? • Load test

    in CI/CD to continuously verify SLOs (service or composite API) — pre-prod • Load testing to help capacity planning — pre-prod, manual • Load test a service to better understand its scaling properties and tune configs — pre-prod, manual • Stress test a service to find its limits & understand how it degrades — pre-prod, manual • In prod, manual or automatic — add extra traffic as a safety margin; a form of chaos testing @hveldstra
  75. What types of performance tests are there? • Soak test

    - a load test that runs for a longer period of time (1-2 hours)
  76. What types of performance tests are there? • Soak test

    - a load test that runs for a longer period of time (1-2 hours) • Spike test - a load test that ramps up load very quickly
  77. Acceptance/functional Testing @hveldstra

  78. Acceptance/functional Testing • Verify that a service (or another unit!)

    conforms to its contract @hveldstra
  79. Acceptance/functional Testing • Verify that a service (or another unit!)

    conforms to its contract • Verify that the results it produces make sense @hveldstra
  80. Auth service • Produces JWTs and not just empty 2×x

    responses • The token contains expected fields @hveldstra
  81. • It’s possible to re-use the same test code for

    both acceptance tests and load tests if your tooling supports it @hveldstra
  82. • It’s possible to re-use the same test code for

    both acceptance tests and load tests if your tooling supports it • (Artillery does!) @hveldstra
  83. Smoke tests @hveldstra

  84. Smoke tests • Main characteristic is their speed @hveldstra

  85. Smoke tests • Main characteristic is their speed • A

    test that determines whether other tests should even run @hveldstra
  86. Smoke tests • Main characteristic is their speed • A

    test that determines whether other tests should even run • Is anything obviously broken? If we plug this thing in and turn it on, is there any smoke? @hveldstra
  87. None
  88. Smoke tests • Main characteristic is their speed • A

    test that determines whether other tests should even run • Is anything obviously broken? If we plug this thing in and turn it on, is there any smoke? • Run a quick happy-path test case @hveldstra
  89. Where does performance testing fit into the delivery process? @hveldstra

  90. Where does performance testing fit into the delivery process? @hveldstra

  91. @hveldstra

  92. Where does Performance testing fit into the delivery process? @hveldstra

  93. Early “write code” stage: help profile code or dependencies The

    rest typically is on a deployed service or API @hveldstra
  94. How do we run these in CI/CD? @hveldstra

  95. How do we run these in CI/CD? • Create a

    parameterized CI job that can run a load test against a $service in an $environment with a $load_profile and (optionally) verify $slos. @hveldstra
  96. How do we run these in CI/CD? • Create a

    parameterized CI job that can run a load test against a $service in an $environment with a $load_profile and (optionally) verify $slos. • This can act as a stage in other pipelines @hveldstra
  97. How do we run these in CI/CD? • Create a

    parameterized CI job that can run a load test against a $service in an $environment with a $load_profile and (optionally) verify $slos. • This can act as a stage in other pipelines • … or be used manually by a dev/tester for ad-hoc testing via a web UI or a CLI (e.g. on Jenkins or AWS CodeBuild) @hveldstra
  98. How do we run these in CI/CD? @hveldstra

  99. How do we run these in CI/CD? • The tests

    don’t run on the CI server itself @hveldstra
  100. How do we run these in CI/CD? • The tests

    don’t run on the CI server itself • Best to be able to run them on your own (cloud) infrastructure @hveldstra
  101. How do we run these in CI/CD? • The tests

    don’t run on the CI server itself • Best to be able to run them on your own (cloud) infrastructure • Flexibility when it comes to VPCs or regions @hveldstra
  102. How do we run these in CI/CD? • The tests

    don’t run on the CI server itself • Best to be able to run them on your own (cloud) infrastructure • Flexibility when it comes to VPCs or regions • Critical for testing internal microservices @hveldstra
  103. How do we run these in CI/CD? • The tests

    don’t run on the CI server itself • Best to be able to run them on your own (cloud) infrastructure • Flexibility when it comes to VPCs or regions • Critical for testing internal microservices • More cost-effective too @hveldstra
  104. @hveldstra

  105. @hveldstra

  106. @hveldstra

  107. @hveldstra

  108. How do we run these in CI/CD? @hveldstra

  109. How do we run these in CI/CD? @hveldstra

  110. How do we run these in CI/CD? ❌ @hveldstra

  111. How do we run these in CI/CD? ✅ ❌ @hveldstra

  112. How do we run these in CI/CD? • What about

    test frequency? @hveldstra
  113. How do we run these in CI/CD? • What about

    test frequency? • No one-size-fits-all approach @hveldstra
  114. How do we run these in CI/CD? • What about

    test frequency? • No one-size-fits-all approach • Possible to test every change for microservices on the critical path, but probably excessive for most services @hveldstra
  115. How do we run these in CI/CD? • What about

    test frequency? • No one-size-fits-all approach • Possible to test every change for microservices on the critical path, but probably excessive for most services • Run on a schedule (e.g. nightly) @hveldstra
  116. How do we run these in CI/CD? • What about

    test frequency? • No one-size-fits-all approach • Possible to test every change for microservices on the critical path, but probably excessive for most services • Run on a schedule (e.g. nightly) • Run before a final promotion of a change to prod @hveldstra
  117. Defining SLOs @hveldstra

  118. Defining SLOs • Setting SLOs should be part of your

    team’s “creating a new microservice” checklist @hveldstra
  119. Defining SLOs • Setting SLOs should be part of your

    team’s “creating a new microservice” checklist • Use & adapt: https://github.com/SkeltonThatcher/run- book-template @hveldstra
  120. @hveldstra

  121. Defining SLOs • Setting SLOs should be part of your

    team’s “creating a new microservice” checklist • Documented alongside API specs, design docs (typically a Confluence/wiki page that follows a template) @hveldstra
  122. Defining SLOs • Setting SLOs should be part of your

    team’s “creating a new microservice” checklist • Documented alongside API specs, design docs (typically a Confluence/wiki page that follows a template) • Involve other teams that may rely on the service! @hveldstra
  123. Defining SLOs • Setting SLOs should be part of your

    team’s “creating a new microservice” checklist • Documented alongside API specs, design docs (typically a Confluence/wiki page that follows a template) • Involve other teams that may rely on the service! • Better to have some SLOs (& revise) than none at all @hveldstra
  124. Which services should we test? @hveldstra

  125. Which services should we test? • What could be tested?

    @hveldstra
  126. Which services should we test? • What could be tested?

    Anything with an API spec. @hveldstra
  127. Which services should we test? • What could be tested?

    Anything with an API spec. • Individual services @hveldstra
  128. Which services should we test? • What could be tested?

    Anything with an API spec. • Individual services • Composite APIs (that’s a “unit” with its own properties & behavior) @hveldstra
  129. Which services should we test? • What could be tested?

    Anything with an API spec. • Individual services • Composite APIs (that’s a “unit” with its own properties & behavior) • Does a microservice have SLOs? Then it should have a performance test to verify those automatically. @hveldstra
  130. How do we encourage collaboration? @hveldstra

  131. How do we encourage collaboration? @hveldstra

  132. How do we encourage collaboration? • No magical solutions @hveldstra

  133. How do we encourage collaboration? • No magical solutions •

    Involve all functions in discussions about performance @hveldstra
  134. How do we encourage collaboration? • No magical solutions •

    Involve all functions in discussions about performance • Remove barriers: @hveldstra
  135. How do we encourage collaboration? • No magical solutions •

    Involve all functions in discussions about performance • Remove barriers: • Use tools that everyone has access to • Use tools that everyone can work with @hveldstra
  136. How do we encourage collaboration? • Monorepos help @hveldstra

  137. How do we encourage collaboration? • Monorepos help • Good

    tooling helps @hveldstra
  138. How do we encourage collaboration? • Monorepos help • Good

    tooling helps • Available to everyone, ideally open source @hveldstra
  139. How do we encourage collaboration? • Monorepos help • Good

    tooling helps • Available to everyone, ideally open source • Easy to install and get started with @hveldstra
  140. How do we encourage collaboration? • Monorepos help • Good

    tooling helps • Available to everyone, ideally open source • Easy to install and get started with • Uses a language that everyone is familiar with @hveldstra
  141. How do we encourage collaboration? • Monorepos help • Good

    tooling helps • Available to everyone, ideally open source • Easy to install and get started with • Uses a language that everyone is familiar with • Make load testing reports & findings available to everyone (e.g. on Confluence/your wiki or KB) @hveldstra
  142. How do we get started? @hveldstra

  143. Artillery 101 @hveldstra

  144. Artillery 101 • Available on npm: npm install -g artillery

    @hveldstra
  145. Artillery 101 • Available on npm: npm install -g artillery

    • The artillery CLI is used to run tests and create HTML reports @hveldstra
  146. Artillery 101 • Available on npm: npm install -g artillery

    • The artillery CLI is used to run tests and create HTML reports • Tests are written in YAML and can be extended with Javascript @hveldstra
  147. Artillery 101 • Supports HTTP, Socketio, WebSocket, Kinesis, HLS out

    of the box. @hveldstra
  148. Artillery 101 • Supports HTTP, Socketio, WebSocket, Kinesis, HLS out

    of the box. • Third-party plugins for SQS, Lambda, SQL etc @hveldstra
  149. Artillery 101 • Supports HTTP, Socketio, WebSocket, Kinesis, HLS out

    of the box. • Third-party plugins for SQS, Lambda, SQL etc • Supports plugins. Out of the box: Statsd/Datadog/ Librato integration. @hveldstra
  150. Artillery 101 • Supports HTTP, Socketio, WebSocket, Kinesis, HLS out

    of the box. • Third-party plugins for SQS, Lambda, SQL etc • Supports plugins. Out of the box: Statsd/Datadog/ Librato integration. • Third-party plugins for other monitoring systems @hveldstra
  151. Artillery 101 • Designed to allow for complex, multi-step virtual

    user behavior to be scripted @hveldstra
  152. Artillery 101 • Designed to allow for complex, multi-step virtual

    user behavior to be scripted • Support for randomizing requests and capturing data from responses and re-using it in other requests @hveldstra
  153. Artillery 101 • Designed to allow for complex, multi-step virtual

    user behavior to be scripted • Support for randomizing requests and capturing data from responses and re-using it in other requests • Supports assertions on metrics such as HTTP latency — ie automated checking of SLOs @hveldstra
  154. Artillery 101 • Runs well in Docker, easy to run

    on ECS or Kubernetes or in a CI/CD pipeline @hveldstra
  155. Artillery 101 • Runs well in Docker, easy to run

    on ECS or Kubernetes or in a CI/CD pipeline • Can generate self-contained HTML reports with charts and graphs @hveldstra
  156. Artillery 101 • Runs well in Docker, easy to run

    on ECS or Kubernetes or in a CI/CD pipeline • Can generate self-contained HTML reports with charts and graphs • Features for creating modular test suites @hveldstra
  157. Config config: target: "" # we don't set a target

    by default environments: dev: target: "https://auth-service-dev.acme-corp.internal" defaults: headers: x-api-key: "0xcoffee" local: target: "http://localhost:8080" processor: “./functions.js" plugins: datadog: {} payload: - path: "./username-password.csv" fields: - username - password @hveldstra
  158. One or more scenarios scenarios: - name: Authenticate with valid

    credentials flow: - post: url: "/auth" json: username: "{{ username }}" password: "{{ password }}" expect: - statusCode: 200 - contentType: json @hveldstra
  159. $ artillery run \ --config config.yaml \ -e dev \

    scenario.yaml @hveldstra
  160. Organizing our test suite @hveldstra

  161. Organizing our test suite • Using a monorepo @hveldstra

  162. Organizing our test suite • Using a monorepo • Easier

    to get started with, extend & maintain @hveldstra
  163. Organizing our test suite • Using a monorepo • Easier

    to get started with, extend & maintain • Easier to share across teams @hveldstra
  164. Organizing our test suite • Using a monorepo • Easier

    to get started with, extend & maintain • Easier to share across teams • Helps code reuse @hveldstra
  165. Organizing our test suite • Using a monorepo • Easier

    to get started with, extend & maintain • Easier to share across teams • Helps code reuse • Easier to work with in CI/CD pipelines @hveldstra
  166. Organizing our test suite https://github.com/shoreditch-ops/acme-corp-api-tests @hveldstra

  167. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  168. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  169. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  170. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  171. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  172. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  173. Why that structure? @hveldstra

  174. Why that structure? • Extensible: @hveldstra

  175. Why that structure? • Extensible: • can add a new

    service or API easily, or add a new scenario to an existing one @hveldstra
  176. Why that structure? • Extensible: • can add a new

    service or API easily, or add a new scenario to an existing one • allows for service-specific config such as environment URLs or data from external CSVs @hveldstra
  177. Why that structure? • Extensible: • can add a new

    service or API easily, or add a new scenario to an existing one • allows for service-specific config such as environment URLs or data from external CSVs • allows for service-specific custom code, e.g. to generate random data in a certain format @hveldstra
  178. Why that structure? • Extensible: • can add a new

    service or API easily, or add a new scenario to an existing one • allows for service-specific config such as environment URLs or data from external CSVs • allows for service-specific custom code, e.g. to generate random data in a certain format • Can encode service-specific load phases and SLOs @hveldstra
  179. { "config": { "phases": [ { "duration": 120, "arrivalRate": 10,

    "rampTo": 20, "name": "Warm up the service" }, { "duration": 240, "arrivalRate": 20, "rampTo": 100, "name": "Ramp to high load" }, { "duration": 600, "arrivalRate": 100, "name": "Sustained high load" } ], "ensure": { "maxErrorRate": 0.1, "p99": 200 } } } @hveldstra
  180. Running a test artillery run \ —config ./services/auth-service/config.yaml \ --overrides

    "$(cat ./services/auth-service/ overrides.slos.json)” \ --e dev ./services/auth-service/login.yaml @hveldstra
  181. Running a test artillery run \ —config ./services/auth-service/config.yaml \ --overrides

    "$(cat ./services/auth-service/ overrides.slos.json)” \ --e dev ./services/auth-service/login.yaml @hveldstra
  182. Running a test artillery run \ —config ./services/auth-service/config.yaml \ --overrides

    "$(cat ./services/auth-service/ overrides.slos.json)” \ --e dev ./services/auth-service/login.yaml Service name, load/SLO override, environment and optionally a scenario → generic CI job @hveldstra
  183. Running a test • Reusable by other jobs / pipeline

    stages • Or via the UI for ad hoc testing - e.g. in Jenkins or AWS CodeBuild @hveldstra
  184. Organizing our test suite acme-corp-api-tests/ - services/ - auth-service/ -

    scenarios/ - config.yaml - functions.js - overrides.slo-response-time.json - package.json - common-config.yaml @hveldstra
  185. Where to start? @hveldstra

  186. Where to start? • Pick one service @hveldstra

  187. Where to start? • Pick one service • Write tests

    using the template & set up a CI job to run them @hveldstra
  188. Where to start? • Pick one service • Write tests

    using the template & set up a CI job to run them • Show & tell to the rest of the team @hveldstra
  189. Where to start? • A good candidate service: @hveldstra

  190. Where to start? • A good candidate service: • Small

    API surface @hveldstra
  191. Where to start? • A good candidate service: • Small

    API surface • Has experienced performance issues, or @hveldstra
  192. Where to start? • A good candidate service: • Small

    API surface • Has experienced performance issues, or • On the critical path for other components, or @hveldstra
  193. Where to start? • A good candidate service: • Small

    API surface • Has experienced performance issues, or • On the critical path for other components, or • Has high performance requirements @hveldstra
  194. Where to start? • A good candidate service: • Small

    API surface • Has experienced performance issues, or • On the critical path for other components, or • Has high performance requirements • For example: an authentication service @hveldstra
  195. So… we’ve looked at @hveldstra

  196. So… we’ve looked at • What performance testing is, and

    different types of performance tests @hveldstra
  197. So… we’ve looked at • What performance testing is, and

    different types of performance tests • Where performance testing fits into the development, testing, and delivery process @hveldstra
  198. So… we’ve looked at • What performance testing is, and

    different types of performance tests • Where performance testing fits into the development, testing, and delivery process • Running performance tests in CI/CD pipelines @hveldstra
  199. So… we’ve looked at • What performance testing is, and

    different types of performance tests • Where performance testing fits into the development, testing, and delivery process • Running performance tests in CI/CD pipelines • Setting and verifying SLOs @hveldstra
  200. So… we’ve looked at • What performance testing is, and

    different types of performance tests • Where performance testing fits into the development, testing, and delivery process • Running performance tests in CI/CD pipelines • Setting and verifying SLOs • The mechanics of setting up a test suite with Artillery @hveldstra
  201. email: h@artillery.io twitter: @hveldstra slides: https://speakerdeck.com/hassy/when-performance-matters- effective-performance-testing-from-the-ground-up Thanks!