Continuous Load Testing & Performance Analysis

Talk about Continuous
 Load Testing & Performance Analysis at 20th November 2014 at WebPerfDays in Barcelona (http://www.webperfdays.org/events/2014-barcelona/)

Transcript

1. Continuous
 Load Testing & Performance Analysis
2. @larsvegas · lars@stormforger.com · https://stormforger.com EHLO, I’m Lars Lars Wolff - @larsvegas • Developer • Digital strategy, concept, Project management • Teams + Process + Ship it! • ex CTO • Agile Coach / Certified ScrumMaster® • Co-Founder and Product Manager
 @ StormForger.com
3. @larsvegas · lars@stormforger.com · https://stormforger.com What is load testing and performance analysis? Test methods for evaluation of external performance features and systems and software quality respectively. More detailed: Via load tests the system you want to test will be loaded with a defined amount of requests. The behavior of the system will thereby be monitored and rated.
4. @larsvegas · lars@stormforger.com · https://stormforger.com We know everything about our Apps! • Code/Application Quality
 (unit tests, code coverage, integration tests) • Server side monitoring
 (new relic, server density, munin, statsd, logs) • Client side tracking
 (Google Analytics, Piwik, Boomerang JS, Kissmetrics, Mixpanel, …)
5. @larsvegas · lars@stormforger.com · https://stormforger.com Actually we don’t know the limits of our application ecosystem! (hopefully we expect them) • What is the timing of request and response?
 → Performance • How much concurrent requests can my App stand?
 → Capacity • Are two app servers twice as good as one app server?
 → capacity + scalability • Will the performance be the same under continuous load?
 → quality
6. @larsvegas · lars@stormforger.com · https://stormforger.com Why should I know the limits of my App ecosystem? • Quality of Service • Improve things (Capacity planning) • $$$ • (there’re even more reasons…)
7. @larsvegas · lars@stormforger.com · https://stormforger.com How to get to know the limits of my App?
8. @larsvegas · lars@stormforger.com · https://stormforger.com Let’s load test! :-)
9. @larsvegas · lars@stormforger.com · https://stormforger.com Sad but true: Load Testing and Performance analysis is complex!
 You have to:* • Testplaning (Goals, Plans, Testdata and Fixtures, …) • Ressource provisioning • Setup test environment • Deployment of test configuration and assets • run test • monitor test target • monitor load generators and test flow • Cluster Teardown • Fetch logs and data • crunch data • filter data • generate stats • create data representations (graphs, tables, …) • analyse • asses, sort and compare results *without claiming to be exhaustive
10. @larsvegas · lars@stormforger.com · https://stormforger.com That’s a lot of work. • 2-5 days • or even more days if your application is more complex
11. @larsvegas · lars@stormforger.com · https://stormforger.com A lot of work per release cycle! Since we are developing in short release cycles our apps performance changes pretty much at every release cycle
12. @larsvegas · lars@stormforger.com · https://stormforger.com Two much work, to do it per release cycle! spending all the time on solving this complex thing is ONE reason we don’t do it
13. @larsvegas · lars@stormforger.com · https://stormforger.com Since, we don’t do it, we take some obvious risks • things break and the app fails
 → Ops Pager call • Slow performance
 → less conversion, no/less bids in RTB, … • Promotion / Slashdotted
 → loss of potential users
14. @larsvegas · lars@stormforger.com · https://stormforger.com Conclusion • We worked hard to assure high quality in
 functional requirements: • code quality, build chains, monitoring server side, monitoring client side • Why do we stop now at points of
 non functional requirements? • request/response performance, request/response quality, architecture capacity, architecture scalability
15. @larsvegas · lars@stormforger.com · https://stormforger.com So, what’s about a world with continuous, fully automated load tests and performance analysis? whats about the idea having tools in our development process to get an understanding of the limits of our systems?
16. @larsvegas · lars@stormforger.com · https://stormforger.com @tisba and I think, that sounds great! :) • we are on a mission to bring this vision in an easy and fairly manner to everyone • We’ll empower you to create, run and analyse your load and performance tests - by click/API call!!!11
17. @larsvegas · lars@stormforger.com
 https://stormforger.com Thank you! Questions? Join the Open Space Session „Continuous Load and Performance Testing“!