Using Data Science to Support Performance Testing Results Analysis

Fast and smart test results analysis in performance testing.

Transcript

1. Using Data Science to Support Performance Testing Results Analysis by Viktoriia Kuznetcova
2. Rapid Performance Testing • Continuous Performance Testing • Either automated, or semi-automated • Fast reliable smart results analysis is crucial • Presentation/Reports for non-specialists (devs, PMs)
3. Complex use cases • Non-standard aggregations • Correlational analysis for multiple data sources • Complex Pass/Fail criteria • Difficult to extract data - e.g. from async queue-based distributed systems • Running analysis on the load generator to avoid transferring huge files across the network
4. Complex Use Cases (cont.) • Generating custom Performance Testing reports • Multidimensional benchmarking to assess changes in performance • Comparing test load and production load
5. R to the Rescue! • Programming language designed for data analysis • Created in University of Auckland • Open source everything • Super powerful • Neat presentation capabilities
6. Demo 0 - Simple aggregations • Use to aggregate data on a remote load generator • Saving on network traffic and time, or for security reasons • https://github.com/hali/Performance-data-processing/ blob/master/scripts/analyze.R
7. Demo 1 - Load Analysis • Analyse load in prod and in test, to compare • https://github.com/hali/Performance-data-processing/ blob/master/scripts/calculate_concurrency.R • https://github.com/hali/Performance-data-processing/ blob/master/scripts/access-log_concurrency.R • bonus: https://github.com/hali/Performance-data- processing/blob/master/scripts/concurrency_graph.R
8. Demo 2 - HAR analysis • Automatically highlight browser-side perf issues • https://github.com/hali/Performance-data-processing/ tree/master/RHARHotspots
9. Demo 3 - Pritcel report • Use as a starting point • Remove analysis for logs you don't have • Add analysis of your own logs • https://github.com/hali/Performance-data-processing/ tree/master/Pritcel
10. Demo 4 - Async queue- driven systems • Processing times are tricky • System 1 creates task, System 2 schedules it, all async • Created time and Scheduled time come as payload • Need to correlate JMeter request time and event times to get processing time Load generator API System 1 System 2 Notification Notification Notification
11. Contact Info • https://www.linkedin.com/in/hali4qa/ • https://testinglass.blogspot.com (has posts on some of the demoed scripts)