Latency Measurement and Results Analysis

EXTENT TALKS: Tbilisi
20.04.2019

Arseniy Marinin, NFT Analyst, Exactpro
Alexander Tregubov, Senior NFT Analyst, Exactpro

Website https://extent.exactpro.com/

Linkedin https://www.linkedin.com/company/exactpro-systems-llc
Instagram https://www.instagram.com/qatbilisi/
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/qatbilisi/
Youtube Channel https://www.youtube.com/c/exactprosystems

Transcript

1. 20 April 2019, Tbilisi, Georgia
   Latency Measurement and
   Results Analysis
   Arseniy Marinin, NFT Analyst, Exactpro
   Alexander Tregubov, Senior NFT Analyst, Exactpro
   

   View Slide

2. 20 April 2019 Tbilisi, Georgia
   Low latency and big data
   TESTING is a process of researching software with the intent of finding bugs.
   RESEARCH comes with data analysis.
   In case of performance testing - HUGE amount of data.
   2
   

   View Slide

3. 20 April 2019 Tbilisi, Georgia
   Gathering the right data
   What do we really want to measure?
   How do we want to measure?
   3
   

   View Slide

4. 20 April 2019 Tbilisi, Georgia
   Gathering the right data
   4
   

   View Slide

5. 20 April 2019 Tbilisi, Georgia
   How do we want to measure?
   B
   A
   5
   

   View Slide

6. 20 April 2019 Tbilisi, Georgia
   Measurement process
   User behavior
   Load composition
   Load configuration: ramp-up vs. constant
   6
   

   View Slide

7. 20 April 2019 Tbilisi, Georgia
   Gathering raw data
   7
   

   View Slide

8. 20 April 2019 Tbilisi, Georgia
   Gathering raw data
   8
   

   View Slide

9. 20 April 2019 Tbilisi, Georgia
   Aggregation
   Distribution
   Detalization and anomalies
   9
   

   View Slide

10. 20 April 2019 Tbilisi, Georgia 10
    

    View Slide

11. 20 April 2019 Tbilisi, Georgia 11
    

    View Slide

12. 20 April 2019 Tbilisi, Georgia
    Average color of image
    #9d9ca0
    12
    

    View Slide

13. 20 April 2019 Tbilisi, Georgia
    Aggregation
    Distribution
    Detalization and anomalies
    13
    

    View Slide

14. 20 April 2019 Tbilisi, Georgia
    Distribution
    14
    

    View Slide

15. 20 April 2019 Tbilisi, Georgia 15
    

    View Slide

16. 20 April 2019 Tbilisi, Georgia 16
    

    View Slide

17. 20 April 2019 Tbilisi, Georgia
    Histogram
    17
    

    View Slide

18. 20 April 2019 Tbilisi, Georgia
    Distribution of data
    18
    

    View Slide

19. 20 April 2019 Tbilisi, Georgia 19
    

    View Slide

20. 20 April 2019 Tbilisi, Georgia
    What about latency stats distribution?
    20
    

    View Slide

21. 20 April 2019 Tbilisi, Georgia
    How real distribution looks like?
    21
    

    View Slide

22. 20 April 2019 Tbilisi, Georgia
    How we should compare 2 releases?
    22
    

    View Slide

23. 20 April 2019 Tbilisi, Georgia
    How we should compare 2 releases?
    23
    

    View Slide

24. 20 April 2019 Tbilisi, Georgia
    Aggregation
    Distribution
    Detalization and anomalies
    24
    

    View Slide

25. 20 April 2019 Tbilisi, Georgia
    John Snow (281 AC - 302 AC, 302 AC - …)
    25
    

    View Slide

26. 20 April 2019 Tbilisi, Georgia
    John Snow (1813 - 1858)
    One of the founders of modern
    epidemiology.
    Cholera outbreak in Soho,
    London, in 1854.
    26
    

    View Slide

27. 20 April 2019 Tbilisi, Georgia
    London Cholera Epidemic of 1854
    27
    

    View Slide

28. 20 April 2019 Tbilisi, Georgia 28
    

    View Slide

29. 20 April 2019 Tbilisi, Georgia 29
    

    View Slide

30. 20 April 2019 Tbilisi, Georgia 30
    

    View Slide

31. 20 April 2019 Tbilisi, Georgia
    Going into details
    31
    

    View Slide

32. 20 April 2019 Tbilisi, Georgia
    Looking at the extremes and anomalies
    32
    

    View Slide

33. 20 April 2019 Tbilisi, Georgia
    The Fastest vs. The Slowest
    169 requests from 47 domains
    4 MB transferred
    Pegs CPU for 4 seconds
    Rendering doesn’t begin for 20 seconds
    33
    

    View Slide

34. 20 April 2019 Tbilisi, Georgia
    The Fastest vs. The Slowest
    34
    

    View Slide

35. 20 April 2019 Tbilisi, Georgia
    Percentile
    35
    

    View Slide

36. 20 April 2019 Tbilisi, Georgia
    Median
    36
    

    View Slide

37. 20 April 2019 Tbilisi, Georgia
    Max 99% percentile graph
    37
    

    View Slide

38. 20 April 2019 Tbilisi, Georgia
    Conclusion
    Keep looking at your data
    Investigate anomalies
    Go into details
    38
    Inspired by:
    Velocity 2011: John Rauser, "Look at Your Data"
    Velocity 2012: John Rauser, "Investigating Anomalies"
    

    View Slide

39. 20 April 2019 Tbilisi, Georgia
    Thanks for your attention!
    39
    

    View Slide