Load Testing with Locust.IO - My Experience

This deck is intended to introduce an open source, python-based load testing tool / library, called 'Locust.io' and its usage.

Transcript

1. © 2010 VMware Inc. All rights reserved Confidential Locust.IO An open source load testing tool Mandar Raibagi @rmandar16 rmandar16@gmail.com
2. 2 Confidential Agenda  High-level overview  Features  Concept  Authors and Information  Demo  Closing remarks  Questions
3. 3 Confidential Locust.IO - High-level overview  OPEN SOURCE load testing tool  PYTHON BASED – plain old Python code  Simulates load from TRUE USER BEHAVIOR  ACTOR or USER PROFILE based testing  Swarms the system with MILLIONS OF SIMULTANEOUS USERS  Helps us BATTLE test  Get decisive PERFORMANCE / LOAD / SCALE METRICS  BENCHMARK API endpoints
4. 4 Confidential Locust.IO - Features  Designing USER BEHAVIOR using PYTHON code  DISTRIBUTED and SCALABLE  HACKABLE (~2500 LOC)  Intuitive WEB UI  Descent COMMAND LINE or - -NO-WEB support  Full support for API testing – extension of python-request  EXTENDABLE – can test any system
5. 5 Confidential Locust.IO - Concept  Wikipedia - Locusts are certain species of short-horned grasshoppers in the family Acrididae which possess both a solitary phase and a swarming phase.  Idea is that during a test, a swarm of locusts (test users) will ATTACK your website.  BEHAVIOR of each locust (test user) is defined by us  SWARMING process is monitored from a web UI in REAL-TIME.  completely EVENT-BASED – No complicated CALLBACKS  uses LIGHT-WEIGHT processes, through gevent  Each locust RUNNING INSIDE its OWN PROCESS (or greenlet)
6. 6 Confidential Locust.IO – Authors and Information http://locust.io/ http://docs.locust.io/en/latest/what-is-locust.html
7. 7 Confidential Locust.IO - Demo
8. 8 Confidential Locust.IO – Demo - Installation  Installation – normally requires Python 2.6+.  Recently started supporting Python 3.x: pip install locustio==0.8a1  For Locust distributed across multiple processes/machines, install ZeroMQ – pip install pyzmq
9. 9 Confidential Locust.IO – Demo - Example Code http://docs.locust.io/en/latest/writing-a-locustfile.html
10. 10 Confidential Locust.IO – Demo - Example Code http://docs.locust.io/en/latest/writing-a-locustfile.html
11. 11 Confidential Locust.IO – Demo – Main Classes or Building Blocks  Client() • Understands how to talk to the service you're testing • Responsible for reporting stats and errors • Only HTTP supported by default  Locust() or HttpLocust() • A simulated user  TaskSet() • The behavior of your locusts or users
12. 12 Confidential Locust.IO – Demo – Defining Tasks inside a TaskSet  Decorator @task(3) def tech(self): self.client.get('/tech')  Collection tasks = [world, opinion, tech] or tasks = {world: 2, opinion: 2, tech: 5}  Scheduling self.schedule_task(get_url, args=(url,))
13. 13 Confidential Locust.IO – Demo – Web UI  Web UI – SWARM Locust and MONITOR Statistics
14. 14 Confidential Locust.IO – Demo – Run Statistics  Web UI – DOWNLOAD DATA  Console statistics on killing the test (ctrl + c) –
15. 15 Confidential Locust.IO – Demo – Run Statistics  Requests Statistics Table -  Exceptions Table -  Response Time Distribution Table – Name # requests 50% 66% 75% 80% 90% 95% 98% 99% 100% PUT DUMMY_TEST_API_1 135 11000 12000 12000 12000 13000 13000 14000 14000 15335 PUT DUMMY_TEST_API_2 162 11000 12000 12000 13000 13000 13000 14000 14000 14250 GET DUMMY_TEST_API_3 485 3100 3400 3600 3700 4100 4400 4700 4800 5138 None Total 782 3800 8900 11000 11000 12000 13000 13000 14000 15335 Method Name # requests # failures Median response time Average response time Min response time Max response time Average Content Size Requests/s PUT DUMMY_TEST_API_1 135 0 11000 11112 7711 15335 3572 0.77 PUT DUMMY_TEST_API_2 162 0 11000 11076 6866 14250 3572 0.92 GET DUMMY_TEST_API_3 485 0 3100 3107 1068 5138 3572 2.77 None Total 782 0 3800 6140 1068 15335 3572 4.46 Count Message Traceback Nodes
16. 16 Confidential Locust.IO – Demo – Run Failures  Web UI – FAILURES
17. 17 Confidential Locust.IO – Demo – Command line Execution  Command line : - -NO-WEB option  locust --locustfile=C:\locust_io_tests\locust_io_test1.py --no-web --clients=10 --hatch-rate=1 --num-request=100 --logfile=C:\locust_io_tests\locust_io_test1.txt --print-stats  locust -f C:\locust_io_tests\locust_io_test1.py --no-web -c 10 -r 1 -n 100 --logfile=C:\locust_io_tests\locust_io_test1.txt --print-stats  locust -f C:\locust_io_tests\locust_io_test1.py --no-web -c 10 -r 1 -n 100 --logfile=C:\locust_io_tests\locust_io_test1.txt --print-stats --show-task-ratio
18. 18 Confidential Locust.IO – Demo – Command line Execution  Command line : - -NO-WEB --show-task-ratio
19. 19 Confidential Locust.IO - Closing Remarks  OPEN SOURCE – Obtain community help and contribute!  PYTHON – Quick learning, Simple installation and Coding!  Simulate TRUE USER BEHAVIOR and LOAD!  Simulate MILLIONS of SIMULTANEOUS USERS!  Build CUSTOM CLIENTS and test anything!  Use WEB UI / COMMAND LINE to RUN tests / MONITOR statistics!  Define success in RESPONSE TIMES and ERROR %  Can be used for FUNCTIONAL or SCALE / PERF testing
20. 20 Confidential Locust.IO – Questions / Feedback / Suggestions