Dan Crosta | @lazlofruvous Performance Testing & Profiling: A Virtuous Cycle Dan Crosta | Magnetic http://late.am/ | @lazlofruvous

Dan Crosta | @lazlofruvous Overview • Performance testing web apps • Profiling with the standard library • Instrumentation • The Virtuous Cycle

Dan Crosta | @lazlofruvous Performance Testing Basics • Generate requests against your app • Record and replay production • Measure response time & error rate

Dan Crosta | @lazlofruvous Types of Performance Testing • Stress Test • Load Test ! (I won’t talk about) • Spike Test • Soak Test

Dan Crosta | @lazlofruvous Stress Test • Generate excessive load • Lots of requests • Slow/difficult requests • Adversarial testing • “How much?” • Identify breaking point • Especially if you control synthetic load

Dan Crosta | @lazlofruvous Load Test • Generate specific, constant load • Expected conditions • Exaggerated conditions • “What If?” • Capacity planning

Dan Crosta | @lazlofruvous Best Practices • Isolate testing from external influences • Use dedicated load testing environment • “Scaled down” copies of all components

Dan Crosta | @lazlofruvous Best Practices • Generate load consistently • Random considered harmful • Automate, automate, automate

Dan Crosta | @lazlofruvous Profiling • Batteries included • cProfile, pstats • Documentation not really included • Goofy, horrible API • Avoid run(), runctx()

Dan Crosta | @lazlofruvous Profiling import cProfile ! profiler = cProfile.Profile()
 profiler.enable() ! # do stuff here ... ! profiler.disable() profiler.dump_stats("myprogram.prof")

Dan Crosta | @lazlofruvous Profiling import pstats ! stats = pstats.Stats("myprogram.prof") stats.sort_stats("calls") stats.print_stats() ! # prints a lot of stuff ...

Dan Crosta | @lazlofruvous Profiling Live Demo!

Dan Crosta | @lazlofruvous Profiling in Practice • “Why is it slow?” • Good for identifying un-optimized code • Tight loops, recursion, lots of function calls • These are candidates for optimization • Good for identifying bottlenecks • Distinguish between slow external resource and slow application code

Dan Crosta | @lazlofruvous Other Profiles • line_profiler • http://pythonhosted.org/line_profiler/ • yappi • https://code.google.com/p/yappi/

Dan Crosta | @lazlofruvous Instrumentation • Use statsd to collect time-series metrics • Lightweight, low-overhead, always-on profiling • Two key instruments: • Counters let you know how many things happened • Timers let you know how long they each took

Dan Crosta | @lazlofruvous Instrumentation from statsd import statsd ! @statsd.timed("login.response_time") def handle_login(username, password): if authenticated(username, password): statsd.increment("login.valid", 1) return redirect("/homepage") else: saatsd.increment("login.invalid", 1) return redirect("/login")

Dan Crosta | @lazlofruvous Instrumentation: Counter

Dan Crosta | @lazlofruvous Instrumentation: Timer

Dan Crosta | @lazlofruvous Instrumentation • Learn what’s normal for your app • Bonus: alert when things are not normal • “Does the real world match expectations?”

Dan Crosta | @lazlofruvous The Virtuous Cycle Instrumentation
 & Alerting Performance Testing Profiling Performance Optimization

Dan Crosta | @lazlofruvous Thanks! Dan Crosta | Magnetic http://late.am/ | @lazlofruvous