Why Load & Performance
Testing the Cloud?
Sebastian Cohnen (@tisba)
Slide 2
Slide 2 text
• Focus on web performance and
architectures
• 6+ years consulting and
development
• Performance engineering & regular
execution of dynamic performance
analysis for customers
EHLO!
Sebastian Cohnen (@tisba)
CEO & CTO StormForger.com
Slide 3
Slide 3 text
StormForger.com
• SaaS Product & Consulting on Performance and
Load Testing
• We take care of the heavy lifting and pave the way
for regular performance assessments
• Free Preflight Tests while in Beta™
Slide 4
Slide 4 text
Load & Performance
Testing
Slide 5
Slide 5 text
–Wikipedia
“In software engineering, performance testing is
in general, a testing practice performed to
determine how a system performs in terms of
responsiveness and stability under a particular
workload.”
Slide 6
Slide 6 text
Performance Testing
• A category of non-functional testing methods
• Induce a well defined workload
• Observe the systems behavior
Slide 7
Slide 7 text
There are many forms…
• Load Testing
• Stress Testing
• Spike Testing
• Soak Testing
• Configuration Testing
Slide 8
Slide 8 text
Whatever you do…
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
Performance
Slide 11
Slide 11 text
≠
Scalability
Performance
Slide 12
Slide 12 text
Performance vs Scalability
Performance
"1 instance can handle 250 API requests per second
with p99 at 50ms…"
Scalability
"10 instances can handle 2500 API requests per
second with p99 at 50ms…"
Slide 13
Slide 13 text
You maybe want both!
• Performance is important, because it defines a
baseline for quality and capacity. It's also a
measurement on efficiency.
• Scalability is important, because it will tell you if
and how efficiently you can grow capacity.
Why should I care?
…about load and performance testing the cloud?
Slide 18
Slide 18 text
"The Cloud is Scalable!"*
…as long as I can afford it to be…
*famous last words…
Slide 19
Slide 19 text
Scaling Applications
Scaling Resources
Slide 20
Slide 20 text
Scaling Applications
Scaling Resources
≠
Slide 21
Slide 21 text
Understanding
Slide 22
Slide 22 text
You need to understand…
• Your (Distributed, μService, …) Application
• Your Software Architecture
• Cloud Environment
• Used Services and their behavior
Slide 23
Slide 23 text
Complexity has not
simply vanished!
Complexity has not
simply vanished!
Slide 24
Slide 24 text
Performance Testing
…and some more reasons for testing!
Slide 25
Slide 25 text
Configuration Testing
• Auto Scaling
• Throughput Provisioning (EBS, DynamoDB, …)
• Instance Types for EC2/RDS/EC/ES/…
• ELB -> Web Server -> App Server -> Backend
Services -> …
Slide 26
Slide 26 text
Performance Testing
• Determine speed, throughput, capacity, … of your
system
• Figure out the capacity per resource
Slide 27
Slide 27 text
Scalability Analysis
• How does my capacity increases with additional
resources?
• Requirement for Capacity Planning and Cost
Estimation
Slide 28
Slide 28 text
Load Testing
• What happens when I'm on hacker news? (Spike
Testing)
• How does your system and environment behave
under serious stress? (Stress Testing)
• What happens over longer period of times? (Soak
Testing)
Proof of Concept
• Design an Architecture on AWS
• Implement a Prototyp
• (Discuss it with AWS Solution Architects)
• Validate! Run Tests, observe, learn and improve!
Slide 31
Slide 31 text
Embrace the Cloud!
• Multiple, scalable test environments used to be
cumbersome and expensive
• Use automatization wherever you can:
• Services, Infrastructure, Servers, Code, Data…
Slide 32
Slide 32 text
One more thing…
Slide 33
Slide 33 text
Is this different than it
used to be with
"traditional hosting"?
Slide 34
Slide 34 text
Thanks! Questions?
Sebastian Cohnen (@tisba)
sebastian@stormforger.com