Real World Problems, Real World Load Testing

Slide 1

Slide 1 text

Real World Problems, Real World Load Tes1ng Jim Holmes Evangelist, Test Studio @aJimHolmes h7p://bit.ly/JimAtTestStudio [email protected]

Slide 2

Slide 2 text

Slide 3

Slide 3 text

What the CEO/customer wants: “Can our site survive as we grow?” “Will our users be happy when the system’s being used?”

Slide 4

Slide 4 text

The cost of degraded performance “According to a 2008 Aberdeen Group study, a 1-‐second delay in page load Ome equals 11 percent fewer page views, a 16 percent decrease in customer saOsfacOon, and a 7 percent loss in conversions. In dollar terms, this means that if a site typically earns $100,000 a day, this year you could lose $2.5 million in sales for every second slower your site is compared to last year (or, for that ma7er, compared to your compeOtor’s website).” -‐-‐ Web Load Tes+ng for Dummies

Slide 5

Slide 5 text

What’s on the menu •  Types of Load TesOng •  SeZng ExpectaOons •  What •  Where •  How •  Demo!

Slide 6

Slide 6 text

Types of Load Tes1ng

Slide 7

Slide 7 text

Soak Tes1ng

Slide 8

Slide 8 text

Failover / Tip Tes1ng

Slide 9

Slide 9 text

Load Tes1ng

Slide 10

Slide 10 text

Set Expecta1ons Early •  What are our realisOc goals? •  “Do we really need to support China when we will only serve our neighborhood?” -‐-‐ @BenCarey •  Need People, Hardware, TIME •  What’s on the table for ﬁxing?

Slide 11

Slide 11 text

WHERE do we test? •  Internal •  Lets us focus on system behavior •  Hardware, cache, app itself •  External •  Focus is on overall system, usability

Slide 12

Slide 12 text

WHAT do we test? •  ExisOng site? Learn the traﬃc pa7erns. •  How do users use your site? •  Got history? Hit your web server logs, eg •  Build your scenarios to mimic •  Think Omes •  Ramp up

Slide 13

Slide 13 text

HOW do we test? •  Control the environment •  Change ONE THING at a Ome •  Automate setup & conﬁg •  Baseline data •  System conﬁg •  Flush caches, app pools, etc •  Warmup scripts

Slide 14

Slide 14 text

A Sample Load Test An online shopping cart applicaOon is required to support 10,000 concurrent users broken out into following acOviOes: •  7500 Virtual Users (VU’s) log in, browse through items and then log off. •  1250 VU’s log in, add items to their shopping cart, check out and then log off. •  750 VU’s log in, request RMA and then log off. •  500 VU’s just log in without any subsequent acOvity. *Each of the below is classified as a ‘User Profile’ or ‘Scenario’. -‐Wikipedia

Slide 15

Slide 15 text

How We Do It. Test Studio (create, execute) Controller / Reporter Agent (Generates Load / VU’s) Agent (Generates Load / VU’s) Agent (Generates Load / VU’s) ApplicaOon

Slide 16

Slide 16 text

Virtual User Agent (Generates Load / VU’s) ApplicaOon HTTP(S) Request to Server HTTP(S) Response from Server HTTP(S) Request to Server HTTP(S) Response from Server