Real World Problems, Real World Load Testing

Deck used for our Industry Technical Presentation (Vendor Session) at StarEast 2012. This deck accompanied my demo of Telerik's Test Studio Load Testing.

Transcript

1. Real  World  Problems,   Real  World  Load  Tes1ng   Jim  Holmes   Evangelist,  Test  Studio   @aJimHolmes   h7p://bit.ly/JimAtTestStudio   Jim.Holmes@Telerik.com  
2. Recent  News…  
3. What  the  CEO/customer  wants:   “Can  our  site  survive  as  we  grow?”     “Will  our  users  be  happy  when  the  system’s   being  used?”    
4. The  cost  of  degraded  performance   “According  to  a  2008  Aberdeen  Group  study,  a  1-­‐second  delay     in  page  load  Ome  equals  11  percent  fewer  page  views,  a  16     percent  decrease  in  customer  saOsfacOon,  and  a  7  percent     loss  in  conversions.  In  dollar  terms,  this  means  that  if  a  site     typically  earns  $100,000  a  day,  this  year  you  could  lose  $2.5     million  in  sales  for  every  second  slower  your  site  is  compared     to   last   year   (or,   for   that   ma7er,   compared   to   your   compeOtor’s  website).”    -­‐-­‐  Web  Load  Tes+ng  for  Dummies    
5. What’s  on  the  menu   •  Types  of  Load  TesOng   •  SeZng  ExpectaOons   •  What   •  Where   •  How   •  Demo!  
6. Types  of  Load  Tes1ng  
7. Soak  Tes1ng  
8. Failover  /  Tip  Tes1ng  
9. Load  Tes1ng  
10. Set  Expecta1ons  Early   •  What  are  our  realisOc  goals?   •  “Do  we  really  need  to  support  China  when  we  will   only  serve  our  neighborhood?”  -­‐-­‐  @BenCarey   •  Need  People,  Hardware,  TIME   •  What’s  on  the  table  for  fixing?  
11. WHERE  do  we  test?   •  Internal   •  Lets  us  focus  on  system  behavior   •  Hardware,  cache,  app  itself   •  External   •  Focus  is  on  overall  system,  usability  
12. WHAT  do  we  test?   •  ExisOng  site?  Learn  the  traffic  pa7erns.   •  How  do  users  use  your  site?   •  Got  history?  Hit  your  web  server  logs,  eg   •  Build  your  scenarios  to  mimic   •  Think  Omes   •  Ramp  up  
13. HOW  do  we  test?   •  Control  the  environment   •  Change  ONE  THING  at  a  Ome   •  Automate  setup  &  config   •  Baseline  data   •  System  config   •  Flush  caches,  app  pools,  etc   •  Warmup  scripts  
14. A  Sample  Load  Test   An   online   shopping   cart   applicaOon   is   required   to   support   10,000   concurrent   users   broken  out  into  following  acOviOes:     •  7500  Virtual  Users  (VU’s)  log  in,  browse  through  items  and  then  log  off.   •  1250  VU’s  log  in,  add  items  to  their  shopping  cart,  check  out  and  then  log  off.   •  750  VU’s  log  in,  request  RMA  and  then  log  off.   •  500  VU’s  just  log  in  without  any  subsequent  acOvity.   *Each  of  the  below  is  classified  as  a  ‘User  Profile’  or  ‘Scenario’.   -­‐Wikipedia  
15. How  We  Do  It.   Test  Studio   (create,  execute)   Controller  /   Reporter   Agent   (Generates  Load  /  VU’s)     Agent   (Generates  Load  /  VU’s)     Agent   (Generates  Load  /  VU’s)     ApplicaOon  
16. Virtual  User   Agent   (Generates  Load  /   VU’s)     ApplicaOon   HTTP(S)  Request  to  Server   HTTP(S)  Response  from  Server   HTTP(S)  Request  to  Server   HTTP(S)  Response  from  Server  
18. Selec1ng  a  VU  Pack…  
19. Jim  Holmes   Jim.Holmes@telerik.com   @aJimHolmes   @telerik  |  facebook.com/telerik