$30 off During Our Annual Pro Sale. View Details »

Benchmarking with NGINX

Benchmarking with NGINX

For the recorded webinar, visit nginx.com/webinars.

When you want to know how many resources to allocate for your NGINX servers or what the capacity of your current NGINX servers is, you need to be able to perform proper benchmark testing, but this can be complicated. In this webinar, you'll learn the considerations that need to go into planning, configuring and running benchmarks.

NGINX Inc

June 19, 2014
Tweet

More Decks by NGINX Inc

Other Decks in Technology

Transcript

  1. About this webinar When you want to know how many

    resources to allocate for your NGINX servers or what the capacity of your current NGINX servers is, you need to be able to perform proper benchmark testing, but this can be complicated. In this webinar, you'll learn the considerations that need to go into planning, configuring and running benchmarks.
  2. Agenda •  Introduc)on   •  Common  Pi/alls   •  Tips

     and  Techniques   •  Demonstra)on   •  Ques)ons    
  3. What is NGINX? Internet N Web Server Serve content from

    disk Application Server FastCGI, uWSGI, Passenger… Proxy Caching, Load Balancing… HTTP traffic þ Application Acceleration þ SSL and SPDY termination þ Performance Monitoring þ High Availability Advanced Features: þ Bandwidth Management þ Content-based Routing þ Request Manipulation þ Response Rewriting þ Authentication þ Video Delivery þ Mail Proxy þ GeoLocation
  4. Why do Benchmarking? •  Stress  test   •  Capacity  planning

      •  Comparison  tes)ng  (bake  off)      
  5. Benchmarking Considerations •  It’s  complicated   •  What  is  the

     goal?   •  What  kind  of  test  environment  do  you  have?   •  What  tes)ng  tools  do  you  have?   •  How  well  can  you  simulate  produc)on  traffic?    
  6. What areas are you testing? •  Web  server   • 

    Applica)on  Server   •  Reverse  Proxy   •  All  of  the  above      
  7. What are you testing? •  Can  you  simulate  produc)on  traffic?

      •  If  not,  what  are  you  concerned  about:   – Connec)ons   – Request  rate   – Bandwidth   – SSL   – All  of  the  Above      
  8. What are you testing? •  Can  you  do  a  full

     produc)on  test?   •  If  not,  do  a  smaller  scale  test  and  extrapolate   •  Know  your  traffic   – Get  vs.  Post,  request/response  sizes,  etc.   •  What  you  need  to  test  vs.  what  you  are   actually  tes)ng  
  9. What are you testing? •  You  may  want  to  test

     a  single  variable   – Connec)ons   – Request  rate   – Bandwidth   – New  SSL  handshakes    
  10. Test Environment •  You  are  always  tes)ng  the  whole  environment

      – Tes)ng  Tools   •  Load  generators   •  Web  Servers   – Systems  under  Test  (SUT)   •  Reverse  proxies   •  Web  servers    
  11. Test Environment •  Good  rule  of  thumb  for  cores  needed:

      – Load  Generators:  2N   – Reverse  Proxies:  N   – Web  Servers:  2N      
  12. Not Knowing What You Are Testing •  Know  your  tes)ng

     tools   •  What  ques)on  does  a  test  answer?   – For  example:   •  How  many  requests  per  second  can  a  SUT  handle?   •  Can  a  SUT  handle  a  certain  number  of  requests  per   second      
  13. Real Clients versus Synthetic Clients Real  Clients   Synthe.c  Clients

      Latency   Low-­‐High   Low   Packet  Loss   Low-­‐High   Low   Bandwidth   Low-­‐High   High   Time  between  requests   Long   Short   Idle  connec)ons   Yes   No  
  14. Unrealistic Synthetic Clients •  Misleading  results   –  System  looks

     good  during  benchmark   –  System  has  problems  with  real  clients   •  Why  is  this?     –  Synthe)c  clients  are  ideal  for  the  server   •  Low  latency,  low  packet  loss,  busy  connec)ons   –  Real  clients  are  not   •  High  latency,  packet  loss,  idle  connec)ons      
  15. Synthetic Clients •  Unrealis)c  synthe)c  clients   – Apache  Bench  (ab)

      •  More  realis)c  synthe)c  clients   – Open  Source   •  Siege,  wrk   – Commercial   •  Spirent,  Ixia,  Cloudtest,  other  cloud  services    
  16. Misconfiguration •  Many  configura)on  se_ngs  can  impact  tests   – 

    Some  Linux  kernel  se_ngs  may  be  too  low  for  heavy  loads   –  Keepalives   –  SSL  key  sizes  make  a  big  difference   –  Compression   –  Benchmark  clients  or  servers  
  17. Tips and Techniques •  Use  mul)ple  approaches   –  Real

     world  simula)ons  for  real  world  results   –  Simple  tests  for  baselines  and  debugging  
  18. Tips and Techniques •  If  you  have  found  the  real

     limit  of  a  SUT  then:   – At  least  one  system  resource  should  be  exhausted   •  CPU   •  Memory   •  Bandwidth   •  Disk  I/O   – If  not,  then  a  bo`leneck  exists  elsewhere  
  19. Tips and Techniques •  Start  with  the  NGINX  defaults  

    •  NGINX  direc)ves  can  impact  performance   – accept_mutex,  worker_processes,   worker_connec)ons,  keepalive_)meout,   lingering_close,  sendfile,  keepalive,  aio,   open_file_cache  
  20. Tips and Techniques •  Some  10G  drivers  don’t  use  cores

     effec)vely   – Some  cores  at  100%,  others  have  low  usage   – Solu)on:  driver  dependent   •  Scrip)ng   •  New  driver  version   •  New  Card  
  21. Tips and Techniques •  Monitor  error  files  and  errors  from

     tes)ng  tools   – Errors  can  return  faster   – You  may  be  hi_ng  system  limits  or  errors   •  Don’t  have  load  genera)on  on  a  SUT   •  Double  check  the  result  figures  
  22. Tips and Techniques •  Run  all  tests  mul)ple  )mes  

      •  If  using  virtualiza)on,  be  aware  of  other  loads   on  the  host   •  If  you  don’t  understand  the  results  -­‐  simplify  
  23. Demonstration Load   Generator   Load   Generator   Apache

      NGINX   ab   siege   Datacenter  
  24. Closing thoughts •  38%  of  the  busiest  websites  use  NGINX

      •  Check  out  the  previous  webinar  on  tuning  at   nginx.com   •  Future  webinars:  nginx.com/webinars   Try  NGINX  F/OSS  (nginx.org)  or  NGINX  Plus  (nginx.com)