Benchmarking with NGINX Introduced by Owen Garrett Presented by Rick Nelson Nginx, Inc.

About this webinar When you want to know how many resources to allocate for your NGINX servers or what the capacity of your current NGINX servers is, you need to be able to perform proper benchmark testing, but this can be complicated. In this webinar, you'll learn the considerations that need to go into planning, configuring and running benchmarks.

Agenda •  Introduc)on   •  Common  Pi/alls   •  Tips  and  Techniques   •  Demonstra)on   •  Ques)ons    

INTRODUCTION

What is NGINX? Internet N Web Server Serve content from disk Application Server FastCGI, uWSGI, Passenger… Proxy Caching, Load Balancing… HTTP traffic þ Application Acceleration þ SSL and SPDY termination þ Performance Monitoring þ High Availability Advanced Features: þ Bandwidth Management þ Content-based Routing þ Request Manipulation þ Response Rewriting þ Authentication þ Video Delivery þ Mail Proxy þ GeoLocation

Why do Benchmarking? •  Stress  test   •  Capacity  planning   •  Comparison  tes)ng  (bake  off)      

Benchmarking Considerations •  It’s  complicated   •  What  is  the  goal?   •  What  kind  of  test  environment  do  you  have?   •  What  tes)ng  tools  do  you  have?   •  How  well  can  you  simulate  produc)on  traffic?    

What areas are you testing? •  Web  server   •  Applica)on  Server   •  Reverse  Proxy   •  All  of  the  above      

What are you testing? •  Can  you  simulate  produc)on  traffic?   •  If  not,  what  are  you  concerned  about:   – Connec)ons   – Request  rate   – Bandwidth   – SSL   – All  of  the  Above      

What are you testing? •  Can  you  do  a  full  produc)on  test?   •  If  not,  do  a  smaller  scale  test  and  extrapolate   •  Know  your  traffic   – Get  vs.  Post,  request/response  sizes,  etc.   •  What  you  need  to  test  vs.  what  you  are   actually  tes)ng  

What are you testing? •  You  may  want  to  test  a  single  variable   – Connec)ons   – Request  rate   – Bandwidth   – New  SSL  handshakes    

Test Environment •  You  are  always  tes)ng  the  whole  environment   – Tes)ng  Tools   •  Load  generators   •  Web  Servers   – Systems  under  Test  (SUT)   •  Reverse  proxies   •  Web  servers    

Test Environment •  Good  rule  of  thumb  for  cores  needed:   – Load  Generators:  2N   – Reverse  Proxies:  N   – Web  Servers:  2N      

COMMON PITFALLS

Not Knowing What You Are Testing •  Know  your  tes)ng  tools   •  What  ques)on  does  a  test  answer?   – For  example:   •  How  many  requests  per  second  can  a  SUT  handle?   •  Can  a  SUT  handle  a  certain  number  of  requests  per   second      

Real Clients versus Synthetic Clients Real  Clients   Synthe.c  Clients   Latency   Low-­‐High   Low   Packet  Loss   Low-­‐High   Low   Bandwidth   Low-­‐High   High   Time  between  requests   Long   Short   Idle  connec)ons   Yes   No  

Unrealistic Synthetic Clients •  Misleading  results   –  System  looks  good  during  benchmark   –  System  has  problems  with  real  clients   •  Why  is  this?     –  Synthe)c  clients  are  ideal  for  the  server   •  Low  latency,  low  packet  loss,  busy  connec)ons   –  Real  clients  are  not   •  High  latency,  packet  loss,  idle  connec)ons      

Synthetic Clients •  Unrealis)c  synthe)c  clients   – Apache  Bench  (ab)   •  More  realis)c  synthe)c  clients   – Open  Source   •  Siege,  wrk   – Commercial   •  Spirent,  Ixia,  Cloudtest,  other  cloud  services    

Misconfiguration •  Many  configura)on  se_ngs  can  impact  tests   –  Some  Linux  kernel  se_ngs  may  be  too  low  for  heavy  loads   –  Keepalives   –  SSL  key  sizes  make  a  big  difference   –  Compression   –  Benchmark  clients  or  servers  

Tips and Techniques •  Use  mul)ple  approaches   –  Real  world  simula)ons  for  real  world  results   –  Simple  tests  for  baselines  and  debugging  

Tips and Techniques •  If  you  have  found  the  real  limit  of  a  SUT  then:   – At  least  one  system  resource  should  be  exhausted   •  CPU   •  Memory   •  Bandwidth   •  Disk  I/O   – If  not,  then  a  bo`leneck  exists  elsewhere  

Tips and Techniques •  Start  with  the  NGINX  defaults   •  NGINX  direc)ves  can  impact  performance   – accept_mutex,  worker_processes,   worker_connec)ons,  keepalive_)meout,   lingering_close,  sendfile,  keepalive,  aio,   open_file_cache  

Tips and Techniques •  Some  10G  drivers  don’t  use  cores  effec)vely   – Some  cores  at  100%,  others  have  low  usage   – Solu)on:  driver  dependent   •  Scrip)ng   •  New  driver  version   •  New  Card  

Tips and Techniques •  Monitor  error  files  and  errors  from  tes)ng  tools   – Errors  can  return  faster   – You  may  be  hi_ng  system  limits  or  errors   •  Don’t  have  load  genera)on  on  a  SUT   •  Double  check  the  result  figures  

Tips and Techniques •  Run  all  tests  mul)ple  )mes     •  If  using  virtualiza)on,  be  aware  of  other  loads   on  the  host   •  If  you  don’t  understand  the  results  -­‐  simplify  

Demonstration Load   Generator   Load   Generator   Apache   NGINX   ab   siege   Datacenter  

Questions and Answers

Closing thoughts •  38%  of  the  busiest  websites  use  NGINX   •  Check  out  the  previous  webinar  on  tuning  at   nginx.com   •  Future  webinars:  nginx.com/webinars   Try  NGINX  F/OSS  (nginx.org)  or  NGINX  Plus  (nginx.com)