NGINX High Availability and Monitoring Introduced by Andrew Alexeev Presented by Owen Garrett Nginx, Inc. 

About this webinar No one likes a broken website. Learn about some of the techniques that NGINX users employ to ensure that server failures are detected and worked around, so that you too can build large-scale, highly-available web services. 

The  cost  of  down-me   

The  causes  of  down-me   “  Through  2015,  80%  of  outages  impac-ng  mission-­‐ cri-cal  services  will  be  caused  by  people  and  process   issues,  and  more  than  50%  of  those  outages  will  be   caused  by  change/configura-on/release  integra-on   and  hand-­‐off  issues.  ”   Configura-on  Management  for  Virtual  and  Cloud   Infrastructures   Ronni  J.  Colville  and  George  Spafford,  Gartner   Hardware  failures,  disasters   People  and  Process   

INTRODUCING  NGINX…   

What  is  NGINX?   Internet N Web Server Serve content from disk Application Server FastCGI, uWSGI, Passenger… Proxy Caching, Load Balancing… HTTP traffic þ Application Acceleration þ SSL and SPDY termination þ Performance Monitoring þ High Availability Advanced Features: þ Bandwidth Management þ Content-based Routing þ Request Manipulation þ Response Rewriting þ Authentication þ Video Delivery þ Mail Proxy þ GeoLocation 

143,000,000 Websites NGINX Accelerates 

22% Top 1 million websites 37% Top 1,000 websites 

NGINX  and  NGINX  Plus   NGINX  F/OSS     nginx.org   3rd  party     modules   Large  community  of  >100  modules   

NGINX  and  NGINX  Plus   NGINX  F/OSS     nginx.org   3rd  party     modules   Large  community  of  >100  modules   NGINX  Plus     Advanced  load  balancing  features   Ease-­‐of-­‐management   Commercial  support   

IMPROVING  AVAILABILITY  WITH  NGINX   

Quick  review  of  load  balancing   server {! listen 80;! ! location / {! proxy_pass http://backend;! }! }! ! upstream backend {! server webserver1:80;! server webserver2:80; ! server webserver3:80;! server webserver4:80;! }! Internet N 

Three  NGINX  Techniques  for  High  Availability   NGINX:  Basic  Error  Checks     NGINX  Plus:  Advanced  Health  Checks     Live  so_ware  upgrades   1 2 3 

1.  Basic  Error  Checks   •  Monitor  transac-ons  as  they  happen   – Retry  transac-ons  that  ‘fail’  where  possible   – Mark  failed  servers  as  dead   

Basic  Error  Checks   server {! listen 80;! ! location / {! proxy_pass http://backend;! proxy_next_upstream error timeout; # http_503..., off! }! }! ! upstream backend {! server webserver1:80 max_fails=1 fail_timeout=10s;! server webserver2:80 max_fails=1 fail_timeout=10s; ! server webserver3:80 max_fails=1 fail_timeout=10s;! server webserver4:80 max_fails=1 fail_timeout=10s;! }! 

More  sophis-cated  retries   server {! listen 80;! ! location / {! # On error/timeout, try the upstream group one more time! error_page 502 504 = @fallback; ! proxy_pass http://backend;! proxy_next_upstream off;! }! ! location @fallback {! proxy_pass http://backend;! proxy_next_upstream off;! }! }! 

2.  Advanced  Health  Checks   •  “Synthe-c  Transac-ons”   – Probes  server  health   – Complex,  custom  tests  are  possible   – Available  in  NGINX  Plus   

Advanced  Health  Checks   server {! listen 80;! ! location / {! proxy_pass http://backend;! health_check;! }! }! ! upstream backend {! zone backend 64k;! server webserver1:80;! server webserver2:80; ! server webserver3:80;! server webserver4:80;! }! health_check:    interval  =  period  between  checks    fails  =  failure  count  before  dead    passes  =  pass  count  before  alive    uri  =  custom  URI     Default:    5  seconds,  1  fail,  1  pass,  uri  =  /   

Advanced  usage   server {! listen 80;! ! location / {! proxy_pass http://backend;! ! health_check uri=/test.php match=statusok;! proxy_set_header Host www.foo.com;! }! }! ! match statusok {! # Used for /test.php health check! status 200;! header Content-Type = text/html;! body ~ "Server[0-9]+ is alive";! }! Health  checks  inherit  all   parameters  from  loca-on   block.     match  blocks  define  the   success  criteria  for  a   health  check     

Edge  cases  –  variables  in  configura-on   server {! location / {! proxy_pass http://backend;! health_check;! proxy_set_header Host $host;! }! }! This  may  not  work  as  expected.     Remember  –  the  health_check   tests  run  in  the  context  of  the   enclosing  loca-on.   

Edge  cases  –  variables  in  configura-on   server {! location / {! proxy_pass http://backend;! health_check;! proxy_set_header Host $host;! }! }! server {! location /internal-check {! internal;! proxy_pass http://backend;! health_check;! proxy_set_header Host www.foo.com;! }! }! This  may  not  work  as  expected.     Remember  –  the  health_check   tests  run  in  the  context  of  the   enclosing  loca-on.   This  is  the  common  alterna-ve.     Use  a  custom  URI  for  the  loca-on.   Tag  the  loca-on  as  internal.   Set  headers  manually.   Useful  for  authen.ca.on.   

Examples  of  using  health  checks   •  Verify  that  pages   don’t  contain  errors   •  Run  internal  tests  (e.g.  test.php  =>  DB  connect)   •  Managed  removal  of  servers    $ touch $DOCROOT/isactive.txt! 

Advantages  of  ‘Health  Checks’   •  Run  tests  asynchronously  (find  errors  faster)   •  Custom  tests  (not  related  to  ‘real’  traffic)   •  More  flexibility  to  specify  success/error   

MORE  NGINX  PLUS  FEATURES…   

Slow  start   •  When  basic  error  checks  and  advanced  health   checks  recover:     upstream backends {! zone backends 64k;! ! server webserver1 slow_start=30s;! }! 

NGINX  Plus  status  monitoring   hkp://demo.nginx.com/  and  hkp://demo.nginx.com/status   Total  data  and  connec-ons   Current  data  and  conns.     Split  per  ‘server  zone’     Cache  sta-s-cs     Upstream  sta-s-cs:    Traffic    Health  and  Error  status     (web)   (JSON)   

3.  Live  so_ware  upgrades   •  Upgrade  your  NGINX  binary  on-­‐the-­‐fly   – No  down-me   – No  dropped  connec-ons   

No  down-me  –  ever!   •  Reload  configura-on  with  SIGHUP      #  nginx –s reload! •  Re-­‐exec  binary  with  copy-­‐and-­‐signal    hkp://nginx.org/en/docs/control.html#upgrade   NGINX  parent  process   NGINX  workers   NGINX  workers   NGINX  workers   NGINX  workers   

In  summary...   Basic  Error  checks  and  retry  logic   On-­‐the-­‐fly  upgrades   Advanced  health  checks  +  slow  start   Extended  status  monitoring   NGINX  F/OSS:     NGINX  Plus:     Compared  to  other  load  balancers  and  ADCs,  NGINX  Plus  is  uniquely  well-­‐suited   to  a  devops-­‐driven  environment.   

Closing  thoughts   •  37%  of  the  busiest  websites  use  NGINX   –  In  most  situa-ons,  it’s  a  drop-­‐in  extension   •  Check  out  the  blogs  on  nginx.com   •  Future  webinars:  nginx.com/webinars   Try  NGINX  F/OSS  (nginx.org)  or  NGINX  Plus  (nginx.com)