Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Builing an HTTP Request Router with NGINX and Lua

Avatar for Scott Francis Scott Francis
September 23, 2015

Builing an HTTP Request Router with NGINX and Lua

Avatar for Scott Francis

Scott Francis

September 23, 2015
Tweet

More Decks by Scott Francis

Other Decks in Technology

Transcript

  1. Quick Stats 300M U N I Q U E V

    I S I T S / M O N T H 70ms AV E R AG E R E S P O N S E T I M E 10,000+ M A X C H E C KO U T S P E R M I N U T E 3 DATA C E N T E R S
  2. Architecture • Multi-tenant Rails monolith • Sharded MySQL persistent store

    • Redis and memcached • ElasticSearch • NGINX
  3. Multiple Data Centers • Planned to bring second data center

    online in early 2015 (Active-Passive). • Downtime costs our merchants money. • How would we perform the DC failover (success-over)? • How could we test the second data center before flipping the switch?
  4. +

  5. Middlewares • Like Rack, but for Lua and NGINX –

    Expose a Lua module that exports two functions: call, and init_worker. • Runs in an NGINX request phase — access_by_lua, rewrite_by_lua, content_by_lua, etc. • Configured via Chef – NGINX configuration file is generated by Chef. • Example Middlewares — request ID generation, Kafka logging, ping, HTTPDB
  6. Middleware Configuration { :ping => { :phase => :rewrite },

    :request_id => { :phase => :rewrite }, :sorting_hat => { :phase => :rewrite, :dict => { :shop_id_cache => "16M", :pod_id_cache => "16M", :oor_pods => "1M", :pod_to_upstream => "1M", }, }, :monitor => { }, }
  7. Circuit Breaker • Don’t access resources when they are down

    or slow • Have a fallback path — use a cached value, return an error, etc.
  8. Toxiproxy • A framework for simulating network conditions in development

    and test • Platform agnostic — Client libraries for Ruby, Go. .NET, Node, PHP • Toxics manipulate network conditions – latency, down connection, bandwidth, and more
  9. Data Centre Move Lead Up • Incremental Rollout — Deployed

    Sorting Hat into production, though it was effectively doing nothing. • Simple Deployment — One button deploys and rollbacks. • Weekly Practice — Use Sorting Hat to move a test shard back and forth.
  10. Take Aways • Follow good software development practices when building

    infrastructure • Testing and resiliency are crucial • Iterate fast and often
  11. The Future • POP Servers • MOAR Middlewares — We’re

    working on more middlewares. • Open source — Contributing our work back to the community.