Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Website on cache: scalable, cheap and realiable...

Riga Dev Day
March 13, 2016
40

Website on cache: scalable, cheap and realiable by Samat Galimov

Riga Dev Day

March 13, 2016
Tweet

Transcript

  1. Meduza • exiled news outlet from Russia, built by the

    former team of the most popular news website Lenta.Ru (>10mln readers monthly, #1 cited in Russian Internet) • Website, iOS, Android and Windows Phone apps with consistent user experience, browser extensions & more • 7 developers in total, 30 people editorial team
  2. Stats • Launched Oct 2014 • 4mln readers, 45mln pageviews

    in Feb 2016 • 200TB in traffic • 400 eur for edge servers, 1 (one) systems engineer
  3. External requirements • traffic bursts 1:10 or more (news websites

    at 9/11) — we've seen in in Feb 2015, Nemtsov assassination — 500k daily users,100k users avg • no serious upfront investment in hardware or services • minimum fee possible • good latency & speed
  4. 'usual' nginx cache with 1min expire • editorial waiting to

    see content live (it's compiling!) • uneven distribution — contant 'you might be on the late server' excuses • website deployment delay (we do debug on prod )
  5. • master - slave arch. • outer world communicates with

    master • sync API on master: 'purge' & 'heat' • master maintains list of all slaves and passes messages to them • clear nginx cache only if upstream returns 200
  6. hardware & hosting for edge • started with Amazon AWS

    (intel virtual) • then bare metal (intel xeon) • then scaleway+online.net (intel atom + arm) • main concern is bandwith per node per price
  7. latency is everything • DNS resolve: 20ms-200ms (amazon vs cloudflare)

    • TLS negotiation: 0-200ms (RFC 5077) • backend page generation: 0-400ms (node vs nginx) • deep system tuning: scheduler, buffers, network cards, etc. • data transfer: closer to the clients (Moscow server) • dom loading: server side rendering for nodejs
  8. operations • 1-click deployment of edge nodes via ansible •

    self-registration of edge CacheM to master • cacheM master notifies on issues with slaves • graphana graphs for traffic etc. • ELK for access logs