Tweet
Tweet

Slide 1

Slide 1 text

Python Nordeste May 2nd 2014 Matt Robenolt Cheating Your Way to #webscale

Slide 2

Slide 2 text

Hello < me irl

Slide 3

Slide 3 text

Lead Operations Engineer

Slide 4

Slide 4 text

Core Contributor

Slide 5

Slide 5 text

So what is #webscale?

Slide 6

Slide 6 text

10 million requests per second 4ms mean response time asynchronous io mongodb

Slide 7

Slide 7 text

10 million requests per second 4ms mean response time asynchronous io mongodb NOPE

Slide 8

Slide 8 text

Disqus only does 150 req/s per web server. * we also write some bad code

Slide 9

Slide 9 text

150 12,960,000 388,800,000 per second per day per month real world #webscale

Slide 10

Slide 10 text

Scale is about hiding the fact that your application is actually really slow.

Slide 11

Slide 11 text

If your application feels fast, then it’s probably good enough.

Slide 12

Slide 12 text

Users hate waiting for shit.

Slide 13

Slide 13 text

So how do we do it?

Slide 14

Slide 14 text

Cheating 101

Slide 15

Slide 15 text

When a user asks for new data, let’s give them old data instead.

Slide 16

Slide 16 text

When a user asks for new data, let’s give them old data instead. Caching

Slide 17

Slide 17 text

When telling us to do something, let’s say we did and maybe do it later.

Slide 18

Slide 18 text

When telling us to do something, let’s say we did and maybe do it later. Queueing

Slide 19

Slide 19 text

Rule #1 Don’t get caught.

Slide 20

Slide 20 text

Rule #2 Don’t get caught.

Slide 21

Slide 21 text

Rule #3 Don’t get caught.

Slide 22

Slide 22 text

HTTP Caching

Slide 23

Slide 23 text

Introducing

Slide 24

Slide 24 text

tl;dr Varnish sits between your application and your users Internet

Slide 25

Slide 25 text

Let’s talk about HTTP. Hypertext Transport Protocol

Slide 26

Slide 26 text

$ curl -v disqus.com

Slide 27

Slide 27 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300

Slide 28

Slide 28 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Request

Slide 29

Slide 29 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Method

Slide 30

Slide 30 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Path

Slide 31

Slide 31 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Version

Slide 32

Slide 32 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Headers

Slide 33

Slide 33 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Response

Slide 34

Slide 34 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Status

Slide 35

Slide 35 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Headers

Slide 36

Slide 36 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300

Slide 37

Slide 37 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 For 300 seconds, all users will get the same response without talking to our application.

Slide 38

Slide 38 text

> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com > Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 With power comes great responsibility.

Slide 39

Slide 39 text

GET / INTERNET Varnish Web servers

Slide 40

Slide 40 text

GET / INTERNET Varnish Web servers CACHED! “Cache-Control: max-age=300”

Slide 41

Slide 41 text

GET / INTERNET Varnish Web servers

Slide 42

Slide 42 text

GET / INTERNET Varnish Web servers CACHED!

Slide 43

Slide 43 text

BUT WAIT… THERE’S MORE

Slide 44

Slide 44 text

COLLAPSING REQUEST

Slide 45

Slide 45 text

GET / INTERNET Varnish Web servers

Slide 46

Slide 46 text

INTERNET Varnish Web servers GET /

Slide 47

Slide 47 text

INTERNET Varnish Web servers GET / If multiple users request the same object, Varnish makes one fetch and returns to all users.

Slide 48

Slide 48 text

Queueing

Slide 49

Slide 49 text

Do as little work as possible, and return a promise that this work will be done.

Slide 50

Slide 50 text

INTERNET Web servers Task workers Slow/Fast Data store Queue POST /foo

Slide 51

Slide 51 text

INTERNET Web servers Task workers Slow/Fast Data store Queue POST /foo

Slide 52

Slide 52 text

INTERNET Web servers Task workers Slow/Fast Data store Queue POST /foo Workers can rate limit, debounce, increment counters, generate a fast materialized view, etc.

Slide 53

Slide 53 text

INTERNET Web servers Task workers Slow/Fast Data store Queue POST /foo Make sure your tasks finish before a user tries to read the data back.

Slide 54

Slide 54 text

Final Thoughts

Slide 55

Slide 55 text

Understand your application. Where can you cheat without disrupting user experience? Is seeing a few seconds old data going to damage a product?

Slide 56

Slide 56 text

Cheating should only enhance user experience.

Slide 57

Slide 57 text

Cheat any way you can, just don’t get caught.

Slide 58

Slide 58 text

Django & Varnish & RabbitMQ & Celery & PostgreSQL & Redis &
 Cassandra & Riak Thanks

Slide 59

Slide 59 text

Questions? I have answers. ^ github.com/mattrobenolt @mattrobenolt some