Cheating Your Way to Webscale

Python Nordeste, May 2nd 2014

Transcript

1. Python Nordeste
   May 2nd 2014
   Matt Robenolt
   Cheating Your Way to
   #webscale
   

   View Slide

2. Hello
   < me irl
   

   View Slide

3. Lead Operations Engineer
   

   View Slide

4. Core Contributor
   

   View Slide

5. So what is #webscale?
   

   View Slide

6. 10 million requests per second
   4ms mean response time
   asynchronous io
   mongodb
   

   View Slide

7. 10 million requests per second
   4ms mean response time
   asynchronous io
   mongodb
   NOPE
   

   View Slide

8. Disqus only does
   150 req/s
   per web server.
   * we also write some bad code
   

   View Slide

9. 150
   12,960,000
   388,800,000
   per second
   per day
   per month
   real world #webscale
   

   View Slide

10. Scale is about hiding the
    fact that your application is
    actually really slow.
    

    View Slide

11. If your application feels
    fast, then it’s probably
    good enough.
    

    View Slide

12. Users hate waiting for shit.
    

    View Slide

13. So how do we do it?
    

    View Slide

14. Cheating 101
    

    View Slide

15. When a user asks for new
    data, let’s give them old
    data instead.
    

    View Slide

16. When a user asks for new
    data, let’s give them old
    data instead.
    Caching
    

    View Slide

17. When telling us to do
    something, let’s say we did
    and maybe do it later.
    

    View Slide

18. When telling us to do
    something, let’s say we did
    and maybe do it later.
    Queueing
    

    View Slide

19. Rule #1
    Don’t get caught.
    

    View Slide

20. Rule #2
    Don’t get caught.
    

    View Slide

21. Rule #3
    Don’t get caught.
    

    View Slide

22. HTTP Caching
    

    View Slide

23. Introducing
    

    View Slide

24. tl;dr
    Varnish sits between your application and your users
    Internet
    

    View Slide

25. Let’s talk about HTTP.
    Hypertext Transport Protocol
    

    View Slide

26. $ curl -v disqus.com
    

    View Slide

27. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    

    View Slide

28. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Request
    

    View Slide

29. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Method
    

    View Slide

30. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Path
    

    View Slide

31. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Version
    

    View Slide

32. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Headers
    

    View Slide

33. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Response
    

    View Slide

34. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Status
    

    View Slide

35. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Headers
    

    View Slide

36. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    

    View Slide

37. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    For 300 seconds, all users
    will get the same response
    without talking to our
    application.
    

    View Slide

38. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    With power comes great
    responsibility.
    

    View Slide

39. GET /
    INTERNET Varnish Web servers
    

    View Slide

40. GET /
    INTERNET Varnish Web servers
    CACHED!
    “Cache-Control: max-age=300”
    

    View Slide

41. GET /
    INTERNET Varnish Web servers
    

    View Slide

42. GET /
    INTERNET Varnish Web servers
    CACHED!
    

    View Slide

43. BUT WAIT…
    THERE’S MORE
    

    View Slide

44. COLLAPSING
    REQUEST
    

    View Slide

45. GET /
    INTERNET Varnish Web servers
    

    View Slide

46. INTERNET Varnish Web servers
    GET /
    

    View Slide

47. INTERNET Varnish Web servers
    GET /
    If multiple users request
    the same object, Varnish
    makes one fetch and
    returns to all users.
    

    View Slide

48. Queueing
    

    View Slide

49. Do as little work as possible,
    and return a promise that
    this work will be done.
    

    View Slide

50. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo
    

    View Slide

51. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo
    

    View Slide

52. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo
    Workers can rate limit,
    debounce, increment
    counters, generate a fast
    materialized view, etc.
    

    View Slide

53. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo
    Make sure your tasks finish
    before a user tries to read
    the data back.
    

    View Slide

54. Final Thoughts
    

    View Slide

55. Understand your application.
    Where can you cheat without disrupting user experience?
    Is seeing a few seconds old data going to damage a product?
    

    View Slide

56. Cheating should only
    enhance user experience.
    

    View Slide

57. Cheat any way you can, just
    don’t get caught.
    

    View Slide

58. Django &
    Varnish &
    RabbitMQ &
    Celery &
    PostgreSQL &
    Redis &

    Cassandra &
    Riak Thanks
    

    View Slide

59. Questions? I have answers.
    ^
    github.com/mattrobenolt
    @mattrobenolt
    some
    

    View Slide