Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cheating Your Way to Webscale

Cheating Your Way to Webscale

Python Nordeste, May 2nd 2014

Matt Robenolt

May 02, 2014
Tweet

More Decks by Matt Robenolt

Other Decks in Programming

Transcript

  1. Python Nordeste
    May 2nd 2014
    Matt Robenolt
    Cheating Your Way to
    #webscale

    View Slide

  2. Hello
    < me irl

    View Slide

  3. Lead Operations Engineer

    View Slide

  4. Core Contributor

    View Slide

  5. So what is #webscale?

    View Slide

  6. 10 million requests per second
    4ms mean response time
    asynchronous io
    mongodb

    View Slide

  7. 10 million requests per second
    4ms mean response time
    asynchronous io
    mongodb
    NOPE

    View Slide

  8. Disqus only does
    150 req/s
    per web server.
    * we also write some bad code

    View Slide

  9. 150
    12,960,000
    388,800,000
    per second
    per day
    per month
    real world #webscale

    View Slide

  10. Scale is about hiding the
    fact that your application is
    actually really slow.

    View Slide

  11. If your application feels
    fast, then it’s probably
    good enough.

    View Slide

  12. Users hate waiting for shit.

    View Slide

  13. So how do we do it?

    View Slide

  14. Cheating 101

    View Slide

  15. When a user asks for new
    data, let’s give them old
    data instead.

    View Slide

  16. When a user asks for new
    data, let’s give them old
    data instead.
    Caching

    View Slide

  17. When telling us to do
    something, let’s say we did
    and maybe do it later.

    View Slide

  18. When telling us to do
    something, let’s say we did
    and maybe do it later.
    Queueing

    View Slide

  19. Rule #1
    Don’t get caught.

    View Slide

  20. Rule #2
    Don’t get caught.

    View Slide

  21. Rule #3
    Don’t get caught.

    View Slide

  22. HTTP Caching

    View Slide

  23. Introducing

    View Slide

  24. tl;dr
    Varnish sits between your application and your users
    Internet

    View Slide

  25. Let’s talk about HTTP.
    Hypertext Transport Protocol

    View Slide

  26. $ curl -v disqus.com

    View Slide

  27. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300

    View Slide

  28. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Request

    View Slide

  29. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Method

    View Slide

  30. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Path

    View Slide

  31. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Version

    View Slide

  32. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Headers

    View Slide

  33. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Response

    View Slide

  34. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Status

    View Slide

  35. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    Headers

    View Slide

  36. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300

    View Slide

  37. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    For 300 seconds, all users
    will get the same response
    without talking to our
    application.

    View Slide

  38. > GET / HTTP/1.1
    > User-Agent: curl/7.24.0
    > Host: disqus.com
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Server: nginx
    < Date: Fri, 02 May 2014 06:38:37 GMT
    < Content-Type: text/html; charset=utf-8
    < Content-Length: 10453
    < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT
    < Vary: Accept-Encoding
    < Expires: Fri, 02 May 2014 06:43:36 GMT
    < Cache-Control: public, max-age=300
    With power comes great
    responsibility.

    View Slide

  39. GET /
    INTERNET Varnish Web servers

    View Slide

  40. GET /
    INTERNET Varnish Web servers
    CACHED!
    “Cache-Control: max-age=300”

    View Slide

  41. GET /
    INTERNET Varnish Web servers

    View Slide

  42. GET /
    INTERNET Varnish Web servers
    CACHED!

    View Slide

  43. BUT WAIT…
    THERE’S MORE

    View Slide

  44. COLLAPSING
    REQUEST

    View Slide

  45. GET /
    INTERNET Varnish Web servers

    View Slide

  46. INTERNET Varnish Web servers
    GET /

    View Slide

  47. INTERNET Varnish Web servers
    GET /
    If multiple users request
    the same object, Varnish
    makes one fetch and
    returns to all users.

    View Slide

  48. Queueing

    View Slide

  49. Do as little work as possible,
    and return a promise that
    this work will be done.

    View Slide

  50. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo

    View Slide

  51. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo

    View Slide

  52. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo
    Workers can rate limit,
    debounce, increment
    counters, generate a fast
    materialized view, etc.

    View Slide

  53. INTERNET Web servers Task workers
    Slow/Fast Data store
    Queue
    POST /foo
    Make sure your tasks finish
    before a user tries to read
    the data back.

    View Slide

  54. Final Thoughts

    View Slide

  55. Understand your application.
    Where can you cheat without disrupting user experience?
    Is seeing a few seconds old data going to damage a product?

    View Slide

  56. Cheating should only
    enhance user experience.

    View Slide

  57. Cheat any way you can, just
    don’t get caught.

    View Slide

  58. Django &
    Varnish &
    RabbitMQ &
    Celery &
    PostgreSQL &
    Redis &

    Cassandra &
    Riak Thanks

    View Slide

  59. Questions? I have answers.
    ^
    github.com/mattrobenolt
    @mattrobenolt
    some

    View Slide