Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cheating Your Way to Webscale
Search
Matt Robenolt
May 02, 2014
Programming
13
1.4k
Cheating Your Way to Webscale
Python Nordeste, May 2nd 2014
Matt Robenolt
May 02, 2014
Tweet
Share
More Decks by Matt Robenolt
See All by Matt Robenolt
Everything is broken and I don't know why.
mattrobenolt
0
44
I am bad at my job.
mattrobenolt
0
190
Everything is broken, and I don't know why. Python edition.
mattrobenolt
1
190
Everything is broken, and I don't know why. Python edition.
mattrobenolt
2
560
Varnish: How We Do It
mattrobenolt
1
210
Everything is broken, and I don't know why.
mattrobenolt
7
1.5k
HTTP for Great Good
mattrobenolt
85
200k
Caching is Hard: Varnish @ Disqus
mattrobenolt
52
2.1M
Developing & Deploying "Large" Scale Web Applications
mattrobenolt
25
1.3k
Other Decks in Programming
See All in Programming
Effect の双対、Coeffect
yukikurage
4
420
衛星の軌道をWeb地図上に表示する
sankichi92
0
260
単体テストの始め方/作り方
toms74209200
0
350
Efficiency and Rock 'n’ Roll (Really!)
hollycummins
0
670
Practical Tips and Tricks for Working with Compose Multiplatform Previews (mDevCamp 2025)
stewemetal
0
110
Step up the performance game with Spring Boot and Project Leyden
mhalbritter
0
170
ts-morph実践:型を利用するcodemodのテクニック
ypresto
1
580
SODA - FACT BOOK
sodainc
1
360
プロダクト改善のために新しいことを始める -useContextからの卒業、Zustandへ-
rebase_engineering
1
110
データベースコネクションプール(DBCP)の変遷と理解
fujikawa8
0
200
eBPFを用いたAIネットワーク監視システム論文の実装 / eBPF Japan Meetup #4
yuukit
3
700
Webからモバイルへ Vue.js × Capacitor 活用事例
naokihaba
0
130
Featured
See All Featured
Writing Fast Ruby
sferik
628
61k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
Navigating Team Friction
lara
186
15k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
How to Ace a Technical Interview
jacobian
276
23k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.3k
We Have a Design System, Now What?
morganepeng
52
7.6k
Rails Girls Zürich Keynote
gr2m
94
13k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
137
34k
Optimizing for Happiness
mojombo
378
70k
Transcript
Python Nordeste May 2nd 2014 Matt Robenolt Cheating Your Way
to #webscale
Hello < me irl
Lead Operations Engineer
Core Contributor
So what is #webscale?
10 million requests per second 4ms mean response time asynchronous
io mongodb
10 million requests per second 4ms mean response time asynchronous
io mongodb NOPE
Disqus only does 150 req/s per web server. * we
also write some bad code
150 12,960,000 388,800,000 per second per day per month real
world #webscale
Scale is about hiding the fact that your application is
actually really slow.
If your application feels fast, then it’s probably good enough.
Users hate waiting for shit.
So how do we do it?
Cheating 101
When a user asks for new data, let’s give them
old data instead.
When a user asks for new data, let’s give them
old data instead. Caching
When telling us to do something, let’s say we did
and maybe do it later.
When telling us to do something, let’s say we did
and maybe do it later. Queueing
Rule #1 Don’t get caught.
Rule #2 Don’t get caught.
Rule #3 Don’t get caught.
HTTP Caching
Introducing
tl;dr Varnish sits between your application and your users Internet
Let’s talk about HTTP. Hypertext Transport Protocol
$ curl -v disqus.com
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Request
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Method
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Path
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Version
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Headers
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Response
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Status
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 Headers
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 For 300 seconds, all users will get the same response without talking to our application.
> GET / HTTP/1.1 > User-Agent: curl/7.24.0 > Host: disqus.com
> Accept: */* > < HTTP/1.1 200 OK < Server: nginx < Date: Fri, 02 May 2014 06:38:37 GMT < Content-Type: text/html; charset=utf-8 < Content-Length: 10453 < Last-Modified: Fri, 30 Aug 2013 00:32:14 GMT < Vary: Accept-Encoding < Expires: Fri, 02 May 2014 06:43:36 GMT < Cache-Control: public, max-age=300 With power comes great responsibility.
GET / INTERNET Varnish Web servers
GET / INTERNET Varnish Web servers CACHED! “Cache-Control: max-age=300”
GET / INTERNET Varnish Web servers
GET / INTERNET Varnish Web servers CACHED!
BUT WAIT… THERE’S MORE
COLLAPSING REQUEST
GET / INTERNET Varnish Web servers
INTERNET Varnish Web servers GET /
INTERNET Varnish Web servers GET / If multiple users request
the same object, Varnish makes one fetch and returns to all users.
Queueing
Do as little work as possible, and return a promise
that this work will be done.
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo Workers can rate limit, debounce, increment counters, generate a fast materialized view, etc.
INTERNET Web servers Task workers Slow/Fast Data store Queue POST
/foo Make sure your tasks finish before a user tries to read the data back.
Final Thoughts
Understand your application. Where can you cheat without disrupting user
experience? Is seeing a few seconds old data going to damage a product?
Cheating should only enhance user experience.
Cheat any way you can, just don’t get caught.
Django & Varnish & RabbitMQ & Celery & PostgreSQL &
Redis & Cassandra & Riak Thanks
Questions? I have answers. ^ github.com/mattrobenolt @mattrobenolt some