Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
I'm doing HTTP wrong
Search
fireteam
May 28, 2012
Technology
2.2k
10
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
I'm doing HTTP wrong
The slides from the presentation at PyGrunn 2012.
fireteam
May 28, 2012
Transcript
58 slides
I am doing HTTP wrong — a presentation by Armin
Ronacher @mitsuhiko
The Web developer's Evolution
echo
request.send_header(…) request.end_headers() request.write(…)
return Response(…)
Why Stop there?
What do we love about HTTP?
Text Based
REST
Cacheable
Content Negotiation
Well Supported
Works where TCP doesn't
Somewhat Simple
Upgrades to custom protocols
Why does my application look like HTTP?
everybody does it
Natural Conclusion
we can do better!
we're a level too low
Streaming: one piece at the time, constant memory usage, no
seeking.
Buffering: have some data in memory, variable memory usage, seeking.
TYPICAL Request / Response Cycle User Agent Proxy Server Application
Stream “Buffered” Dispatcher View
In Python Terms def application(environ, start_response): # Step 1: acquire
data data = environ['wsgi.input'].read(...) # Step 2: process data response = process_data(data) # Step 3: respond start_response('200 OK', [('Content-Type', 'text/plain')]) return [response]
One Level Up s = socket.accept() f = s.makefile('rb') requestline
= f.readline() headers = [] while 1: headerline = f.readline() if headerline == '\r\n': break headers.append(headerline)
Weird Mixture on the app request.headers <- buffered request.form <-
buffered request.files <- buffered to disk request.body <- streamed
HTTP's Limited signalling Strict Request / Response The only communication
during request from the server to the client is closing the connection once you started accepting the body.
Bailing out early def application(request): # At this point, headers
are parsed, everything else # is not parsed yet. if request.content_length > TWO_MEGABYTES: return error_response() ...
Bailing out a little bit later def application(request): # Read
a little bit of data request.input.read(4096) # You just committed to accepting data, now you have to # read everything or the browser will be very unhappy and # Just time out. No more responding with 413 ...
Rejecting Form fields -> memory File uploads -> disk What's
your limit? 16MB in total? All could go to memory. Reject file sizes individually? Needs overall check as well!
The Consequences How much data do you accept? Limit the
overall request size? Not helpful because all of it could be in-memory
It's not just limiting Consider a layered system How many
of you write code that streams? What happens if you pass streamed data through your layers?
A new approach
Dynamic typing made us lazy
we're trying to solve both use cases in one we're
not supporting either well
How we do it Hide HTTP from the apps HTTP
is an implementation detail
Pseudocode user_pagination = make_pagination_schema(User) @export( specs=[('page', types.Int32()), ('per_page', types.Int32())], returns=user_pagination,
semantics='select', http_path='/users/' ) def list_users(page, per_page): users = User.query.paginate(page, per_page) return users.to_dict()
Types are specific user_type = types.Object([ ('username', types.String(30)), ('email', types.Optional(types.String(250))),
('password_hash', types.String(250)), ('is_active', types.Boolean()), ('registration_date', types.DateTime()) ])
Why? Support for different input/output formats keyless transport support for
non-HTTP no hash collision attacks :-) Predictable memory usage
Comes for free Easier to test Helps documenting the public
APIs Catches common errors early Handle errors without invoking code Predictable dictionary ordering
Strict vs Lenient
Rule of Thumb Be strict in what you send, but
generous in what you receive — variant of Postel's Law
Being Generous In order to be generous you need to
know what to receive. Just accepting any input is a security disaster waiting to happen.
Support unsupported types { "foo": [1, 2, 3], "bar": {"key":
"value"}, "now": "Thu, 10 May 2012 14:16:09 GMT" } foo.0=1& foo.1=2& foo.2=3& bar.key=value& now=Thu%2C%2010%20May%202012%2014:16:09%20GMT
Solves the GET issue GET has no body parameters have
to be URL encoded inconsistency with JSON post requests
Where is the streaming?
There is none
there are always two sides to an API
If the server has streaming endpoints — the client will
have to support them as well
For things that need actual streaming we have separate endpoints.
streaming is different
but we can stream until we need buffering
Discard useless stuff { "foo": [list, of, thousands, of, items,
we don't, need], "an_important_key": "we're actually interested in" }
What if I don't make an API?
modern web apps are APIs
Dumb client? Move the client to the server
Q&A
Oh hai. We're hiring http://fireteam.net/careers
Other Decks in Technology
See All in Technology
こんなに変わった! AWSネットワークの常識
shotashiratori
16
11k
目を覚ませ!僕らの認証情報が何者かに侵略されてるぞ
tkito
2
470
OCI Artifact による自動運転タクシーに向けた OTA 基盤の構築
biwashi
1
160
AI-Ready なデータ基盤ってなに? & 新卒未経験が、いま考えていること
imuta
0
960
エンタープライズ事業未経験のデータ屋、 AI時代にN回撃沈する.pdf
ryoskdara_
0
160
AWS Emulator を評価する: 手元で動く AWS の API の仕組み、始め方、確かめ方
inariku
PRO
2
780
Kansai WordPress Meetup@大阪『ブロックテーマを作る上で知っておくべきこと
torounit
0
250
パスワードのない鍵で、 データを手元に置く
ginjihayashi
0
380
Context parameters で作る!はじめての型クラス
kworkdev
PRO
1
160
OpenTelemetryで変わった 新時代のCloudWatchメトリクス監視
tsukuboshi
0
460
業務の傍らで進める改善ループエンジニアリング
bengo4com
1
210
Linuxネットワークスタック進化の軌跡: 過去10年の歩みと最新トレンド
orimanabu
4
760
Featured
See All Featured
Claude Code どこまでも/ Claude Code Everywhere
nwiizo
68
61k
Highjacked: Video Game Concept Design
rkendrick25
PRO
2
580
A Modern Web Designer's Workflow
chriscoyier
699
190k
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
konakalab
0
670
AI Search: Where Are We & What Can We Do About It?
aleyda
0
8.2k
Google's AI Overviews - The New Search
badams
0
1.7k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
3.1k
Dominate Local Search Results - an insider guide to GBP, reviews, and Local SEO
greggifford
PRO
0
400
How to Align SEO within the Product Triangle To Get Buy-In & Support - #RIMC
aleyda
2
1.9k
AI: The stuff that nobody shows you
jnunemaker
PRO
12
1.3k
Navigating Weather and Climate Data
rabernat
0
590
Paper Plane
katiecoart
PRO
6
55k