$30 off During Our Annual Pro Sale. View Details »

HTTP and Your Angry Dog

HTTP and Your Angry Dog

As presented at Whisky Web, Confoo, PFCongres and Fronteers User Group. An overview of some intermediate level HTTP features and how they might be useful in practice.

Ross Tuck

April 13, 2013
Tweet

More Decks by Ross Tuck

Other Decks in Programming

Transcript

  1. Ross Tuck
    HTTP And
    Your Angry Dog
    April 13th, Whisky Web

    View Slide

  2. Who Am I?

    View Slide

  3. Ross Tuck

    View Slide

  4. Team Lead at Ibuildings
    Codemonkey
    Token Foreigner
    REST nut

    View Slide

  5. @rosstuck

    View Slide

  6. Today's topic:

    View Slide

  7. Dogs

    View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. HTTP & Dogs

    View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. The Agenda

    View Slide

  18. Basics

    View Slide

  19. Client Server
    Request
    Response

    View Slide

  20. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request

    View Slide

  21. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    2 Parts

    View Slide

  22. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    The body

    View Slide

  23. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146




    My application
    ...
    Request
    The body

    View Slide

  24. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    The body

    View Slide

  25. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request

    View Slide

  26. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    The headers

    View Slide

  27. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    The good stuff

    View Slide

  28. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request

    View Slide

  29. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    GET, POST, PUT, DELETE

    View Slide

  30. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    Relative URL
    HTTP version

    View Slide

  31. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request
    Key/Value pairs

    View Slide

  32. POST /gists HTTP/1.1
    Authorization: Basic xxxxxxxx
    Host: api.github.com
    Content-Length: 146
    {
    "description": "the description for this gist",
    "public": false,
    "files": {
    ...
    Request

    View Slide

  33. View Slide

  34. HTTP/1.1 201 Created
    Date: Sun, 09 Sep 2012 11:42:41 GMT
    Content-Length: 1848
    Location: https://api.github.com/gists/a43a0cf58
    {
    "description": "the description for this gist",
    "comments": 0,
    "created_at": "2012-09-09T11:42:40Z",
    ...
    Response
    Status code

    View Slide

  35. Status Codes

    View Slide

  36. View Slide

  37. View Slide

  38. • 2xx
    • 3xx
    • 4xx
    • 5xx
    OK!
    Over there!
    Client screwed up!
    Server screwed up!

    View Slide

  39. Content Negotiation

    View Slide

  40. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  41. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    {
    "cute": true,
    "big": false,
    "data_dog": true
    }
    Response

    View Slide

  42. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT

    true


    Response

    View Slide

  43. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  44. GET /dogs/corgi.json HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  45. /dogs/corgi.json !== /dogs/corgi.xml

    View Slide

  46. Imagine the URL as your primary key.

    View Slide

  47. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  48. GET /dogs/corgi?_format=json HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  49. POST /dogs/corgi?_format=json HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  50. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Request

    View Slide

  51. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json
    Request

    View Slide

  52. More POWAH

    View Slide

  53. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json
    Request

    View Slide

  54. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json, application/xml
    Request
    How do I choose?

    View Slide

  55. View Slide

  56. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json, application/xml
    Request

    View Slide

  57. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Content-Type: application/json
    {
    "cute": true,
    "big": false,
    "data_dog": true
    }
    Response

    View Slide

  58. Nifty.

    View Slide

  59. GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json, application/xml
    Request

    View Slide

  60. text/html, text/plain

    View Slide

  61. text/html;key=value, text/plain

    View Slide

  62. text/html;key=value;foo=bar, text/plain

    View Slide

  63. text/html, text/plain

    View Slide

  64. text/html, text/plain;q=0.5
    Quality
    (Default 1.0)

    View Slide

  65. text/html, text/plain;q=0.5, text/*;q=0.1
    Wildcards

    View Slide

  66. text/html, text/plain;q=0.5, */*;q=0.1
    Anything at all

    View Slide

  67. Accept Headers
    Little weird...

    View Slide

  68. But not so scary.

    View Slide

  69. text/html,application/xhtml+xml,
    application/xml;q=0.9,*/*;q=0.8

    View Slide

  70. Cool...

    View Slide

  71. What the heck is it good for?

    View Slide

  72. Accept is a “Pattern”

    View Slide

  73. Accept-Language
    Accept-Encoding
    Accept-Charset
    Accept-Ranges

    View Slide

  74. Content-Language
    Content-Encoding
    (works differently)
    Content-Range

    View Slide

  75. Resource vs Representation
    /dog/corgi
    JSON,
    Dutch,
    Gzipped,
    /dog/corgi

    View Slide

  76. Resource vs Representation

    View Slide

  77. Best way to version your API.
    Arguably.
    Right now.

    View Slide

  78. /v1/dogs/corgi

    View Slide

  79. Accept: application/vnd.dogipedia-v1+json

    View Slide

  80. Accept: application/vnd.dogipedia-v2+json

    View Slide

  81. View Slide

  82. View Slide

  83. Vary

    View Slide

  84. Client Server
    GET /dogs/corgi HTTP/1.1
    Host: api.example.com

    View Slide

  85. Client Server
    GET /dogs/corgi HTTP/1.1
    Host: api.example.com

    View Slide

  86. Client Server
    Proxy
    GET /dogs/corgi HTTP/1.1
    Host: api.example.com

    View Slide

  87. Client Server
    Proxy
    GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json, text/plain
    User-Species: cat

    View Slide

  88. Same URL.
    Different output.
    WTF should I return?

    View Slide

  89. Client Server
    Proxy
    GET /dogs/corgi HTTP/1.1
    Host: api.example.com
    Accept: application/json, text/plain
    User-Species: cat

    View Slide

  90. Client Server
    Proxy

    View Slide

  91. Here's how.
    Hint:
    Involves the
    Vary header!

    View Slide

  92. Client Server
    Proxy
    /dogs/corgi
    Accept: application/json, text/plain
    User-Species: cat

    View Slide

  93. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Content-Type: application/json
    Vary: Accept
    {“json”: “omgz”}
    Response

    View Slide

  94. Client Server
    Proxy
    URL and Accept?
    Okay, I got this.

    View Slide

  95. Some time later...

    View Slide

  96. Client Server
    Proxy
    /dogs/corgi
    Accept: application/json, text/plain
    User-Species: aardvark

    View Slide

  97. Client Server
    Proxy
    Valid cache.
    I has it.

    View Slide

  98. Client Server
    Proxy
    ZZ Z
    Z
    Z
    Z

    View Slide

  99. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Content-Type: application/json
    Vary: Accept
    {“json”: “omgz”}
    Response

    View Slide

  100. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Content-Type: application/json
    Vary: Accept, User-Species
    {“json”: “omgz”}
    Response

    View Slide

  101. HTTP/1.1 200 OK
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Content-Type: application/json
    Vary: Accept, User-Species
    {“json”: “dogs rule, cats drool”}
    Response

    View Slide

  102. Request headers.
    Not Response!

    View Slide

  103. Bad Reputation?
    2 Reasons

    View Slide

  104. 1. Accept-Encoding
    -Language

    View Slide

  105. 2. Internet Explorer

    View Slide

  106. Caching

    View Slide

  107. Expires
    Pragma
    Cache-Control

    View Slide

  108. Expires
    Pragma
    Cache-Control

    View Slide

  109. Expires
    Cache-Control
    HTTP 1.0
    HTTP 1.1

    View Slide

  110. HTTP/1.1 200 OK
    Expires: Thu, 07 Feb 2013 22:00:00 GMT
    {“herp”: “derp”}
    Response

    View Slide

  111. HTTP/1.1 200 OK
    Cache-Control: max-age=120
    {“herp”: “derp”}
    Response

    View Slide

  112. HTTP/1.1 200 OK
    Cache-Control: max-age=120
    Response

    View Slide

  113. Expires
    Cache-Control
    HTTP 1.0
    HTTP 1.1

    View Slide

  114. HTTP/1.1 200 OK
    Cache-Control: max-age=120
    {“herp”: “derp”}
    Response

    View Slide

  115. HTTP/1.1 200 OK
    Cache-Control: max-age=120, s-maxage=120
    {“herp”: “derp”}
    Response
    Dude, Where's
    my dash?

    View Slide

  116. public
    private
    no-store
    no-cache
    no-transform
    must-revalidate
    proxy-revalidate

    View Slide

  117. Mark Nottingham's Caching Tutorial
    http://www.mnot.net/cache_docs/
    Much better
    than me.

    View Slide

  118. Conditional Requests

    View Slide

  119. Conditional Requests

    View Slide

  120. The Part About ETags

    View Slide

  121. Conditional Requests

    View Slide

  122. DELETE /ross/reputation HTTP/1.1
    Host: api.joind.in
    If-Talk-Quality: Crap
    Request

    View Slide

  123. if ($talkQuality === 'Crap') {
    delete($rossReputation);
    }
    Not real code
    Server

    View Slide

  124. What kind of conditions?

    View Slide

  125. If-Match
    If-None-Match
    If-Modified-Since
    If-Unmodified-Since
    If-Range
    ETags
    Datetimes
    Either

    View Slide

  126. Wait a second, Ross.
    Audience

    View Slide

  127. What the heck is an
    ETag, anyways?

    View Slide

  128. A string.
    Any string.

    View Slide

  129. One rule:

    View Slide

  130. Represent the current state.

    View Slide

  131. “14”
    “a381bedb5d4478053eb04be35f8798dd”
    “winnie-the-pooh”

    View Slide

  132. ...for the current representation.

    View Slide

  133. etag(“v14-json-en”) !== etag(“v14-xml-en”)
    Don't cross the streams
    Server

    View Slide

  134. Last Modified Date sounds easier...
    Audience

    View Slide

  135. Wed, 15 Nov 1995 04:58:08 GMT
    One second
    of precision

    View Slide

  136. Caching With Conditionals

    View Slide

  137. Use Case

    View Slide

  138. GET /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    Request

    View Slide

  139. HTTP/1.1 200 OK
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Vary: Accept
    ETag: "f4e15911542b92b44bb38186e71cc8f5"
    "history": [
    {
    "version": "529f6311d5518977534b6e1fd313...",
    ...
    Response

    View Slide

  140. ...
    "user": {
    "gravatar_id": "c26bfcbd5f786591e036fa0",
    "avatar_url": "https://secure.gravatar...",
    "login": "rosstuck",
    "url": "https://api.github.com/users/rosstuck",
    "id": 146766
    },
    "change_status": {
    "additions": 1,
    "deletions": 0,
    "total": 1
    },
    Response

    View Slide

  141. "url": "https://api.github.com/gists/348...",
    "committed_at": "2012-08-26T17:40:03Z"
    }
    ],
    "git_pull_url": "git://gist.github.com/34819...",
    "forks": [
    ],
    "html_url": "https://gist.github.com/3481910",
    "git_push_url": "[email protected]:3481910.git",
    "comments": 0,
    "user": {
    Response

    View Slide

  142. View Slide

  143. HTTP/1.1 200 OK
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Vary: Accept
    ETag: "f4e15911542b92b44bb38186e71cc8f5"
    {
    "history": [
    {
    "version": "529f6311d5518970903cb5427534b6e1fd313aca",
    "user": {
    "gravatar_id": "c26bfcbd5f786591e036fa0958a11e8b",
    "avatar_url": "https://secure.gravatar.com/avatar/c26bfcbd5f786591e036fa0958a11e8b?d=https://a2...
    "login": "rosstuck",
    "url": "https://api.github.com/users/rosstuck",
    "id": 146766
    },
    "change_status": {
    "additions": 1,
    "deletions": 0,
    "total": 1
    },
    "url": "https://api.github.com/gists/3481910/529f6311d5518970903cb5427534b6e1fd313aca",
    "committed_at": "2012-08-26T17:40:03Z"
    }
    ],
    "git_pull_url": "git://gist.github.com/3481910.git",
    Response
    "forks": [
    ],
    "html_url": "https://gist.github.com/3481910",
    "git_push_url": "[email protected]:3481910.git",
    "comments": 0,
    "user": {
    "gravatar_id": "c26bfcbd5f786591e036fa0958a11e8b",
    "avatar_url": "https://secure.gravatar.com/avatar/c26bfcbd5f78659....",}
    "login": "rosstuck",
    "url": "https://api.github.com/users/rosstuck",
    "id": 146766
    },
    "public": true,
    "created_at": "2012-08-26T17:40:03Z",
    "files": {
    "gistfile1.txt": {
    "type": "text/plain",
    "filename": "gistfile1.txt",
    "raw_url": "https://gist.github.com/raw/3481910/8b6946739e8098408ee3af96...
    "content": "Hello PFC!",
    "language": null,
    "size": 10
    }
    },
    "description": "",
    "url": "https://api.github.com/gists/3481910",
    "updated_at": "2012-08-26T17:40:03Z",
    "id": "3481910"
    }

    View Slide

  144. View Slide

  145. GET /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-None-Match: "f4e15911542b92b44bb38186e71cc8f5"
    Request

    View Slide

  146. HTTP/1.1 304 Not Modified
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Vary: Accept
    ETag: "f4e15911542b92b44bb38186e71cc8f5"
    Response

    View Slide

  147. HTTP/1.1 304 Not Modified
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Vary: Accept
    ETag: "f4e15911542b92b44bb38186e71cc8f5"
    Response

    View Slide

  148. HTTP/1.1 304 Not Modified
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Vary: Accept
    ETag: "f4e15911542b92b44bb38186e71cc8f5"
    Response
    No giant body!

    View Slide

  149. Caching.
    You has it.

    View Slide

  150. GET /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-None-Match: "a381bedb5d4478053eb04be35f8798dd"
    Request

    View Slide

  151. GET /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-None-Match: "ross-is-a-poo-poo-head"
    Request

    View Slide

  152. HTTP/1.1 200 OK
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Vary: Accept
    ETag: "f4e15911542b92b44bb38186e71cc8f5"
    "history": [
    {
    "version": "529f6311d5518977534b6e1fd313...",
    Response

    View Slide

  153. Recap

    View Slide

  154. No ETag
    Old ETag
    Matching ETag
    Full Body
    Full Body
    No Body



    View Slide

  155. ...on supported servers.

    View Slide

  156. Why?

    View Slide

  157. Parsing
    Bandwidth
    Response time
    Probably
    ...Maybe

    View Slide

  158. However...

    View Slide

  159. “The fastest request is one you don't make.”
    - Jesus

    View Slide

  160. More Fun With ETags

    View Slide

  161. Optimistic Concurrency Control
    “Record Versioning”

    View Slide

  162. Request

    View Slide

  163. GET /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-None-Match: "f4e15911542b92b44bb38186e71cc8f5"
    Request

    View Slide

  164. PATCH /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-None-Match: "f4e15911542b92b44bb38186e71cc8f5"
    Request

    View Slide

  165. PATCH /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-Match: "f4e15911542b92b44bb38186e71cc8f5"
    Request

    View Slide

  166. PATCH /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-Match: "f4e15911542b92b44bb38186e71cc8f5"
    { "description": "cheese om nom nom" }
    Request

    View Slide

  167. Response

    View Slide

  168. Response
    HTTP/1.1 200 OK
    Server: nginx/1.0.13
    Date: Sat, 01 Sep 2012 14:01:38 GMT
    ETag: "899b76047a5e68445668374c2e0faa32"
    {
    "description": "cheese om nom nom",
    "user": {
    "login": "rosstuck",
    ...

    View Slide

  169. It works.

    View Slide

  170. So what?

    View Slide

  171. What if I send something...

    View Slide

  172. View Slide

  173. PATCH /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-Match: "899b76047a5e68445668374c2e0faa32"
    { "description": "cheese om nom nom" }
    Request

    View Slide

  174. PATCH /gists/3481910 HTTP/1.1
    Host: api.github.com
    Accept: */*
    If-Match: "stay-puft-marshmellow-dog!"
    { "description": "cheese om nom nom" }
    Request

    View Slide

  175. HTTP/1.1 412 Precondition Failed
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Response

    View Slide

  176. Response

    View Slide

  177. if (“stay-puft-marshmellow-dog” == “f4e1591..”) {
    patchTheRecord();
    }
    Server

    View Slide

  178. if (“stay-puft-marshmellow-dog” == “f4e1591..”) {
    patchTheRecord();
    } else {
    sendScary412Message();
    }
    Server

    View Slide

  179. Your ETag is out of date.

    View Slide

  180. “Two guys on the same record” problem

    View Slide

  181. Other Scary Precondition Errors

    View Slide

  182. Disclaimer
    New Stuff Ahead

    View Slide

  183. DELETE /gists/3481910 HTTP/1.1
    Host: api.github.com
    Request

    View Slide

  184. HTTP/1.1 428 Precondition Required
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Response

    View Slide

  185. Am I operating on the latest version?

    View Slide

  186. DELETE /gists/3481910 HTTP/1.1
    Host: api.github.com
    Request

    View Slide

  187. DELETE /gists/3481910 HTTP/1.1
    Host: api.github.com
    If-Match: "f4e15911542b92b44bb38186e71cc8f5"
    Request

    View Slide

  188. HTTP/1.1 204 No Content
    Server: nginx/1.0.13
    Date: Sun, 26 Aug 2012 18:00:43 GMT
    Response

    View Slide

  189. Look before you leap.

    View Slide

  190. Tooling

    View Slide

  191. View Slide

  192. View Slide

  193. View Slide

  194. Epilogue: HTTP & Dogs

    View Slide

  195. Content Negotiation
    Vary
    Caching
    Preconditions

    View Slide

  196. Treat it like your framework.

    View Slide

  197. View Slide