~/ on the Range

What do download accelerators, streaming video, and the HTML 5 `` element have in common? They all rely on range requests to transmit data efficiently. Range requests are HTTP's brilliantly simple answer to the question, "How do I download just a portion of a given file?" Although they're supported by most web frameworks, Node doesn't handle range requests out of the box. To that end, we'll be going down to the bare metal as we examine the relevant portions of the HTTP/1.1 spec, learn how to handle such requests correctly, and discuss why they're so useful.

Transcript

1. ~/ on the Range
   

   View Slide

2. HTTP/1.1
   “Physics for Physics Majors”
   

   View Slide

3. Client
   Server
   Proxy
   Gateway
   Tunnel
   Request
   Response
   

   View Slide

4. // HTTP server.
   var http = require("http")
   http.Server(function (request, response) {
   response.writeHead(200, {
   "Content-Type": "text/plain",
   "Content-Length": 13
   })
   response.end("Hello, world.")
   }).listen(1900)
   

   View Slide

5. // TCP client (`telnet`).
   var net = require("net")
   net.connect(1900, function () {
   this.write("GET / HTTP/1.1\r\n")
   this.end("\r\n")
   }).pipe(process.stdout)
   

   View Slide

6. HTTP/1.1 200 OK
   Content-Type: text/plain
   Content-Length: 13
   Date: Tue, 16 Oct 2012 00:20:00 GMT
   Connection: keep-alive
   Hello, world.
   Status Line
   Headers
   Body
   

   View Slide

7. 100
   200
   300
   400
   500
   Information
   Request acknowledged
   Success
   Received, understood, and accepted
   Redirection
   Client intervention
   Client Error
   Bad request
   Server Error
   Unfulfillable request
   

   View Slide

8. General
   Request
   Response
   Entity
   Cache-Control, Connection, Date, Pragma, Trailer,
   Transfer-Encoding, Upgrade, Via, Warning
   Accept, Accept-Charset, Accept-Encoding, Accept-Language,
   Authorization, Expect, From, Host, If-Match, If-Modified-
   Since, If-None-Match, If-Range, If-Unmodified-Since, Max-
   Forwards, Proxy-Authorization, Range, Referer, TE, User-Agent
   Accept-Ranges, Age, ETag, Location, Proxy-
   Authenticate, Retry-After, Server, Vary, WWW-
   Authenticate
   Allow, Content-Encoding, Content-Language, Content-
   Length, Content-Location, Content-MD5, Content-Range,
   Content-Type, Expires, Last-Modified
   

   View Slide

9. Encoding
   

   View Slide

10. HTTP/1.1 200 OK
    Content-Type: text/plain
    Date: Tue, 16 Oct 2012 00:20:00 GMT
    Connection: keep-alive
    Transfer-Encoding: chunked
    d
    Hello, world.
    0
    Hex Size
    Indicators
    parseInt("d", 16) == 13
    

    View Slide

11. Content-Type
    Content-Encoding
    The media type of the entity
    gzip compression; requires decoding
    

    View Slide

12. Methods
    

    View Slide

13. GET
    HEAD
    POST
    PUT
    DELETE
    OPTIONS
    TRACE
    CONNECT
    

    View Slide

14. Ranges
    

    View Slide

15. Request Response
    Range, If-Range
    Accept-Ranges: bytes
    Range, If-Range
    Content-Range
    Range, If-Range
    Content-Length
    Content-Range
    Content-Length
    Content-Range
    Date
    Content-Range
    ETag, Content-Location,
    Expires, Cache-Control, Vary
    

    View Slide

16. First 500 bytes
    Next 500 bytes
    Last 500 bytes
    Last 500 bytes
    First and last bytes only
    bytes=0-499
    bytes=500-999
    bytes=-500
    bytes=9500-
    bytes=0-0,-1
    npm install range-parser
    

    View Slide

17. First 500 bytes
    Next 500 bytes
    All except for first 500 bytes
    Last 500 bytes
    Requested range not satisfiable
    bytes 0-499/1234
    bytes 500-999/1234
    bytes 500-1233/1234
    bytes 734-1233/1234
    *
    

    View Slide

18. Non-Contiguous
    Ranges
    

    View Slide

19. HTTP/1.1 206 Partial Content
    Date: Tue, 16 Oct 2012 00:20:00 GMT
    Accept-Ranges: bytes
    Content-Type: multipart/byteranges; boundary=BOUNDARY
    --BOUNDARY
    ...
    --BOUNDARY
    ...
    --BOUNDARY--
    

    View Slide

20. Content-Type: text/plain
    Content-Range: bytes 1602-1745/422279
    The Hypertext Transfer Protocol (HTTP) is an application-level
    protocol for distributed, collaborative, hypermedia information
    systems.
    Content-Type: text/plain
    Content-Range: bytes 193409-193586/422279
    be displayed when this actually happens. The indication need not be a
    dialog box; it could be an icon (for example, a picture of a rotting
    fish) or some other indicator.
    

    View Slide

21. Partial PUT
    Requests
    

    View Slide

22. var contentRange, pattern, match
    if (request.method == "PUT") {
    contentRange = request.headers["content-range"]
    pattern = /^bytes ((?:\d+-\d+)|\*)\/(\d+|\*)$/
    if ((match = pattern.exec(contentRange))) {
    request.pipe(fs.createWriteStream("target", {
    "start": +match[1].split("-")[0] || 0,
    "flags": "r+"
    }))
    }
    }
    

    View Slide

23. Use Cases
    

    View Slide

24. Resuming interrupted downloads
    Efficiently retrieving large media files
    Download accelerators
    Granular caching
    

    View Slide

25. Thank you!
    @kitcambridge
    git.io/kit
    

    View Slide