Upgrade to Pro — share decks privately, control downloads, hide ads and more …

HTTP 1.1 Sequential Requests

HTTP 1.1 Sequential Requests

This represent my journey for sending multiple requests at once when I cannot use modern HTTP capabilities, discovering and rediscovering RFCs. Teaching me that I didn't really need anything new, but to compose with different existing things.

Grégoire Hébert

December 08, 2023
Tweet

More Decks by Grégoire Hébert

Other Decks in Technology

Transcript

  1. Grégoire Hébert Principal Engineer Author of "Memex - La route

    du REST" @gregoirehebert @gheb_dev Les-Tilleuls.coop
  2. Conclusions What is the subject? 02 03 Summary 04 01

    New specification Research for specifications
  3. When might we encounter this situation ? ➔ Submission of

    forms in several passes 1 What is the subject?
  4. When might we encounter this situation ? ➔ Submission of

    forms in several passes ➔ Contact multiple endpoints at the same time 1 What is the subject?
  5. When might we encounter this situation ? ➔ Submission of

    forms in several passes ➔ Contact multiple endpoints at the same time ➔ Exploit a resource right after its creation without waiting for the result to be sent 1 What is the subject?
  6. When might we encounter this situation ? ➔ Submission of

    forms in several stages ➔ Contact multiple endpoints at the same time ➔ Exploit a resource right after its creation without waiting for the result to be sent ➔ Addresses the inability of a server to determine the optimum order of processing for a logical collection of requests, before attempting to process any of the individual requests 1 What is the subject?
  7. When might we encounter this situation ? ➔ Because not

    every company has a modern MACH architecture, with a nice orchestration flow. ➔ Or because in your Kubernetes Cluster your Ingress is a proxy that only deals with port 80 from within 1 What is the subject?
  8. What does respecting HTTP entail? ➔ Being stateless ➔ Being

    resource oriented 1 What is the subject?
  9. Resolutions thanks to the evolutions of the HTTP protocol. ➔

    With HTTP/1.1 ◆ using pipelining ➔ With HTTP/2 ◆ using multiplexing 1 What is the subject?
  10. What are the limitations? ➔ Using HTTP/1.1 ◆ with pipelining

    ➔ Using HTTP/2 ◆ with multiplexing 1 What is the subject?
  11. What are the limitations? ➔ Using HTTP/1.1 ◆ with pipelining

    • HOL, Hop by Hop, disabled by default • Idempotent only ➔ Using HTTP/2 ◆ with multiplexing • Order is not guaranteed 1 What is the subject?
  12. Moreover, what if we cannot use HTTP/2? How much of

    my X followers still uses HTTP/1.1? 1 What is the subject?
  13. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html ➔ https://www.odata.org/documentation/#stq=batch&stp=1 ◆ http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_BatchRequests ◆ http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_Preferencecontinueon

    errorodatacontin ◆ http://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part1-protocol.html#sec_Preferencerespondas ync ➔ https://developers.google.com/gmail/api/guides/batch ➔ https://developers.google.com/people/v1/batch ➔ https://cloud.google.com/storage/docs/batch 1 2 Research for specification
  14. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ Send everything to one endpoint ➔ Use

    a new multipart/http Content-Type header 1 2 Research for specification
  15. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ Send everything to one endpoint ➔ Use

    a new multipart/http Content-Type header ➔ Each subquery is described in the payload, separated by a delimiting string 1 2 Research for specification
  16. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ Send everything to one endpoint ➔ Use

    a new multipart/http Content-Type header ➔ Each subquery is described in the payload, separated by a delimiting string ➔ Each subquery is identified by a Content-ID to associate its response 1 2 Research for specification
  17. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ Send everything to one endpoint ➔ Use

    a new multipart/http Content-Type header ➔ Each subquery is described in the payload, separated by a delimiting string ➔ Each subquery is identified by a Content-ID to associate its response ➔ We cannot assume the execution order of each subquery 1 2 Research for specification
  18. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ Send everything to one endpoint ➔ Use

    a new multipart/http Content-Type header ➔ Each subquery is described in the payload, separated by a delimiting string ➔ Each subquery is identified by a Content-ID to associate its response ➔ We cannot assume the execution order of each subquery ➔ The main query cannot be cached, but the subqueries can be 1 2 Research for specification
  19. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 ➔ Send everything to one endpoint ➔ Use

    a new multipart/http Content-Type header ➔ Each subquery is described in the payload, separated by a delimiting string ➔ Each subquery is identified by a Content-ID to associate its response ➔ We cannot assume the execution order of each subquery ➔ The main query cannot be cached, but the subqueries can be ➔ The queries are encapsulated by a message/http or an application/http 1 2 Research for specification
  20. ➔ https://datatracker.ietf.org/doc/html/draft-snell-http-batch-01 1 2 Research for specification ➔ Send everything

    to one endpoint ➔ Use a new multipart/http Content-Type header ➔ Each subquery is described in the payload, separated by a delimiting string ➔ Each subquery is identified by a Content-ID to associate its response ➔ We cannot assume the execution order of each subquery ➔ The main query cannot be cached, but the subqueries can be ➔ The queries are encapsulated by a message/http or an application/http
  21. ➔ Multipart ➔ multipart/mixed (Can contain anything) ➔ multipart/alternative (For

    emails Usually) ➔ multipart/digest (Like alternative, but usually messages from multiple people) ➔ multipart/parallel (Intended to be processed in parallel if the software can) ➔ A sub message can be Multipart itself 1 2 Research for specification
  22. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. 1 2 Research for specification
  23. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. ➔ Values of a response body in the query part of the url or in the body of subsequent queries. 1 2 Research for specification
  24. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. ➔ Values of a response body in the query part of the url or in the body of subsequent queries. ➔ Individual processing in the order of reception. 1 2 Research for specification
  25. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. ➔ Values of a response body in the query part of the url or in the body of subsequent queries. ➔ Individual processing in the order of reception. ➔ Stop at the first error, unless the continue-on-error preference is specified. 1 2 Research for specification
  26. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. ➔ Values of a response body in the query part of the url or in the body of subsequent queries. ➔ Individual processing in the order of reception. ➔ Stop at the first error, unless the continue-on-error preference is specified. ➔ Apply all, or nothing. 1 2 Research for specification
  27. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. ➔ Values of a response body in the query part of the url or in the body of subsequent queries. ➔ Individual processing in the order of reception. ➔ Stop at the first error, unless the continue-on-error preference is specified. ➔ Apply all, or nothing. ➔ The queries are encapsulated by a message/http or an application/http 1 2 Research for specification
  28. ➔ Odata ➔ Dependencies between queries ➔ References in the

    URL of subsequent queries. ➔ Values of a response body in the query part of the url or in the body of subsequent queries. ➔ Individual processing in the order of reception. ➔ Stop at the first error, unless the continue-on-error preference is specified. ➔ Apply all, or nothing. ➔ The queries are encapsulated by a message/http or an application/http ➔ Odata has been standardized by OASIS and approved as an international ISO/IEC standard. 1 2 Research for specification
  29. 1 2 3 New specification ➔ Must use a Multipart

    Message (Mixed or Parallel) How to send multiple messages?
  30. 1 2 3 New specification ➔ verb ➔ path ➔

    headers ➔ and body In addition to the header fields that begin with "Content-" each sub-request MUST redefine the start-line with its own How does a sub-message defines itself?
  31. 1 2 3 New specification Each individual HTTP request and

    response message MUST be encapsulated using the application/http Content-Type RFC 9112 How to detect it is a not just data?
  32. 1 2 3 New specification Each part MUST specify a

    Content-ID header specifying a reference identifier for the HTTP request message. RFC 2392 How to associate requests and responses?
  33. 1 2 3 New specification Client applications MUST NOT assume

    such ordering of responses and MUST use the Content-ID headers to correlate HTTP request and response messages in the Batch Requests and Batch Responses. RFC 2392
  34. Client applications MUST NOT assume such ordering of responses and

    MUST use the Content-ID headers to correlate HTTP request and response messages in the Batch Requests and Batch Responses. Even if some or all of the contained requests fail individually, the server SHOULD respond with a status of either 200 OK, 202 Accepted or eventually 207 Multi-Status. 1 2 3 New specification draft-snell-http-batch-01
  35. Client applications MUST NOT assume such ordering of responses and

    MUST use the Content-ID headers to correlate HTTP request and response messages in the Batch Requests and Batch Responses. Even if some or all of the contained requests fail individually, the server SHOULD respond with a status of either 200 OK, 202 Accepted or eventually 207 Multi-Status. 1 2 3 New specification This is also possible for sub-responses
  36. 1 2 3 New specification With this 202 status, the

    response SHOULD include an indication of the requests current status and either include a Location header to a status monitor or some estimate of when the user can expect the request to be fulfilled with an optional Retry-After header indicating the time the client should wait before querying the service for status. RFC 9110 How do you know it's ready with a 202?
  37. 1 2 3 New specification The HTTP request MUST only

    contain the path portion of the URL. Restricting the sub-request to lie within the service domain, mitigating cross-origin security issues and allowing to forward and dispatch the request locally. Security Considerations
  38. 1 2 3 New specification Each body part that represents

    a single request MUST NOT include authentication or authorization related HTTP headers, Expect, From, Max-Forwards, Range, or TE headers. If you provide an Authorization header for the outer request, then that header applies to all of the individual calls. Security Considerations (controvertial, see Odata spec vs James Snell's Draft)
  39. 1 2 3 New specification Each body part that represents

    a single request MUST NOT include authentication or authorization related HTTP headers, Expect, From, Max-Forwards, Range, or TE headers. If you provide an Authorization header for the outer request, then that header applies to all of the individual calls. When the server receives the batch request, it applies the outer request query parameters and headers (as appropriate) to each part, and then treats each part as if it was a separate HTTP request.
  40. 4 Conclusions ➔ I really thought I'd add my bit

    to the mix ➔ I put my ego away and learned that: ➔ You don't always need something new ➔ To better use the platform ➔ To better use the standards ➔ To study and be a critic