Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LDNWebPerf October 2017 - Lucas Pardue

LDNWebPerf October 2017 - Lucas Pardue

Web browsers are the archetypal HTTP client. They provide myriad features, and are often backed by developer, security and UX teams, to name a few. Browsers are user facing and grab attention, particularly in web performance terms. But what of the non-browsers? The RESTful API clients, voice assistants, and IoT clients of the world. These less glamorous, and often invisible, diverse set of tools and libraries are the workhorses that play a major part in keeping the Internet, the Web and the services running atop all ticking over nicely.

This talk begins with the question "What is a browser?" and considers the answer through the lens of HTTP features and capabilities. Spoiler: the answer is complicated. We'll consider how Browsers play an important part in driving the evolution of the web and whether non-browsers will be left behind

Along the way we'll also look at HTTP clients in the context of market reach, and some case studies arising from Lucas' direct experience.

London Web Performance Group

October 26, 2017

More Decks by London Web Performance Group

Other Decks in Technology


  1.  BBC 2017 https://www.w3.org/2001/tag/doc/evergreen-web/ Browsers that do not stay up-to-date

    place stress on the ecosystem… Some products support only a limited subset of the web platform. Vendors create these subsetted web runtimes for a number of reasons: 1. Intentionally 2. Unintentionally
  2.  BBC 2017 The W3C TAG doesn’t define what a

    browser is. The W3C TAG doesn’t define what a Browser is. Sir Tim Berners-Lee, Creator of first web browser
  3.  BBC 2017 https://www.w3.org/2001/tag/doc/evergreen-web/ Vendors must decide whether their products

    can browse the public web. Products that can load arbitrary content are “browsers”. Browsers must be regularly updated, especially to fix security and interoperability bugs — ideally with an automatic, secure update mechanism.
  4.  BBC 2017 An HTTP “client” is a program that

    establishes a connection to a server for the purpose of sending one or more HTTP requests. An HTTP “server” is a program that accepts connections in order to service HTTP requests by sending HTTP responses. The same program might act as a client on some connections and a server on others. HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.1
  5.  BBC 2017 Implementation Diversity “People started using HTTP for

    things other than Web browsing (so-called “HTTP APIs”), and they began using the Web itself in new ways, with the advent of AJAX and later HTML5.” Mark Nottingham (IETF HTTPbis WG Co-chair) “RFC2616 is Dead” https://www.mnot.net/blog/2014/06/07/rfc2616_is_dead
  6.  BBC 2017 The term “user agent” refers to any

    of the various client programs that initiate a request, including (but not limited to) browsers, spiders, command-line tools, custom applications, and mobile apps. When considering the design of HTTP, it is easy to fall into a trap of thinking that all user agents are general-purpose browsers... That is not the case in practice. Common HTTP user agents include household appliances, stereos, scales, firmware update scripts, command-line programs, mobile apps, and communication devices in a multitude of shapes and sizes. HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.2
  7.  BBC 2017 Peter Gasston (rehab), State of the Browser

    6 https://speakerdeck.com/stopsatgreen/growing-up-getting-serious-number-sotb6
  8.  BBC 2017 Daniel Stenberg (curl) Lesser HTTPS for Non-Browsers

  9.  BBC 2017 Architecture of Node.js’ Internal Codebase Aren Li

    https://medium.com/yet-another-node-js- blog/architecture-of-node-js-internal-codebase- 57cd8376b71f#.4dybaub1m
  10.  BBC 2017 Architecture of Node.js’ Internal Codebase Aren Li

    https://medium.com/yet-another-node-js- blog/architecture-of-node-js-internal-codebase- 57cd8376b71f#.4dybaub1m Javascript Interpreter Networking Node.js + addons User Interface Browser Engine? XML Parser Rendering Engine? Data Persistence Display Backend
  11.  BBC 2017 Terminology • Requests – the number of

    successful requests to stream or download a programme. We only count successful requests, where a stream or a download actually starts, rather than “clicks” which can be repeated if the user does not see an immediate reaction on the website. Requests are made up of two components: • Stream – click to play instantly. • Download – save to your device to play later. We report download playback, rather than downloads, where possible.
  12.  BBC 2017 TV Devices, 49% Tablet, 25% Mobile, 11%

    Computer, 15% Apples to Oranges Tablet, 12.86% Mobile, 38.40% Desktop, 48.06% Console, 0.68% StatCounter Global Stats “stats are based on over 15 billion page views per month recorded across more than 2.5 million websites” http://gs.statcounter.com/platform-market-share/all/united-kingdom/#monthly-201707-201707-bar iPlayer requests UK Market Share
  13.  BBC 2017 TV Devices, 49% Tablet, 26% Mobile, 12%

    Computer, 17% 55.1% 12.9% 12.0% 9.2% 8.3% 2.6% Requests by device type http://gs.statcounter.com/browser-market-share/desktop/united-kingdom#monthly-201706-201707 https://www.statista.com /statistics/487815/market -share-tablet-device- vendors-uk/ https://www.statista.com/statistics/387227/market-share-of-smartphone-manufacturers-in-the-uk/
  14.  BBC 2017 Classes of HTTP clients ? Media Streaming

    Web browsers (micro) Service APIs IoT ? ? “Apps” (downloads, reporting, gaming, etc) ? Caches ? Scripts, DevOps, etc ? Assistants ? ?
  15.  BBC 2017 “Some” characteristics of non-browsers HTTP clients •

    Awareness and display of context • No URL bar • No “padlock” • No DOM • Same-origin policy, Cookies • Handling errors • Exposing the error to the right layer • Even Fetch and MSE has this kind of problem • Rectifying the error • No user to reload page or tell an admin • “Limited” devices • Monolithic images • Device support lifetime and maintenance schedules • Sometimes never connected • Sometimes contain multiple HTTP clients • “Good enough” / “Means to an end” • Lack of awareness for considerations outside immediate problem domain • Select a client with minimal feature set required to get the job done • Security can be a barrier
  16.  BBC 2017 “Non-GUI” Features of HTTP clients • Methods

    • GET, POST, PUT, custom • HTTP authentication • Basic, digest, NTLM, Negotiate • HTTP Redirect • Limiting • Caching • Cookies • Content parsing and recursion • JavaScript or JSON support • Recover and continue • Compression/decompression • Content-encoding, transport-encoding • IDN, response charset • Protocol evolution • HTTP/2, QUIC • HTTP Alternative Services • Happy eyeballs • TLS • Trusted CAs • Certificate transparency • HSTS • Preload • Session resumption • Cert revocation • Client certificates • SNI • Extended validation • Proxy support • HTTP/HTTPS • SOCKS • OAuth • Etc, etc Lucas’ low tech “caniuse.com”
  17.  BBC 2017 Closing “Subsets and legacy browsers can hold

    the lowest common denominator down, preventing the spread of platform-wide progress. This risk is magnified by use; more instances of subsetted browsers lead to a more heterogeneous web. Runtimes which are not general-purpose browsers or do not see much use pose less risk.” https://www.w3.org/2001/tag/doc/evergreen-web/