LDNWebPerf October 2017 - Lucas Pardue

Web browsers are the archetypal HTTP client. They provide myriad features, and are often backed by developer, security and UX teams, to name a few. Browsers are user facing and grab attention, particularly in web performance terms. But what of the non-browsers? The RESTful API clients, voice assistants, and IoT clients of the world. These less glamorous, and often invisible, diverse set of tools and libraries are the workhorses that play a major part in keeping the Internet, the Web and the services running atop all ticking over nicely.

This talk begins with the question "What is a browser?" and considers the answer through the lens of HTTP features and capabilities. Spoiler: the answer is complicated. We'll consider how Browsers play an important part in driving the evolution of the web and whether non-browsers will be left behind

Along the way we'll also look at HTTP clients in the context of market reach, and some case studies arising from Lucas' direct experience.

Transcript

1.  BBC 2017 Blurred lines: Browsers vs. Non-browsers? October 2017 Lucas Pardue, Project Engineer
2.  BBC 2017 #THICKE
3.  BBC 2017 Browser Non-browser
4.  BBC 2017
5.  BBC 2017 https://www.w3.org/2001/tag/doc/evergreen-web/ Browsers that do not stay up-to-date place stress on the ecosystem… Some products support only a limited subset of the web platform. Vendors create these subsetted web runtimes for a number of reasons: 1. Intentionally 2. Unintentionally
6.  BBC 2017 The W3C TAG doesn’t define what a browser is. The W3C TAG doesn’t define what a Browser is. Sir Tim Berners-Lee, Creator of first web browser
7.  BBC 2017 https://www.w3.org/2001/tag/doc/evergreen-web/ Vendors must decide whether their products can browse the public web. Products that can load arbitrary content are “browsers”. Browsers must be regularly updated, especially to fix security and interoperability bugs — ideally with an automatic, secure update mechanism.
8.  BBC 2017 An HTTP “client” is a program that establishes a connection to a server for the purpose of sending one or more HTTP requests. An HTTP “server” is a program that accepts connections in order to service HTTP requests by sending HTTP responses. The same program might act as a client on some connections and a server on others. HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.1
9.  BBC 2017 Implementation Diversity “People started using HTTP for things other than Web browsing (so-called “HTTP APIs”), and they began using the Web itself in new ways, with the advent of AJAX and later HTML5.” Mark Nottingham (IETF HTTPbis WG Co-chair) “RFC2616 is Dead” https://www.mnot.net/blog/2014/06/07/rfc2616_is_dead
10.  BBC 2017 The term “user agent” refers to any of the various client programs that initiate a request, including (but not limited to) browsers, spiders, command-line tools, custom applications, and mobile apps. When considering the design of HTTP, it is easy to fall into a trap of thinking that all user agents are general-purpose browsers... That is not the case in practice. Common HTTP user agents include household appliances, stereos, scales, firmware update scripts, command-line programs, mobile apps, and communication devices in a multitude of shapes and sizes. HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.2
11.  BBC 2017 http://www.bbc.co.uk/rd
12.  BBC 2017 Peter Gasston (rehab), State of the Browser 6 https://speakerdeck.com/stopsatgreen/growing-up-getting-serious-number-sotb6
13.  BBC 2017 Daniel Stenberg (curl) Lesser HTTPS for Non-Browsers https://daniel.haxx.se/blog/2017/01/10/lesser-https-for-non-browsers/
14.  BBC 2017 A Reference Architecture for Web Browsers, Grosskurth and Godfrey, 2005
15.  BBC 2017 Architecture of Mozilla, Grosskurth and Godfrey, 2005
16.  BBC 2017 Architecture of Lynx, Grosskurth and Godfrey, 2005
17.  BBC 2017 Architecture of Node.js’ Internal Codebase Aren Li https://medium.com/yet-another-node-js- blog/architecture-of-node-js-internal-codebase- 57cd8376b71f#.4dybaub1m
18.  BBC 2017 Architecture of Node.js’ Internal Codebase Aren Li https://medium.com/yet-another-node-js- blog/architecture-of-node-js-internal-codebase- 57cd8376b71f#.4dybaub1m Javascript Interpreter Networking Node.js + addons User Interface Browser Engine? XML Parser Rendering Engine? Data Persistence Display Backend
19.  BBC 2017 https://www.smashingmagazine.com/2017/03/beyond- browser-web-desktop-apps/ https://docs.google.com/presentation/d/1gqK9F4lGAY3TZud AtdcxzMQNEE7PcuQrGu83No3l0lw/edit#slide=id.g14ebf0a b58_0_0
20.  BBC 2017 http://www.bbc.co.uk/mediacentre/search/?term=iplayer%20performance%20pack
21.  BBC 2017 Terminology • Requests – the number of successful requests to stream or download a programme. We only count successful requests, where a stream or a download actually starts, rather than “clicks” which can be repeated if the user does not see an immediate reaction on the website. Requests are made up of two components: • Stream – click to play instantly. • Download – save to your device to play later. We report download playback, rather than downloads, where possible.
22.  BBC 2017 iPlayer Performance Pack, July 2017
23.  BBC 2017 Requests by device type iPlayer Performance Pack, July 2017
24.  BBC 2017 TV Devices, 49% Tablet, 25% Mobile, 11% Computer, 15% Apples to Oranges Tablet, 12.86% Mobile, 38.40% Desktop, 48.06% Console, 0.68% StatCounter Global Stats “stats are based on over 15 billion page views per month recorded across more than 2.5 million websites” http://gs.statcounter.com/platform-market-share/all/united-kingdom/#monthly-201707-201707-bar iPlayer requests UK Market Share
25.  BBC 2017 TV Devices, 49% Tablet, 26% Mobile, 12% Computer, 17% 55.1% 12.9% 12.0% 9.2% 8.3% 2.6% Requests by device type http://gs.statcounter.com/browser-market-share/desktop/united-kingdom#monthly-201706-201707 https://www.statista.com /statistics/487815/market -share-tablet-device- vendors-uk/ https://www.statista.com/statistics/387227/market-share-of-smartphone-manufacturers-in-the-uk/
26.  BBC 2017 Classes of HTTP clients ? Media Streaming Web browsers (micro) Service APIs IoT ? ? “Apps” (downloads, reporting, gaming, etc) ? Caches ? Scripts, DevOps, etc ? Assistants ? ?
27.  BBC 2017 “Some” characteristics of non-browsers HTTP clients • Awareness and display of context • No URL bar • No “padlock” • No DOM • Same-origin policy, Cookies • Handling errors • Exposing the error to the right layer • Even Fetch and MSE has this kind of problem • Rectifying the error • No user to reload page or tell an admin • “Limited” devices • Monolithic images • Device support lifetime and maintenance schedules • Sometimes never connected • Sometimes contain multiple HTTP clients • “Good enough” / “Means to an end” • Lack of awareness for considerations outside immediate problem domain • Select a client with minimal feature set required to get the job done • Security can be a barrier
28.  BBC 2017 “Non-GUI” Features of HTTP clients • Methods • GET, POST, PUT, custom • HTTP authentication • Basic, digest, NTLM, Negotiate • HTTP Redirect • Limiting • Caching • Cookies • Content parsing and recursion • JavaScript or JSON support • Recover and continue • Compression/decompression • Content-encoding, transport-encoding • IDN, response charset • Protocol evolution • HTTP/2, QUIC • HTTP Alternative Services • Happy eyeballs • TLS • Trusted CAs • Certificate transparency • HSTS • Preload • Session resumption • Cert revocation • Client certificates • SNI • Extended validation • Proxy support • HTTP/HTTPS • SOCKS • OAuth • Etc, etc Lucas’ low tech “caniuse.com”
29.  BBC 2017 Closing “Subsets and legacy browsers can hold the lowest common denominator down, preventing the spread of platform-wide progress. This risk is magnified by use; more instances of subsetted browsers lead to a more heterogeneous web. Runtimes which are not general-purpose browsers or do not see much use pose less risk.” https://www.w3.org/2001/tag/doc/evergreen-web/
30.  BBC 2017 Thanks! October 2017 Lucas Pardue, Project Engineer @SimmerVigor @BBCRD