LDNWebPerf October 2017 - Lucas Pardue

Web browsers are the archetypal HTTP client. They provide myriad features, and are often backed by developer, security and UX teams, to name a few. Browsers are user facing and grab attention, particularly in web performance terms. But what of the non-browsers? The RESTful API clients, voice assistants, and IoT clients of the world. These less glamorous, and often invisible, diverse set of tools and libraries are the workhorses that play a major part in keeping the Internet, the Web and the services running atop all ticking over nicely.

This talk begins with the question "What is a browser?" and considers the answer through the lens of HTTP features and capabilities. Spoiler: the answer is complicated. We'll consider how Browsers play an important part in driving the evolution of the web and whether non-browsers will be left behind

Along the way we'll also look at HTTP clients in the context of market reach, and some case studies arising from Lucas' direct experience.

Transcript

1.  BBC 2017
   Blurred lines: Browsers vs.
   Non-browsers?
   October 2017
   Lucas Pardue, Project Engineer
   

   View Slide

2.  BBC 2017
   #THICKE
   

   View Slide

3.  BBC 2017
   Browser
   Non-browser
   

   View Slide

4.  BBC 2017
   

   View Slide

5.  BBC 2017
   https://www.w3.org/2001/tag/doc/evergreen-web/
   Browsers that do not stay up-to-date place stress on the ecosystem…
   Some products support only a limited subset of the web platform.
   Vendors create these subsetted web runtimes for a number of reasons:
   1. Intentionally
   2. Unintentionally
   

   View Slide

6.  BBC 2017
   The W3C TAG
   doesn’t define what a
   browser is.
   The W3C
   TAG doesn’t
   define what a
   Browser is.
   Sir Tim Berners-Lee, Creator of first web browser
   

   View Slide

7.  BBC 2017
   https://www.w3.org/2001/tag/doc/evergreen-web/
   Vendors must decide whether their products
   can browse the public web. Products that can
   load arbitrary content are “browsers”. Browsers
   must be regularly updated, especially to fix
   security and interoperability bugs — ideally
   with an automatic, secure update mechanism.
   

   View Slide

8.  BBC 2017
   An HTTP “client” is a program that establishes a connection
   to a server for the purpose of sending one or more HTTP
   requests.
   An HTTP “server” is a program that accepts connections in
   order to service HTTP requests by sending HTTP responses.
   The same program might act as a client on some connections
   and a server on others.
   HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.1
   

   View Slide

9.  BBC 2017
   Implementation Diversity
   “People started using HTTP for
   things other than Web browsing
   (so-called “HTTP APIs”), and they
   began using the Web itself in
   new ways, with the advent of
   AJAX and later HTML5.”
   Mark Nottingham (IETF HTTPbis WG Co-chair)
   “RFC2616 is Dead”
   https://www.mnot.net/blog/2014/06/07/rfc2616_is_dead
   

   View Slide

10.  BBC 2017
    The term “user agent” refers to any of the various client
    programs that initiate a request, including (but not limited
    to) browsers, spiders, command-line tools, custom
    applications, and mobile apps.
    When considering the design of HTTP, it is easy to fall into
    a trap of thinking that all user agents are general-purpose
    browsers...
    That is not the case in practice. Common HTTP user agents
    include household appliances, stereos, scales, firmware
    update scripts, command-line programs, mobile apps, and
    communication devices in a multitude of shapes and sizes.
    HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.2
    

    View Slide

11.  BBC 2017
    http://www.bbc.co.uk/rd
    

    View Slide

12.  BBC 2017
    Peter Gasston (rehab),
    State of the Browser 6
    https://speakerdeck.com/stopsatgreen/growing-up-getting-serious-number-sotb6
    

    View Slide

13.  BBC 2017
    Daniel Stenberg (curl)
    Lesser HTTPS for Non-Browsers
    https://daniel.haxx.se/blog/2017/01/10/lesser-https-for-non-browsers/
    

    View Slide

14.  BBC 2017
    A Reference Architecture for Web Browsers, Grosskurth and Godfrey, 2005
    

    View Slide

15.  BBC 2017
    Architecture of Mozilla, Grosskurth and Godfrey, 2005
    

    View Slide

16.  BBC 2017
    Architecture of Lynx, Grosskurth and Godfrey, 2005
    

    View Slide

17.  BBC 2017
    Architecture of Node.js’ Internal Codebase
    Aren Li
    https://medium.com/yet-another-node-js-
    blog/architecture-of-node-js-internal-codebase-
    57cd8376b71f#.4dybaub1m
    

    View Slide

18.  BBC 2017
    Architecture of Node.js’ Internal Codebase
    Aren Li
    https://medium.com/yet-another-node-js-
    blog/architecture-of-node-js-internal-codebase-
    57cd8376b71f#.4dybaub1m
    Javascript
    Interpreter
    Networking
    Node.js + addons
    User
    Interface
    Browser
    Engine?
    XML Parser
    Rendering
    Engine?
    Data Persistence
    Display Backend
    

    View Slide

19.  BBC 2017
    https://www.smashingmagazine.com/2017/03/beyond-
    browser-web-desktop-apps/
    https://docs.google.com/presentation/d/1gqK9F4lGAY3TZud
    AtdcxzMQNEE7PcuQrGu83No3l0lw/edit#slide=id.g14ebf0a
    b58_0_0
    

    View Slide

20.  BBC 2017
    http://www.bbc.co.uk/mediacentre/search/?term=iplayer%20performance%20pack
    

    View Slide

21.  BBC 2017
    Terminology
    • Requests – the number of successful requests to stream or download a
    programme. We only count successful requests, where a stream or a download
    actually starts, rather than “clicks” which can be repeated if the user does not see
    an immediate reaction on the website. Requests are made up of two components:
    • Stream – click to play instantly.
    • Download – save to your device to play later. We report download playback, rather than
    downloads, where possible.
    

    View Slide

22.  BBC 2017
    iPlayer Performance Pack, July 2017
    

    View Slide

23.  BBC 2017
    Requests by device type
    iPlayer Performance Pack, July 2017
    

    View Slide

24.  BBC 2017
    TV Devices,
    49%
    Tablet, 25%
    Mobile, 11%
    Computer,
    15%
    Apples to Oranges
    Tablet,
    12.86%
    Mobile,
    38.40%
    Desktop,
    48.06%
    Console,
    0.68%
    StatCounter Global Stats
    “stats are based on over 15 billion page views per month recorded across more than 2.5
    million websites”
    http://gs.statcounter.com/platform-market-share/all/united-kingdom/#monthly-201707-201707-bar
    iPlayer requests UK Market Share
    

    View Slide

25.  BBC 2017
    TV Devices,
    49%
    Tablet, 26%
    Mobile, 12%
    Computer, 17%
    55.1%
    12.9%
    12.0%
    9.2%
    8.3% 2.6%
    Requests by device type
    http://gs.statcounter.com/browser-market-share/desktop/united-kingdom#monthly-201706-201707
    https://www.statista.com
    /statistics/487815/market
    -share-tablet-device-
    vendors-uk/
    https://www.statista.com/statistics/387227/market-share-of-smartphone-manufacturers-in-the-uk/
    

    View Slide

26.  BBC 2017
    Classes of HTTP clients
    ?
    Media
    Streaming
    Web
    browsers
    (micro) Service
    APIs
    IoT
    ?
    ?
    “Apps”
    (downloads,
    reporting, gaming, etc)
    ?
    Caches
    ?
    Scripts,
    DevOps, etc
    ?
    Assistants
    ? ?
    

    View Slide

27.  BBC 2017
    “Some” characteristics of non-browsers HTTP clients
    • Awareness and display of context
    • No URL bar
    • No “padlock”
    • No DOM
    • Same-origin policy, Cookies
    • Handling errors
    • Exposing the error to the right layer
    • Even Fetch and MSE has this kind of problem
    • Rectifying the error
    • No user to reload page or tell an admin
    • “Limited” devices
    • Monolithic images
    • Device support lifetime and maintenance schedules
    • Sometimes never connected
    • Sometimes contain multiple HTTP clients
    • “Good enough” / “Means to an end”
    • Lack of awareness for considerations outside immediate problem domain
    • Select a client with minimal feature set required to get the job done
    • Security can be a barrier
    

    View Slide

28.  BBC 2017
    “Non-GUI” Features of HTTP clients
    • Methods
    • GET, POST, PUT, custom
    • HTTP authentication
    • Basic, digest, NTLM, Negotiate
    • HTTP Redirect
    • Limiting
    • Caching
    • Cookies
    • Content parsing and recursion
    • JavaScript or JSON support
    • Recover and continue
    • Compression/decompression
    • Content-encoding, transport-encoding
    • IDN, response charset
    • Protocol evolution
    • HTTP/2, QUIC
    • HTTP Alternative Services
    • Happy eyeballs
    • TLS
    • Trusted CAs
    • Certificate transparency
    • HSTS
    • Preload
    • Session resumption
    • Cert revocation
    • Client certificates
    • SNI
    • Extended validation
    • Proxy support
    • HTTP/HTTPS
    • SOCKS
    • OAuth
    • Etc, etc
    Lucas’ low tech “caniuse.com”
    

    View Slide

29.  BBC 2017
    Closing
    “Subsets and legacy browsers can hold the lowest common denominator
    down, preventing the spread of platform-wide progress. This risk is
    magnified by use; more instances of subsetted browsers lead to a more
    heterogeneous web. Runtimes which are not general-purpose browsers
    or do not see much use pose less risk.”
    https://www.w3.org/2001/tag/doc/evergreen-web/
    

    View Slide

30.  BBC 2017
    Thanks!
    October 2017
    Lucas Pardue, Project Engineer
    @SimmerVigor
    @BBCRD
    

    View Slide