Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LDNWebPerf October 2017 - Lucas Pardue

LDNWebPerf October 2017 - Lucas Pardue

Web browsers are the archetypal HTTP client. They provide myriad features, and are often backed by developer, security and UX teams, to name a few. Browsers are user facing and grab attention, particularly in web performance terms. But what of the non-browsers? The RESTful API clients, voice assistants, and IoT clients of the world. These less glamorous, and often invisible, diverse set of tools and libraries are the workhorses that play a major part in keeping the Internet, the Web and the services running atop all ticking over nicely.

This talk begins with the question "What is a browser?" and considers the answer through the lens of HTTP features and capabilities. Spoiler: the answer is complicated. We'll consider how Browsers play an important part in driving the evolution of the web and whether non-browsers will be left behind

Along the way we'll also look at HTTP clients in the context of market reach, and some case studies arising from Lucas' direct experience.

London Web Performance Group

October 26, 2017
Tweet

More Decks by London Web Performance Group

Other Decks in Technology

Transcript

  1.  BBC 2017
    Blurred lines: Browsers vs.
    Non-browsers?
    October 2017
    Lucas Pardue, Project Engineer

    View Slide

  2.  BBC 2017
    #THICKE

    View Slide

  3.  BBC 2017
    Browser
    Non-browser

    View Slide

  4.  BBC 2017

    View Slide

  5.  BBC 2017
    https://www.w3.org/2001/tag/doc/evergreen-web/
    Browsers that do not stay up-to-date place stress on the ecosystem…
    Some products support only a limited subset of the web platform.
    Vendors create these subsetted web runtimes for a number of reasons:
    1. Intentionally
    2. Unintentionally

    View Slide

  6.  BBC 2017
    The W3C TAG
    doesn’t define what a
    browser is.
    The W3C
    TAG doesn’t
    define what a
    Browser is.
    Sir Tim Berners-Lee, Creator of first web browser

    View Slide

  7.  BBC 2017
    https://www.w3.org/2001/tag/doc/evergreen-web/
    Vendors must decide whether their products
    can browse the public web. Products that can
    load arbitrary content are “browsers”. Browsers
    must be regularly updated, especially to fix
    security and interoperability bugs — ideally
    with an automatic, secure update mechanism.

    View Slide

  8.  BBC 2017
    An HTTP “client” is a program that establishes a connection
    to a server for the purpose of sending one or more HTTP
    requests.
    An HTTP “server” is a program that accepts connections in
    order to service HTTP requests by sending HTTP responses.
    The same program might act as a client on some connections
    and a server on others.
    HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.1

    View Slide

  9.  BBC 2017
    Implementation Diversity
    “People started using HTTP for
    things other than Web browsing
    (so-called “HTTP APIs”), and they
    began using the Web itself in
    new ways, with the advent of
    AJAX and later HTML5.”
    Mark Nottingham (IETF HTTPbis WG Co-chair)
    “RFC2616 is Dead”
    https://www.mnot.net/blog/2014/06/07/rfc2616_is_dead

    View Slide

  10.  BBC 2017
    The term “user agent” refers to any of the various client
    programs that initiate a request, including (but not limited
    to) browsers, spiders, command-line tools, custom
    applications, and mobile apps.
    When considering the design of HTTP, it is easy to fall into
    a trap of thinking that all user agents are general-purpose
    browsers...
    That is not the case in practice. Common HTTP user agents
    include household appliances, stereos, scales, firmware
    update scripts, command-line programs, mobile apps, and
    communication devices in a multitude of shapes and sizes.
    HTTP/1.1 Message Syntax and Routing, RFC 7230, Section 2.2

    View Slide

  11.  BBC 2017
    http://www.bbc.co.uk/rd

    View Slide

  12.  BBC 2017
    Peter Gasston (rehab),
    State of the Browser 6
    https://speakerdeck.com/stopsatgreen/growing-up-getting-serious-number-sotb6

    View Slide

  13.  BBC 2017
    Daniel Stenberg (curl)
    Lesser HTTPS for Non-Browsers
    https://daniel.haxx.se/blog/2017/01/10/lesser-https-for-non-browsers/

    View Slide

  14.  BBC 2017
    A Reference Architecture for Web Browsers, Grosskurth and Godfrey, 2005

    View Slide

  15.  BBC 2017
    Architecture of Mozilla, Grosskurth and Godfrey, 2005

    View Slide

  16.  BBC 2017
    Architecture of Lynx, Grosskurth and Godfrey, 2005

    View Slide

  17.  BBC 2017
    Architecture of Node.js’ Internal Codebase
    Aren Li
    https://medium.com/yet-another-node-js-
    blog/architecture-of-node-js-internal-codebase-
    57cd8376b71f#.4dybaub1m

    View Slide

  18.  BBC 2017
    Architecture of Node.js’ Internal Codebase
    Aren Li
    https://medium.com/yet-another-node-js-
    blog/architecture-of-node-js-internal-codebase-
    57cd8376b71f#.4dybaub1m
    Javascript
    Interpreter
    Networking
    Node.js + addons
    User
    Interface
    Browser
    Engine?
    XML Parser
    Rendering
    Engine?
    Data Persistence
    Display Backend

    View Slide

  19.  BBC 2017
    https://www.smashingmagazine.com/2017/03/beyond-
    browser-web-desktop-apps/
    https://docs.google.com/presentation/d/1gqK9F4lGAY3TZud
    AtdcxzMQNEE7PcuQrGu83No3l0lw/edit#slide=id.g14ebf0a
    b58_0_0

    View Slide

  20.  BBC 2017
    http://www.bbc.co.uk/mediacentre/search/?term=iplayer%20performance%20pack

    View Slide

  21.  BBC 2017
    Terminology
    • Requests – the number of successful requests to stream or download a
    programme. We only count successful requests, where a stream or a download
    actually starts, rather than “clicks” which can be repeated if the user does not see
    an immediate reaction on the website. Requests are made up of two components:
    • Stream – click to play instantly.
    • Download – save to your device to play later. We report download playback, rather than
    downloads, where possible.

    View Slide

  22.  BBC 2017
    iPlayer Performance Pack, July 2017

    View Slide

  23.  BBC 2017
    Requests by device type
    iPlayer Performance Pack, July 2017

    View Slide

  24.  BBC 2017
    TV Devices,
    49%
    Tablet, 25%
    Mobile, 11%
    Computer,
    15%
    Apples to Oranges
    Tablet,
    12.86%
    Mobile,
    38.40%
    Desktop,
    48.06%
    Console,
    0.68%
    StatCounter Global Stats
    “stats are based on over 15 billion page views per month recorded across more than 2.5
    million websites”
    http://gs.statcounter.com/platform-market-share/all/united-kingdom/#monthly-201707-201707-bar
    iPlayer requests UK Market Share

    View Slide

  25.  BBC 2017
    TV Devices,
    49%
    Tablet, 26%
    Mobile, 12%
    Computer, 17%
    55.1%
    12.9%
    12.0%
    9.2%
    8.3% 2.6%
    Requests by device type
    http://gs.statcounter.com/browser-market-share/desktop/united-kingdom#monthly-201706-201707
    https://www.statista.com
    /statistics/487815/market
    -share-tablet-device-
    vendors-uk/
    https://www.statista.com/statistics/387227/market-share-of-smartphone-manufacturers-in-the-uk/

    View Slide

  26.  BBC 2017
    Classes of HTTP clients
    ?
    Media
    Streaming
    Web
    browsers
    (micro) Service
    APIs
    IoT
    ?
    ?
    “Apps”
    (downloads,
    reporting, gaming, etc)
    ?
    Caches
    ?
    Scripts,
    DevOps, etc
    ?
    Assistants
    ? ?

    View Slide

  27.  BBC 2017
    “Some” characteristics of non-browsers HTTP clients
    • Awareness and display of context
    • No URL bar
    • No “padlock”
    • No DOM
    • Same-origin policy, Cookies
    • Handling errors
    • Exposing the error to the right layer
    • Even Fetch and MSE has this kind of problem
    • Rectifying the error
    • No user to reload page or tell an admin
    • “Limited” devices
    • Monolithic images
    • Device support lifetime and maintenance schedules
    • Sometimes never connected
    • Sometimes contain multiple HTTP clients
    • “Good enough” / “Means to an end”
    • Lack of awareness for considerations outside immediate problem domain
    • Select a client with minimal feature set required to get the job done
    • Security can be a barrier

    View Slide

  28.  BBC 2017
    “Non-GUI” Features of HTTP clients
    • Methods
    • GET, POST, PUT, custom
    • HTTP authentication
    • Basic, digest, NTLM, Negotiate
    • HTTP Redirect
    • Limiting
    • Caching
    • Cookies
    • Content parsing and recursion
    • JavaScript or JSON support
    • Recover and continue
    • Compression/decompression
    • Content-encoding, transport-encoding
    • IDN, response charset
    • Protocol evolution
    • HTTP/2, QUIC
    • HTTP Alternative Services
    • Happy eyeballs
    • TLS
    • Trusted CAs
    • Certificate transparency
    • HSTS
    • Preload
    • Session resumption
    • Cert revocation
    • Client certificates
    • SNI
    • Extended validation
    • Proxy support
    • HTTP/HTTPS
    • SOCKS
    • OAuth
    • Etc, etc
    Lucas’ low tech “caniuse.com”

    View Slide

  29.  BBC 2017
    Closing
    “Subsets and legacy browsers can hold the lowest common denominator
    down, preventing the spread of platform-wide progress. This risk is
    magnified by use; more instances of subsetted browsers lead to a more
    heterogeneous web. Runtimes which are not general-purpose browsers
    or do not see much use pose less risk.”
    https://www.w3.org/2001/tag/doc/evergreen-web/

    View Slide

  30.  BBC 2017
    Thanks!
    October 2017
    Lucas Pardue, Project Engineer
    @SimmerVigor
    @BBCRD

    View Slide