Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rust, Wright's Law, and the Future of Low-Latency Systems

Bryan Cantrill
October 07, 2022
23

Rust, Wright's Law, and the Future of Low-Latency Systems

Talk given at P99 CONF 2021. Video: https://www.youtube.com/watch?v=cuvp-e4ztC0

Bryan Cantrill

October 07, 2022
Tweet

Transcript

  1. Brought to you by
    Rust, Wright's Law, and the
    Future of Low-Latency
    Systems
    Bryan Cantrill
    CTO, Oxide Computer Company

    View Slide

  2. OXIDE
    The Arbiter of Systems Performance
    • If it needs to be said: software must execute within the confines of
    hardware — and the hardware is the ultimate arbiter of performance
    • The history of improving systems performance consists broadly of:
    ○ Process revolutions that propelled all designs
    ○ Architectural revolutions that optimized for certain use cases
    ○ Software revolutions that allowed us to better utilize hardware
    • All of these revolutions must live within the confines of economics!

    View Slide

  3. OXIDE
    Process Revolution: Moore’s Law
    • In his 1965 paper, Gordon Moore did not actually coin a law per se —
    he just a bunch of incredibly astute and prescient observations
    • The term “Moore’s Law” would be coined by Carver Mead in 1971 as
    part of his work on determining ultimate physical limits
    • Moore updated the law in 1975 to be a doubling of transistor density
    every two years (Denard scaling would be outlined in detail in 1974)
    • For many years, Moore’s Law could be inferred to be doublings of
    transistor density, speed, and economics

    View Slide

  4. OXIDE
    Moore’s Law: The Good Old Days
    • The 1980s and early 1990s were great for Moore’s Law — so much so
    that computers needed a “turbo button” to counteract its effects (!!)
    • But even in those halcyon years, Moore’s Law was leaving DRAM
    behind: memory was becoming denser but no faster
    • An increasing number of workloads began hitting the memory wall
    • Caching — an essential architectural revolution born in the 1960s —
    was necessary but insufficient...

    View Slide

  5. OXIDE
    Moore’s Law: The Good Old Days?
    • By the mid-1990s, it had become clear that symmetric multiprocessing
    was the path to deliver throughput on multi-threaded workloads
    • SMP necessitated its own software revolution (multi-threaded systems),
    but did little for single-threaded latency
    • Deep pipelining and VLIW were — largely — failed experiments
    • For single-threaded workloads, microprocessors turned to out-of-order
    and speculative execution to hide memory latency
    • Even in simpler times, scaling with Moore’s Law was a challenge!

    View Slide

  6. OXIDE
    Moore’s Law Deceleration
    • In August 2018, GlobalFoundries suddenly stopped 7nm development,
    citing economics — it was simply too expensive to stay competitive
    • GlobalFoundries’ departure left TSMC and Samsung on 7nm — and
    Intel on 14nm, struggling to get to 10nm
    • Intel’s Cannon Lake was three years late and an unmitigated disaster —
    and for Ice Lake/Cascade Lake, Intel is intermixing 14nm and 10nm
    • Moving to 3nm/5nm requires moving beyond FinFETs to GAAFETs —
    and to EUV photolithography; new nodes are very expensive!

    View Slide

  7. OXIDE
    Aside: Process nodes
    • You may well wonder: when a process node is “7nm” or “5nm”, what
    exactly is seven nanometers or five nanometers long? (And, um, how big
    is a silicon atom anyway?)
    • Answer to the second question: ~210 picometers!
    • Answer to the first question: nothing! Unbelievably, the name of the
    process node no longer measures anything at all (!!) — it is merely a
    rough expression of transistor density (and implication of process)
    • E.g. 7nm ≈ 100MTr/mm2 (but there are lots of caveats)

    View Slide

  8. OXIDE
    The End of Moore’s Law
    • Increased transistor density is continuing to be possible, but at a greatly
    slowed pace — and at outsized cost
    • Moore’s Law has ceased to exist as an economic law
    • But is there another way of looking at it?

    View Slide

  9. OXIDE
    Wright’s Law
    • In 1936, Theodore Wright studied the costs of aircraft manufacturing,
    finding that the cost dropped with experience
    • Over time, when volume doubled, unit costs dropped by 10-15%
    • This phenomenon has been observed in other technological domains
    • In 2013, Jessika Trancik et al. found Wright’s Law to hold better
    predictive power for transistor cost than Moore’s Law!
    • Wright’s Law seems to hold, especially for older process nodes

    View Slide

  10. OXIDE
    Wright’s Law: Ramifications
    • If Wright’s Law continues to hold, compute will be economically viable in
    more and more places that were previously confined to hard logic
    • This is true even on die, where chiplets have made it easier than ever to
    build a heterogeneous system — and where mixed process nodes have
    demanded more sophistication
    • Quick, how many cores are on your server? (Don’t forget the hidden
    ones!)

    View Slide

  11. OXIDE
    Wright’s Law: Performance Ramifications
    • Having more compute in many more places is particularly germane to
    system performance:
    ○ More compute close to data (SmartNICs, open-channel SSDs,
    on-spindle compute) lowers latency
    ○ Bringing data to special-purpose compute (GPGPUs, FPGAs)
    increases throughput
    • But security and multi-tenancy cannot be an afterthought!
    • We need to rethink our system software

    View Slide

  12. OXIDE
    Aside: A Researcher’s Call to Rethink
    Timothy Roscoe, OSDI 2021 Keynote, It's Time for Operating Systems to Rediscover Hardware

    View Slide

  13. OXIDE
    The Needed Software Revolution
    • Much of the coming compute is, at some level, special purpose
    • These systems are much less balanced than our general-purpose
    systems — with much less memory and/or non-volatile storage
    • The overhead of dynamic environments (Java, Go, Python, etc.) is
    unacceptably high — and the development benefit questionable
    • Languages traditional used in this domain — C and C++ — both have
    well-known challenges around safety and composability
    • Enter Rust, and its killer feature...

    View Slide

  14. OXIDE
    Rust: no_std
    • Rust is a revolutionary language in many respects, but one that may be
    underappreciated is its ability to not depend on its own standard library
    • Much of what is valuable about the language — sum types, ownership
    model, traits, hygienic macros — is in core, not the standard library
    • Crates marked “no_std” will not perform any heap allocations — and
    any such allocation is a compile-time error!
    • But no_std crates can depend on other no_std crates — lending real
    composability to a domain for whom it has been entirely deprived

    View Slide

  15. OXIDE
    Rust: no_std binaries
    • Rust no_std binaries are stunningly small
    ○ E.g., at Oxide, we are developing a message-passing, memory
    protected system entirely in Rust (Rust microkernel, Rust tasks);
    minimal systems are 30K — and entirely realistic ones are < 200K!
    • no_std is without real precedent in other languages or environments; it
    allows Rust to be put in essentially arbitrarily confined contexts
    • Rust is the first language since C to meaningfully exist at the boundary
    of hardware and systems software!

    View Slide

  16. OXIDE
    Rust, Wright’s Law and the Future
    • Wright’s Law will continue to hold, resulting in more compute in more
    places — and this compute will be essential for systems performance
    • These compute elements will be increasingly special-purpose, and are
    going to require purpose-fit software
    • Rust is proving to be an excellent fit for these use cases!
    • We fully expect many more open source, de novo hardware-facing
    Rust-based systems — and thanks to no_std they will be able to
    leverage one another; the Rust revolution is here!

    View Slide

  17. Brought to you by
    Bryan Cantrill
    [email protected]
    @bcantrill

    View Slide