Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Platform values, Rust, and the implications for system software

Platform values, Rust, and the implications for system software

Talk given at Scale By The Bay 2018. Video is at https://www.youtube.com/watch?v=2wZ1pCpJUIM. If you are interested in this talk, you might also be interested in my talk on Platform as a Reflection of Values from Node Summit 2017: https://vimeo.com/230142234

Bryan Cantrill

November 16, 2018
Tweet

More Decks by Bryan Cantrill

Other Decks in Technology

Transcript

  1. Platform values, Rust, and the
    implications for system software
    CTO
    [email protected]
    Bryan Cantrill
    @bcantrill

    View Slide

  2. Software platform and values
    • Deciding on programming language, operating system, etc. —
    that is, deciding on a software platform — is a big, important
    decision with lasting consequences (even for small work)
    • One doesn’t merely select a language for where it is, but also
    where it’s going — and what it represents
    • In the post-open source world of software infrastructure, this is
    reflected and guided by a community’s values
    • Beyond right tool for the job, it is the right values for the job…
    • …and then the right software for the values

    View Slide

  3. Some software platform values
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  4. Platform core values
    • All of these values are important — but they are in tension
    • Platforms aspire to balance all of them — but for every platform
    some small subset represents its core values
    • These core values attract a like-minded community — and
    become self-reinforcing…

    View Slide

  5. Platform core values: C
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  6. Platform core values: K
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  7. Platform core values: OpenBSD
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  8. Platform core values: Awk
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  9. Platform core values: Scala?
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  10. Values: The challenge for system software
    • For system software — that is, the software that abstracts
    hardware and serves as a generic platform — values are
    defined by expectations
    • For some values, if your system software doesn’t have it, the
    system itself can’t ever have it!

    View Slide

  11. Values we demand of system software:
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  12. Platform core values: C
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  13. Platform core values: C++
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  14. The system software disconnect
    • There is a tremendous divide between the values held by
    system software programming languages and the values that
    must be held by the software itself
    • Over the years (decades), we in system software have been
    able to deliver relatively secure, relatively reliable systems…
    • …but it is despite the programming language, not because of it!
    • Can we do better?

    View Slide

  15. Enter Rust?
    • Rust is a systems software programming language designed
    around safety, parallelism, and speed
    • Rust has a novel system of ownership, whereby it can statically
    determine when a memory object is no longer in use
    • This allows for the power of a garbage-collected language, but
    with the performance of manual memory management
    • This is important because — unlike C — Rust is highly
    composable, allowing for more sophisticated (and higher
    performing!) primitives

    View Slide

  16. Platform core values: Rust
    • Approachability
    • Availability
    • Compatibility
    • Composability
    • Debuggability
    • Expressiveness
    • Extensibility
    • Interoperability
    • Integrity
    • Maintainability
    • Measurability
    • Operability
    • Performance
    • Portability
    • Resiliency
    • Rigor
    • Robustness
    • Safety
    • Security
    • Simplicity
    • Stability
    • Thoroughness
    • Transparency
    • Velocity

    View Slide

  17. Rust performance (my experience)
    Source: http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/

    View Slide

  18. Rust: Beyond ownership
    • Rust has a number of other features that make it highly
    compelling for systems software implementation:
    • Algebraic types allow robust, concise error handling
    • Hygienic macros allow for safe syntax extensions
    • Foreign function interface allows for full-duplex integration
    with C without sacrificing performance
    • “unsafe” keyword allows for some safety guarantees to be
    surgically overruled (though with obvious peril)
    • Also: terrific community, thriving ecosystem, etc.

    View Slide

  19. Systems software in Rust: The promise
    • Rust’s values are an excellent fit for systems software
    • The beauty of Rust is that it shifts cognitive load from the
    operation of software back to the developer
    • Further, Rust’s ownership restricts what the software can do
    • The process can be frustrating but the end result is often
    satisfying:
    Source: Programming Rust by Jim Blandy and Jason Orendorff, page 262

    View Slide

  20. Systems software in Rust: The peril
    • While Rust’s advantages are themselves clear, it’s less clear
    what the advantage is when replacing otherwise working code
    • Rust may be especially challenging in large, monolithic systems
    like an OS kernel which often have multiply-owned structures
    • So an OS kernel — despite its historic appeal and superficial fit
    for Rust — may represent more challenge than its worth
    • But what of hybrid approaches?

    View Slide

  21. Hybrid approach I: Rust subsystems
    • One appeal of Rust is its ability to interoperate with C
    • One hybrid approach to explore would be to retain a traditional
    (C-based) system while allowing for Rust-based components
    • This would allow for an incremental approach — and instead of
    rewriting, Rust can be used for new development
    • There is a prototype example of this in FreeBSD; others are
    presumably possible

    View Slide

  22. Hybrid approach II: Rust system components
    • System software is much broader than the operating system!
    • System software consists of many user-level components:
    utilities, daemons, service-/device-/fault- management facilities,
    debuggers, etc.
    • And then the distributed system that represents a multi-
    computer control plane — that itself includes many components
    • These components are much more prone to run-time failure!
    • Many of these are an excellent candidate for Rust!

    View Slide

  23. Hybrid approach III: Rust-based firmware
    • Below the operating system lurks hardware-facing special-
    purpose software: firmware
    • Firmware is a sewer of unobservable software with a long
    history of infamous quality problems
    • Firmware has some of the same challenges as kernel
    development (e.g., dealing with allocation failures), but may
    otherwise be more amenable to Rust
    • This is especially true when/where firmware is in user-space
    and is network-facing! (e.g., OpenBMC)

    View Slide

  24. Looking forward: Systems software in Rust
    • Rust represents something that we haven’t seen in a long time:
    a modern language that represents an alternative throughout
    the stack of software abstraction: we can have nice things!
    • Rust’s values are an excellent fit for system software!
    • Rust’s interoperability allows hybrid approaches, allowing for
    productive kernel incrementalism rather than whole-system
    rewrites
    • Firmware and user-level operating system software are two very
    promising candidates for implementation in Rust!

    View Slide