Platform values, Rust, and the implications for system software CTO [email protected] Bryan Cantrill @bcantrill

Software platform and values • Deciding on programming language, operating system, etc. — that is, deciding on a software platform — is a big, important decision with lasting consequences (even for small work) • One doesn’t merely select a language for where it is, but also where it’s going — and what it represents • In the post-open source world of software infrastructure, this is reflected and guided by a community’s values • Beyond right tool for the job, it is the right values for the job… • …and then the right software for the values

Some software platform values • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values • All of these values are important — but they are in tension • Platforms aspire to balance all of them — but for every platform some small subset represents its core values • These core values attract a like-minded community — and become self-reinforcing…

Platform core values: C • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values: K • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values: OpenBSD • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values: Awk • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values: Scala? • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Values: The challenge for system software • For system software — that is, the software that abstracts hardware and serves as a generic platform — values are defined by expectations • For some values, if your system software doesn’t have it, the system itself can’t ever have it!

Values we demand of system software: • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values: C • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Platform core values: C++ • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

The system software disconnect • There is a tremendous divide between the values held by system software programming languages and the values that must be held by the software itself • Over the years (decades), we in system software have been able to deliver relatively secure, relatively reliable systems… • …but it is despite the programming language, not because of it! • Can we do better?

Enter Rust? • Rust is a systems software programming language designed around safety, parallelism, and speed • Rust has a novel system of ownership, whereby it can statically determine when a memory object is no longer in use • This allows for the power of a garbage-collected language, but with the performance of manual memory management • This is important because — unlike C — Rust is highly composable, allowing for more sophisticated (and higher performing!) primitives

Platform core values: Rust • Approachability • Availability • Compatibility • Composability • Debuggability • Expressiveness • Extensibility • Interoperability • Integrity • Maintainability • Measurability • Operability • Performance • Portability • Resiliency • Rigor • Robustness • Safety • Security • Simplicity • Stability • Thoroughness • Transparency • Velocity

Rust performance (my experience) Source: http://dtrace.org/blogs/bmc/2018/09/28/the-relative-performance-of-c-and-rust/

Rust: Beyond ownership • Rust has a number of other features that make it highly compelling for systems software implementation: • Algebraic types allow robust, concise error handling • Hygienic macros allow for safe syntax extensions • Foreign function interface allows for full-duplex integration with C without sacrificing performance • “unsafe” keyword allows for some safety guarantees to be surgically overruled (though with obvious peril) • Also: terrific community, thriving ecosystem, etc.

Systems software in Rust: The promise • Rust’s values are an excellent fit for systems software • The beauty of Rust is that it shifts cognitive load from the operation of software back to the developer • Further, Rust’s ownership restricts what the software can do • The process can be frustrating but the end result is often satisfying: Source: Programming Rust by Jim Blandy and Jason Orendorff, page 262

Systems software in Rust: The peril • While Rust’s advantages are themselves clear, it’s less clear what the advantage is when replacing otherwise working code • Rust may be especially challenging in large, monolithic systems like an OS kernel which often have multiply-owned structures • So an OS kernel — despite its historic appeal and superficial fit for Rust — may represent more challenge than its worth • But what of hybrid approaches?

Hybrid approach I: Rust subsystems • One appeal of Rust is its ability to interoperate with C • One hybrid approach to explore would be to retain a traditional (C-based) system while allowing for Rust-based components • This would allow for an incremental approach — and instead of rewriting, Rust can be used for new development • There is a prototype example of this in FreeBSD; others are presumably possible

Hybrid approach II: Rust system components • System software is much broader than the operating system! • System software consists of many user-level components: utilities, daemons, service-/device-/fault- management facilities, debuggers, etc. • And then the distributed system that represents a multi- computer control plane — that itself includes many components • These components are much more prone to run-time failure! • Many of these are an excellent candidate for Rust!

Hybrid approach III: Rust-based firmware • Below the operating system lurks hardware-facing special- purpose software: firmware • Firmware is a sewer of unobservable software with a long history of infamous quality problems • Firmware has some of the same challenges as kernel development (e.g., dealing with allocation failures), but may otherwise be more amenable to Rust • This is especially true when/where firmware is in user-space and is network-facing! (e.g., OpenBMC)

Looking forward: Systems software in Rust • Rust represents something that we haven’t seen in a long time: a modern language that represents an alternative throughout the stack of software abstraction: we can have nice things! • Rust’s values are an excellent fit for system software! • Rust’s interoperability allows hybrid approaches, allowing for productive kernel incrementalism rather than whole-system rewrites • Firmware and user-level operating system software are two very promising candidates for implementation in Rust!