Uniting Rust Servers and Clients through OpenAPI Adam Leventhal 

OXIDE What is Oxide? ● Rack scale computer ● On-premises, private cloud in a box ● All software included 2 

OXIDE 3 

OXIDE Software? ● The hardware is very cool–also there’s a ton of software that drives the platform ● Data services like storage and VPC (networking) ● A control plane for configuration and control ● Divided up into services for various components 4 

OXIDE Why OpenAPI? ● “Seems popular, let’s use it” ● Used to describe internal services and the external API ● A rare opportunity to be normal! ● (foreshadowing) Vibrant, open ecosystem of tooling! 5 

OXIDE Why Rust? ● In 2020 we knew there was lots of software at lots of layers we’d be building: embedded OS, host OS, hypervisor, storage, control plane, etc. ● Rust was becoming the consensus choice for low level systems ● Was not obvious (to me) that it was ideal for higher level software ● Memory safety is important up and down the stack ● So is memory consumption and general robustness ● … and it’s kind of in the name of the company 6 

OXIDE Dropshot: our HTTP API framework ● Evaluated Rust web frameworks and didn’t find what we needed ● Dropshot has OpenAPI at its core–define the API in Rust code, and emit the OpenAPI description ● Many OpenAPI advocates say (and said) that this is basically wrong ● We wanted one source of truth, the code was changing fast, so the code was going to be the truth ● Built dropshot into all our services… and started to consume those APIs (Dave Pacheco and I gave a talk on Dropshot in 2020; link at the end) 7 

OXIDE OpenAPI Ecosystem Fail ● Our perception of the strength of the ecosystem was a huge attraction ● Largely disappointed for what we needed ● E.g. generated Rust client bindings 8 

OXIDE openapi-generator 9 

OXIDE openapi-generator 10 

OXIDE Progenitor ● Generate Rust client bindings specifically for our Dropshot-based services ● Fully automated–no manual intervention required ● Immediately reactive to changes in the spec documents 11 

OXIDE Client Generation 12 

OXIDE Aside: Rust Macros ● In Rust “!” is short for “pay attention, anything could be about to happen!” ● Rust macros are spectacular: write Rust code to generate Rust tokens ● The ecosystem around it is also spectacular: derive an AST from tokens, quote tokens, produce great error messages, etc. ● Progenitor hides all the complexity of compiling OpenAPI into Rust within a macro invocation which makes it dead simple to use 13 

OXIDE Progenitor in action ● Idiomatic SDKs ● Automatic client generation eliminates a whole class of common errors ● Like good tests: fearless refactors 14 

OXIDE Pagination ● One goal of Dropshot was to make it easy to do things the right way and hard to do them the wrong way e.g. we wanted one pagination mechanism ● Upshot: in Progenitor we can automatically iterate over pages of items: 15 

OXIDE Surprises ● Started using Progenitor for our internal APIs–now using it for our public SDK ● Progenitor: ~20K LOC; SDK: ~60K LOC (generated) ● Using the same AST in Progenitor to emit our end-user CLI ● And an HTTP mocking library (in part, to test the CLI) 16 

OXIDE Lessons 17 

OXIDE Lesson: Rust is great for API clients ● I get it: Rust people are a meme! ● But seriously! ● If you’re patient with Rust it will pay you back 18 

OXIDE Lesson: Write the tools you need ● I was much too reluctant to start on Progenitor ● If I had understood the complexity of OpenAPI and JSON Schema like I do now I might have been even more reluctant ● You don’t need to solve the general problem! ● For progenitor we would change the API to simplify generation! ● Bryan Cantrill gave a great talk on “the primacy of toolmaking” (link at the end) 19 

OXIDE Lesson: Own your strategic weirdness ● Our choice of OpenAPI was definitely right… even though the ecosystem wasn’t what we had hoped ● We knew we wanted to do things a bit differently at the HTTP API layer ● Dropshot and Progenitor were important components we needed to build ● We don’t build everything from scratch (even though it seems like it) ● Areas where you have hopes and ambitions deserve at least a prototype! 20 

OXIDE Thanks! ● Oxide https://oxide.computer ● Dropshot https://github.com/oxidecomputer/dropshot ● 2020 API Spec Conf Talk https://www.youtube.com/watch?v=EmSjZbSzA3A ● Progenitor https://github.com/oxidecomputer/progenitor ● Typify (JSON Schema → types) https://github.com/oxidecomputer/typify ● Oxide Podcast: client generation https://share.transistor.fm/s/8460f5be ● Bryan Cantrill: Sharpening the Axe https://www.youtube.com/watch?v=_GpBkplsGus 21