Writing Redis Modules in Rust

- A short introduction to Rust.
- Explain how it is very well suited to writing Redis modules.
- Show how this can be done.

Transcript

1. presented by Writing Redis Modules in Rust Gavrie Philipson Redis Labs
2. presented by • A systems programming language originating at Mozilla • Rust is blazingly fast and memory-efficient • Rust’s rich type system and ownership model guarantee memory safety and thread safety What is Rust?
3. presented by • Rust has a friendly compiler with useful error messages • Rust has top-notch tooling: • Integrated package manager and build tool (Cargo) • Support for all major editors and IDEs • Rust was the “most loved programming language” in the Stack Overflow Developer Survey for 2016, 2017, and 2018 What is Rust?
4. presented by • Redis Modules run in the same memory space as Redis • A bad memory access can crash the whole process • Memory leaks will waste precious Redis memory • We need safe memory usage! • This is the primary issue that Rust addresses Why use Rust for writing Redis modules?
5. presented by • Redis is (mostly) single-threaded • A slow module will slow down Redis • We need performance! • Rust’s safety does not mean reduced performance • We can have nice things Why use Rust for writing Redis modules?
6. presented by How did we get to Rust? 1970 1976 1982 1988 1995 2001 2007 2013 2019 C 1972 • First there was C • Used to make a portable version of Unix • Relatively small and simple language • Runs everywhere
7. presented by Writing safe code in C is Hard 1970 1976 1982 1988 1995 2001 2007 2013 2019 C 1972 • Features raw pointers, raw memory access, pointer arithmetic • Memory management is handled explicitly by the developer • No standard way to handle errors • Writing safe code requires significant expertise
8. presented by Then came C++ 1970 1976 1982 1988 1995 2001 2007 2013 2019 C++ 1985 C 1972 • Smart pointers and RAII • Zero-cost abstractions • Exceptions for error handling • Rich standard library
9. presented by Then came C++, but… 1970 1976 1982 1988 1995 2001 2007 2013 2019 C++ 1985 C 1972 • Enormously complex language • Inscrutable error messages • Includes all of unsafe C inside
10. presented by Big raise in VM-based languages — not suitable for our goal 1970 1976 1982 1988 1995 2001 2007 2013 2019 Java 1995 JS 1995 C++ 1985 C 1972
11. presented by Functional Languages become mainstream: Scala, F# 1970 1976 1982 1988 1995 2001 2007 2013 2019 Scala 2004 Java 1995 JS 1995 C++ 1985 C 1972
12. presented by Back to bare metal: Go and Rust 1970 1976 1982 1988 1995 2001 2007 2013 2019 Scala 2004 Rust 2010 Go 2009 Java 1995 JS 1995 C++ 1985 C 1972
13. presented by Strong type system, type inference, functional look and feel 1970 1976 1982 1988 1995 2001 2007 2013 2019 TypeScript 2012 Scala 2004 Swift 2014 Kotlin 2011 Rust 2010 Go 2009 Java 1995 JS 1995 C++ 1985 C 1972
14. presented by typedef struct Person { char *name; int age; } Person; int main() { Person person = { .name="Joe", .age=25 }; Person *somebody = &person; printf("%s is %d\n", somebody ->name, somebody ->age); } What does Rust look like? Let’s start with some C first
15. presented by struct Person { name: &'static str, age: i32, } fn main() { let person = Person { name: "Joe", age: 25 }; let somebody = &person; println!("{} is {}", somebody.name, somebody.age); } The equivalent code in Rust
16. presented by struct Person { name: &'static str, age: i32, } fn main() { let person = Person { name: "Joe", age: 25 }; person.age = 30; // Error: person is immutable } Rust: Immutable is the default
17. presented by struct Person { name: &'static str, age: i32, } fn main() { let mut person = Person { name: "Joe", age: 25 }; person.age = 30; } Rust: Immutable is the default
18. presented by struct Vec<T> { buf: ..., len: usize, } struct String { vec: Vec<u8>, } Rust Generics: Similar to C++
19. presented by let mut languages: Vec<&str> = Vec ::new(); languages.push("C"); languages.push("C ++"); languages.push("Rust"); Creating an initialized vector
20. presented by let mut languages = Vec ::new(); languages.push("C"); languages.push("C ++"); languages.push("Rust"); Creating an initialized vector: Type inference
21. presented by let languages = { let mut temp = Vec ::new(); temp.push("C"); temp.push("C ++"); temp.push("Rust"); temp }; Creating an initialized vector: Keeping it immutable
22. • Code that modifies other code at compile time • Can create new syntax in the language • Hygienic (like Scheme, unlike C) • Works on tokens in the AST presented by let languages = vec!["C", "C ++", "Rust"]; Avoiding the boilerplate: Macros!
23. presented by enum Event { KeyPress(char), MouseClick(i32, i32), } let e1 = Event ::KeyPress('c'); let e2 = Event ::MouseClick(100, 200); Sum types: Similar to tagged unions or variants
24. presented by enum Result<T, E> { Ok(T), Err(E), } Sum types: Result
25. presented by enum Result<T, E> { Ok(T), Err(E), } let f: Result<File, Error> = File ::open("foo"); Result can contain either a value or an error
26. presented by enum Result<T, E> { Ok(T), Err(E), } let f: Result<File, Error> = File ::open("foo"); match f { Ok(file) => { file.bytes(); } Err(err) => { println!("{}", err); } } Using the Result with pattern matching
27. presented by enum Option<T> { None, Some(T), } Rust’s solution for values that may be absent: Option
28. presented by fn main() { let person = Person { name: "Joe", age: 25 }; let somebody: Option<Person> = Some(person); let nobody: Option<Person> = None; } Using Rust’s Option type
29. presented by fn main() { let person = Person { name: "Joe", age: 25 }; let somebody: Option<Person> = Some(person); match somebody { Some(p) => println!("{} is {}", p.name, p.age), None => println!("Nobody there"), } } Rust Option with Pattern Matching
30. presented by Prior Art: • redis-cell by Brandur, 2016 • redismodule by Faineance, 2016 • redis-multi-map by Garrett Squire, 2018 Creating a Rust API for writing Redis Modules
31. presented by /* Callback for Redis Commands */ typedef int (*RedisModuleCmdFunc)( RedisModuleCtx *ctx, RedisModuleString **argv, int argc ); redismodules.h
32. presented by pub type RedisModuleCmdFunc = Option< unsafe extern "C" fn( ctx: *mut RedisModuleCtx, argv: *mut *mut RedisModuleString, argc: c_int, ) -> c_int, >; Rust bindings as autogenerated by the Bindgen tool
33. presented by We would like to: • Allow people to write pure Rust code without needing unsafe • Hide the C interfaces behind a pure Rust API • Have an API that is idiomatic Rust and doesn’t rely on C-like types (such as an argv/argc pair) • Add a high-level abstraction over the Rest Modules API Low-level bindings are not user-friendly
34. presented by fn my_command( ctx: &Context, args: Vec<String>, ) -> Result<RedisValue, RedisError> { … } Idiomatic Rust binding
35. presented by • For each C callback, we need to define a pure Rust function that is then wrapped with a C-compatible callback that Redis can call • Macros to the rescue How do we avoid writing lots of boilerplate code?
36. presented by fn hello(ctx: &Context, args: Vec<String>) -> RedisResult { let key = args.into_iter().skip(1).next_string()?; let greeting = format!("Hello, {}!", key); Ok(greeting.into()) } redis_module! { name: "hello", version: 1, commands: [["hello", hello, ""]], } A sample Redis module in Rust
37. presented by 127.0.0.1:6379> MODULE LOAD libhello.dylib OK 127.0.0.1:6379> HELLO (error) ERR wrong number of arguments for 'HELLO' command 127.0.0.1:6379> HELLO world "Hello, world!" 127.0.0.1:6379> Running the module
38. presented by • https://github.com/RedisLabsModules/redismodule-rs/ • Still in early stages • Contributions are more than welcome! • Feel free to contact me at gavrie@redislabs.com Check out the code
39. presented by Thank you!
40. presented by • Memory management based on scope and on move semantics • No Garbage Collector (GC) • No runtime • Zero-cost abstractions • “We can have nice things” Rust: Performance
41. presented by • Strong ownership model for data • Compiler determines lifetimes of variables (no use-after-free possible) • No null references • Dereferencing a raw pointer is unsafe • Mutable static variables are unsafe Rust: Safety
42. presented by • Creating raw bindings between Rust and C code is simple • Our main task has been (and still is): • Come up with a clean, safe and idiomatic Rust API for modules • Hide all the ugly stuff from the module developer What do we need?
43. presented by Jack Moffitt, developer for the Rust powered browser Servo, said: • “All of [ our security critical bugs with Firefox WebAudio ] were array out of bounds or use-after-free errors, and all of them would have been prevented by the Rust compiler” Why Rust?
44. presented by fn main() { let person = Person { name: "Joe", age: 25 }; let somebody: Option<Person> = Some(person); match somebody { Some(p) => println!("{} is {}", p.name, p.age), None => println!("Nobody there"), } println!("{}", somebody); // ERROR: Value was consumed } Rust Move Semantics
45. presented by int main() { Person person = { .name="Joe", .age=25 }; Person *nobody = NULL; printf("%s is %d\n", nobody ->name, nobody ->age); } // SEGFAULT! C compiler allows dereferencing a NULL pointer