Slide 1

Slide 1 text

2019/11/10 “Rust: Imperative Language 2.0” 1 Rust: Imperative Language 2.0 Masaki Hara (@qnighy) Software Engineer at Wantedly, Inc. 2019/11/10 fpc2019japan

Slide 2

Slide 2 text

2019/11/10 “Rust: Imperative Language 2.0” 2 Myself • Software Engineer at Wantedly • Ruby on Rails and Golang • Wantedly People (Business card scanner & SNS) • Current interest: Rust • Former interests: formal methods, competitive programming, math, ...

Slide 3

Slide 3 text

2019/11/10 “Rust: Imperative Language 2.0” 3 Myself in Rust • Language specs, features, and tricks

Slide 4

Slide 4 text

2019/11/10 “Rust: Imperative Language 2.0” 4 Table of contents • Rust • Lifetime story • Shared XOR mutable • Ownership • Lifetimes • References as locks • Advanced topics

Slide 5

Slide 5 text

2019/11/10 “Rust: Imperative Language 2.0” 5 Rust

Slide 6

Slide 6 text

2019/11/10 “Rust: Imperative Language 2.0” 6 Rust: https://www.rust-lang.org/ Part I > Rust

Slide 7

Slide 7 text

2019/11/10 “Rust: Imperative Language 2.0” 7 Rust: https://www.rust-lang.org/ • Capable of being nearly as low-level as C • Nearly as fast as C – see Benchmarks Game results. • Safe abstractions – guaranteed freedom from UB. Part I > Rust

Slide 8

Slide 8 text

2019/11/10 “Rust: Imperative Language 2.0” 8 Rust: https://www.rust-lang.org/ • Best-effort safeguard for memory-leak freedom, panic- freedom, portability, language forward-compatibility and cross-compilability. • Functional-esque abstractions – parametric polymorphism (generics), ADT (enums), type classes (traits) • Language designs focused on consistency and orthogonality. Part I > Rust

Slide 9

Slide 9 text

2019/11/10 “Rust: Imperative Language 2.0” 9 Rust: https://www.rust-lang.org/ • Officially distributed tools – everyone uses the same tool • Rustup – compiler version manager and installer • Cargo – package manager and build tool • Rustfmt – formatter • Clippy – linter • RLS – language server (editor & IDE support) • Great effort on documentation • Great effort on community inclusion Part I > Rust

Slide 10

Slide 10 text

2019/11/10 “Rust: Imperative Language 2.0” 10 Rust: https://www.rust-lang.org/ • It achieves performance and safety at the same time thanks to references, ownership and lifetime. &'a T &'a mut T Box References Smart pointers Part I > Rust

Slide 11

Slide 11 text

2019/11/10 “Rust: Imperative Language 2.0” 11 Rust: https://www.rust-lang.org/ • It achieves performance and safety at the same time thanks to references, ownership and lifetime. &'a T &'a mut T Box Uncopyable (i.e. ownership is managed) Part I > Rust

Slide 12

Slide 12 text

2019/11/10 “Rust: Imperative Language 2.0” 12 Rust: https://www.rust-lang.org/ • It achieves performance and safety at the same time thanks to references, ownership and lifetime. &'a T &'a mut T Box Lifetimes Part I > Rust

Slide 13

Slide 13 text

2019/11/10 “Rust: Imperative Language 2.0” 13 Rust: https://www.rust-lang.org/ • It achieves performance and safety at the same time thanks to references, ownership and lifetime. &T &mut T Box Part I > Rust

Slide 14

Slide 14 text

2019/11/10 “Rust: Imperative Language 2.0” 14 Rust good points • They’re classified as: Great ecosystems included Not unique but advanced Almost inherent to Rust (for now) • rustfmt ← gofmt • clippy ← flake8 • Cargo ← npm • rustup ← rbenv • ADT • Traits (type classes) • RAII • Move semantics • Ownership • Lifetimes • Shared XOR mutable Part I > Rust

Slide 15

Slide 15 text

2019/11/10 “Rust: Imperative Language 2.0” 15 Rust good points • They’re classified as: Great ecosystems included Not unique but advanced Almost inherent to Rust (for now) • rustfmt ← gofmt • clippy ← flake8 • Cargo ← npm • rustup ← rbenv • ADT • Traits (type classes) • RAII • Move semantics • Ownership • Lifetimes • Shared XOR mutable This talk’s focus Part I > Rust

Slide 16

Slide 16 text

2019/11/10 “Rust: Imperative Language 2.0” 16 Lifetime story

Slide 17

Slide 17 text

2019/11/10 “Rust: Imperative Language 2.0” 17 Lifetime story Ownership Shared XOR mutable Compiler optimization Easy-to-reason behavior Fearless concurrency Lifetime Less allocation Part II > Lifetime story

Slide 18

Slide 18 text

2019/11/10 “Rust: Imperative Language 2.0” 18 Lifetime story Ownership Shared XOR mutable Compiler optimization Easy-to-reason behavior Fearless concurrency Lifetime Less allocation Performance is important, but this talk doesn’t focus on it. Part II > Lifetime story

Slide 19

Slide 19 text

2019/11/10 “Rust: Imperative Language 2.0” 19 Lifetime story Ownership Shared XOR mutable Compiler optimization Easy-to-reason behavior Fearless concurrency Lifetime Less allocation Part II > Lifetime story

Slide 20

Slide 20 text

2019/11/10 “Rust: Imperative Language 2.0” 20 Lifetime story Ownership Shared XOR mutable Compiler optimization Easy-to-reason behavior Fearless concurrency Lifetime Less allocation Part II > Lifetime story

Slide 21

Slide 21 text

2019/11/10 “Rust: Imperative Language 2.0” 21 Example 1: shallow clone a = [[0] * 2] * 2 a[0][0] = 100 print(a) # [[100, 0], [100, 0]] Python Part II > Lifetime story

Slide 22

Slide 22 text

2019/11/10 “Rust: Imperative Language 2.0” 22 Example 1: shallow clone a = [[0] * 2] * 2 a[0][0] = 100 print(a) # [[100, 0], [100, 0]] Python Part II > Lifetime story

Slide 23

Slide 23 text

2019/11/10 “Rust: Imperative Language 2.0” 23 Example 1: shallow clone • Non-reproducing mimic let mut a = vec![vec![0; 2]; 2]; a[0][0] = 100; eprintln!("{:?}", a); // [[100, 0], [0, 0]] Rust Part II > Lifetime story

Slide 24

Slide 24 text

2019/11/10 “Rust: Imperative Language 2.0” 24 Example 1: shallow clone • Non-reproducing mimic let mut a = vec![vec![0; 2]; 2]; a[0][0] = 100; eprintln!("{:?}", a); // [[100, 0], [0, 0]] Rust Part II > Lifetime story

Slide 25

Slide 25 text

2019/11/10 “Rust: Imperative Language 2.0” 25 Example 1: shallow clone • Reproducing mimic let a = vec![Rc::new(RefCell::new(vec![0; 2])); 2]; (&a[0]).borrow_mut()[0] = 100; eprintln!("{:?}", a); // [RefCell { value: [100, 0] }, RefCell { value: [100, 0] }] Rust Part II > Lifetime story

Slide 26

Slide 26 text

2019/11/10 “Rust: Imperative Language 2.0” 26 Example 1: shallow clone • Reproducing mimic let a = vec![Rc::new(RefCell::new(vec![0; 2])); 2]; (&a[0]).borrow_mut()[0] = 100; eprintln!("{:?}", a); // [RefCell { value: [100, 0] }, RefCell { value: [100, 0] }] Rust Part II > Lifetime story

Slide 27

Slide 27 text

2019/11/10 “Rust: Imperative Language 2.0” 27 Example 1: shallow clone • Reproducing mimic let a = vec![Rc::new(RefCell::new(vec![0; 2])); 2]; (&a[0]).borrow_mut()[0] = 100; eprintln!("{:?}", a); // [RefCell { value: [100, 0] }, RefCell { value: [100, 0] }] Rust RefCell opts into this behavior Part II > Lifetime story

Slide 28

Slide 28 text

2019/11/10 “Rust: Imperative Language 2.0” 28 Example 1: shallow clone • Reproducing mimic let a = vec![Rc::new(RefCell::new(vec![0; 2])); 2]; (&a[0]).borrow_mut()[0] = 100; eprintln!("{:?}", a); // [RefCell { value: [100, 0] }, RefCell { value: [100, 0] }] Rust RefCell opts into this behavior Part II > Lifetime story

Slide 29

Slide 29 text

2019/11/10 “Rust: Imperative Language 2.0” 29 Shared XOR mutable

Slide 30

Slide 30 text

2019/11/10 “Rust: Imperative Language 2.0” 30 What was the problem? a = [[0] * 2] * 2 a[0][0] = 100 print(a) # [[100, 0], [100, 0]] Python Part III > Shared XOR mutable

Slide 31

Slide 31 text

2019/11/10 “Rust: Imperative Language 2.0” 31 What was the problem? a = [[0] * 2] * 2 a[0][0] = 100 print(a) # [[100, 0], [100, 0]] Python Part III > Shared XOR mutable

Slide 32

Slide 32 text

2019/11/10 “Rust: Imperative Language 2.0” 32 What was the problem? •Mutation is the problem a = [[0] * 2] * 2 a[0][0] = 100 print(a) # [[100, 0], [100, 0]] Python Part III > Shared XOR mutable

Slide 33

Slide 33 text

2019/11/10 “Rust: Imperative Language 2.0” 33 What was the problem? •Mutation is the problem →declarative programming Part III > Shared XOR mutable

Slide 34

Slide 34 text

2019/11/10 “Rust: Imperative Language 2.0” 34 What was the problem? •Mutation is the problem →declarative programming •Mutating a shared reference is the problem →imperative programming 2.0! Part III > Shared XOR mutable

Slide 35

Slide 35 text

2019/11/10 “Rust: Imperative Language 2.0” 35 Shared XOR mutable •A reference can be shared with others. •It can be mutable (writable). •But not both. Part III > Shared XOR mutable

Slide 36

Slide 36 text

2019/11/10 “Rust: Imperative Language 2.0” 36 Shared XOR mutable •A reference can be shared with others. •It can be mutable (writable). •But not both. &T &mut T Part III > Shared XOR mutable

Slide 37

Slide 37 text

2019/11/10 “Rust: Imperative Language 2.0” 37 flock(2) Part III > Shared XOR mutable

Slide 38

Slide 38 text

2019/11/10 “Rust: Imperative Language 2.0” 38 flock(2) For reading a file For writing to a file Part III > Shared XOR mutable

Slide 39

Slide 39 text

2019/11/10 “Rust: Imperative Language 2.0” 39 PostgreSQL row-level locking Part III > Shared XOR mutable

Slide 40

Slide 40 text

2019/11/10 “Rust: Imperative Language 2.0” 40 PostgreSQL row-level locking (simplified) Requested Lock Mode Current Lock Mode FOR SHARE FOR UPDATE FOR SHARE X FOR UPDATE X X Part III > Shared XOR mutable

Slide 41

Slide 41 text

2019/11/10 “Rust: Imperative Language 2.0” 41 Shared XOR mutable •A reference can be shared with others. •It can be mutable (writable). •But not both. Rust applies the general principle for shareable resources to memory regions. Part III > Shared XOR mutable

Slide 42

Slide 42 text

2019/11/10 “Rust: Imperative Language 2.0” 42 Ownership

Slide 43

Slide 43 text

2019/11/10 “Rust: Imperative Language 2.0” 43 Detecting copies • Copying mutable references may break the shared XOR mutable principle. fn dup(x: &mut i32) -> (&mut i32, &mut i32) { (x, x) } Rust Part IV > Ownership

Slide 44

Slide 44 text

2019/11/10 “Rust: Imperative Language 2.0” 44 Detecting copies • That means, we cannot copy a general type T. (unless T: Copy is specified) fn dup(x: T) -> (T, T) { (x, x) } Rust Linear type by default (More accurately this is an affine type. We can drop values at any time) Part IV > Ownership

Slide 45

Slide 45 text

2019/11/10 “Rust: Imperative Language 2.0” 45 Lifetimes

Slide 46

Slide 46 text

2019/11/10 “Rust: Imperative Language 2.0” 46 Sequential mutation • Ownership forbids multiple uses. • This is too restrictive. let mut a = Vec::new(); a.push(42); a.push(84); eprintln!("{:?}", a); Rust push mutates a push mutates a eprintln! reads a Part V > Lifetimes

Slide 47

Slide 47 text

2019/11/10 “Rust: Imperative Language 2.0” 47 Sequential mutation • This code is OK because a is sequentially mutated. • Ensure a is unique at a time let mut a = Vec::new(); a.push(42); a.push(84); eprintln!("{:?}", a); Rust &'a mut Vec &'b mut Vec &'c Vec 'a 'b 'c Part V > Lifetimes

Slide 48

Slide 48 text

2019/11/10 “Rust: Imperative Language 2.0” 48 References as locks

Slide 49

Slide 49 text

2019/11/10 “Rust: Imperative Language 2.0” 49 Example 2: just readonly • Is this a value class? class Revision { constructor( readonly name: string, readonly files: ReadonlyMap, ) { } } TypeScript Part VI > References as locks

Slide 50

Slide 50 text

2019/11/10 “Rust: Imperative Language 2.0” 50 Example 2: just readonly • Unexpected mutation const worktree: Map = new Map(); worktree.set("README.md", "foo"); const rev1 = new Revision("rev1", worktree); worktree.set("a.ts", ""); const rev2 = new Revision("rev2", worktree); TypeScript Part VI > References as locks

Slide 51

Slide 51 text

2019/11/10 “Rust: Imperative Language 2.0” 51 Freedom from third-party mutation • ReadonlyArray may be modified from another owner. • T[] may be modified from another owner too. • (There’s Object.freeze, but it’s irrelevant to types) Part VI > References as locks

Slide 52

Slide 52 text

2019/11/10 “Rust: Imperative Language 2.0” 52 Freedom from third-party mutation • &T cannot be modified from another owner. • &mut T cannot be modified from another owner too. • Acquiring &T or &mut T is like acquiring locks statically. Part VI > References as locks

Slide 53

Slide 53 text

2019/11/10 “Rust: Imperative Language 2.0” 53 Freedom from third-party mutation • &T cannot be modified from another owner. • &mut T cannot be modified from another owner too. • Acquiring &T or &mut T is like acquiring locks statically. Rust references are easy to reason about. Part VI > References as locks

Slide 54

Slide 54 text

2019/11/10 “Rust: Imperative Language 2.0” 54 Easy to reason about → optimization • Famous XOR-swap is optimized away to simple moves. pub fn swap(x: &mut u32, y: &mut u32) { *x ^= *y; *y ^= *x; *x ^= *y; } Rust Part VI > References as locks

Slide 55

Slide 55 text

2019/11/10 “Rust: Imperative Language 2.0” 55 Easy to reason about → optimization • Famous XOR-swap is optimized away to simple moves. …in some old Rust versions. • Rust noalias annotations are sometimes too aggressive that it occasionally hits LLVM bugs. • Current version of Rust (1.39.0) disables noalias annotations for &mut references. Part VI > References as locks

Slide 56

Slide 56 text

2019/11/10 “Rust: Imperative Language 2.0” 56 Easy to reason about → optimization • Values behind &T is constant across function calls: (optimized to always return 0) pub fn observe_diff(x: &i32, f: fn(i32)) -> i32 { let a = *x; f(a); a - *x } Rust Part VI > References as locks

Slide 57

Slide 57 text

2019/11/10 “Rust: Imperative Language 2.0” 57 Almost functionally interpretable • Actions on &mut references are almost functionally interpretable. pub fn f(v: &mut Vec, x: i32); Rust f :: [Int] -> Int -> [Int] Haskell Part VI > References as locks

Slide 58

Slide 58 text

2019/11/10 “Rust: Imperative Language 2.0” 58 Advanced topics

Slide 59

Slide 59 text

2019/11/10 “Rust: Imperative Language 2.0” 59 Deep immutability • &T is deep-immutable by default. This is a mere consequence of the shared XOR mutable principle. &'a mut &'b mut T &'a &'b mut T &'a mut &'b T &'a &'b T &'b T &'b T &'a T &'a mut T Part VII > Advanced topics

Slide 60

Slide 60 text

2019/11/10 “Rust: Imperative Language 2.0” 60 Spatial Splitting • &mut references are often temporally split. let mut a = Vec::new(); a.push(42); a.push(84); eprintln!("{:?}", a); Rust Part VII > Advanced topics

Slide 61

Slide 61 text

2019/11/10 “Rust: Imperative Language 2.0” 61 Spatial Splitting • They are sometimes spatially split too. let mut x = (1, 2); std::mem::swap(&mut x.0, &mut x.1); eprintln!("{:?}", x); Rust Part VII > Advanced topics

Slide 62

Slide 62 text

2019/11/10 “Rust: Imperative Language 2.0” 62 Spatial Splitting • They are sometimes spatially split too. let mut x = (1, 2); std::mem::swap(&mut x.0, &mut x.1); eprintln!("{:?}", x); Rust Different parts of x are borrowed simultaneously. Part VII > Advanced topics

Slide 63

Slide 63 text

2019/11/10 “Rust: Imperative Language 2.0” 63 Spatial Splitting • They are sometimes spatially split too. • Structs/enums: Rust automatically detects borrowing of different fields. • Slices: there’s split_at_mut which does this explicitly . Part VII > Advanced topics

Slide 64

Slide 64 text

2019/11/10 “Rust: Imperative Language 2.0” 64 Concurrent mutations • We’ve avoided concurrent mutations until now. • They’re useful if used carefully: • Logging • Debugging • Manipulating graph-like structures • Generating unique IDs • More than simple paralellism • Essentially concurrent business logics Part VII > Advanced topics

Slide 65

Slide 65 text

2019/11/10 “Rust: Imperative Language 2.0” 65 Concurrent mutations • Interior mutability opts into concurrent mutations. • Interior mutability primitives Cell Atomic* RefCell RwLock Mutex Part VII > Advanced topics

Slide 66

Slide 66 text

2019/11/10 “Rust: Imperative Language 2.0” 66 Concurrent mutations • Interior mutability opts into concurrent mutations. • Interior mutability primitives Cell Atomic* RefCell RwLock Mutex Single-threaded Thread-safe Part VII > Advanced topics

Slide 67

Slide 67 text

2019/11/10 “Rust: Imperative Language 2.0” 67 Concurrent mutations • Interior mutability opts into concurrent mutations. • Interior mutability primitives Cell Atomic* RefCell RwLock Mutex Lockless Lockable Part VII > Advanced topics

Slide 68

Slide 68 text

2019/11/10 “Rust: Imperative Language 2.0” 68 Interior mutability • Interior mutability types are mutated through shared references. fn increment(counter: &Mutex) { let mut counter = counter.lock().unwrap(); *counter += 1; // automatically unlocked } Rust Part VII > Advanced topics

Slide 69

Slide 69 text

2019/11/10 “Rust: Imperative Language 2.0” 69 Interior mutability • Interior mutability types are mutated through shared references. fn increment(counter: &Mutex) { let mut counter = counter.lock().unwrap(); *counter += 1; // automatically unlocked } Rust Part VII > Advanced topics

Slide 70

Slide 70 text

2019/11/10 “Rust: Imperative Language 2.0” 70 Interior mutability with locks • RefCell, Mutex, RwLock all put T behind locks. • Only one can reference its contents at a time. Mutex i32 42 Part VII > Advanced topics

Slide 71

Slide 71 text

2019/11/10 “Rust: Imperative Language 2.0” 71 Interior mutability with locks • RefCell, Mutex, RwLock all put T behind locks. • Only one can reference its contents at a time. •→it still obeys shared XOR mutable rules. The container is shared. The contents are mutable. Part VII > Advanced topics

Slide 72

Slide 72 text

2019/11/10 “Rust: Imperative Language 2.0” 72 Interior mutability without locks • Cell, Atomic* can be considered to do lock-mutate- unlock at once. • These are like limited versions of RefCell / Mutex optimized for specific use-cases (from users’ point of view). Part VII > Advanced topics

Slide 73

Slide 73 text

2019/11/10 “Rust: Imperative Language 2.0” 73 Reference-counted pointers • Rc / Arc are reference-counted smart pointers. • Pointee are immutable by default (shared XOR mutable!) • Often combined with interior mutability. This is analogous to usual “references” in many other programming languages. • When used without interior mutability, this works like purely functional data structure.

Slide 74

Slide 74 text

2019/11/10 “Rust: Imperative Language 2.0” 74 Table of contents • Rust • Lifetime story • Shared XOR mutable • Ownership • Lifetimes • References as locks • Advanced topics Thank you for listening!