Rusty Python

Rust, a new language for our programmer too belt.

Presented at PyCon Otto, 6-7 April 2017, Florence https://www.pycon.it/conference/talks/rusty-python

Interactive slides: http://slides.com/naufraghi/pycon-otto

Transcript

1. Rusty Python Rust, a new language for our programmer tool belt Matteo Bertini @naufraghi PyCon Otto | 6-9 April 2017 | Florence 1
2. Rust Born as a personal project started in 2006 by Mozilla employee Graydon Hoare Mozilla began sponsoring the project in 2009 and announced it in 2010 A young language, with a growing usage in production (Dropbox, Mozilla, Canonical, OVH, ...) 2 . 1
3. 2 . 2
4. $ cargo new hello-world --bin Created binary (application) `hello-world` project $ cd hello-world/ $ tree . ├── Cargo.toml └── src └── main.rs 1 directory, 2 files Hello, World! install the Rust toolchain with `rustup` 3
5. Hello, World! fn main() { println!("Hello, World!"); } huhu? `println!`? 4
6. How we run it? `cargo run` $ cargo run Compiling hello-world v0.1.0 (file:///home/naufraghi/.../hello-world) Finished dev [unoptimized + debuginfo] target(s) in 0.41 secs Running `target/debug/hello-world` Hello world! 5
7. Docstrings? //! Programma di esempio per _PyCon Otto_, //! l'immancabile «Hello, World!». fn main() { println!("Hello, World!"); } `cargo doc --open` 6
8. Integrated docs Cool, but in Python have `doctest`, I can't live without it! 7
9. And Rust "borrowed" the idea! //! Libreria di esempio per _PyCon Otto_, //! l'immancabile «Hello, World!». //! //! ``` //! use hello_world_lib::greater; //! let message = greater("World"); //! assert_eq!(message, "Hello, World!"); //! ``` pub fn greater(name: &str) -> String { format!("Hello, {}!", name) } in fact a lot of ideas from Python 8
10. Run `cargo test --doc` $ cargo test --doc Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs Doc-tests hello-world-lib running 1 test test greater_0 ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured 9
11. Let me see that code again... pub fn greater(name: &str) -> String { format!("Hello, {}!", name) } Where is the `return`? It's implicit, the last expression is returned ... beware of forgetting some `return` in Python after a switch :) 10
12. Optional arguments pub fn greater2(optional_name: Option<&str>) -> String { //! Restituisce un saluto di default se il nome è None //! //! ``` //! use hello_world_lib::greater2; //! let message = greater2(None); //! assert_eq!(message, "Hello, World!"); //! //! let message = greater2(Some("Matteo")); //! assert_eq!(message, "Hello, Matteo!"); //! ``` let name = match optional_name { Some(name) => name, None => "World", }; format!("Hello, {}!", name) } no default values... yet `if` and `match`are expressions in Rust 11
13. Managing None var = some_optional_value() if var is not None: res = do_something_with(var) else: res = None res = if let Some(var) = some_optional_value() { Some(do_something_with(var)) } else { None } Python Rust 12
14. `None` is a serious thing fn main() { let out = if let Some(value) = some_optional_value("some") { do_something_with(value) } else { None }; println!("out: {:?}", out); } 13
15. Nice error messages 14 . 1
16. Rust Playground https://is.gd/WIkAma 14 . 2
17. In Rust we have other ways some_optional_value().map(do_something_with) pub enum Option<T> { None, Some(T), } `Option::map` will not call `do_something_with` if the value is `None` 15
18. `Option` docs 16
19. Methods struct Circle { x: f64, y: f64, radius: f64, } impl Circle { fn area(&self) -> f64 { std::f64::consts::PI * (self.radius * self.radius) } } self | &self | &mut self 17
20. Traits struct Circle { x: f64, y: f64, radius: f64, } trait HasArea { fn area(&self) -> f64; } impl HasArea for Circle { fn area(&self) -> f64 { std::f64::consts::PI * (self.radius * self.radius) } } this is what your lib users will need to implement 18
21. Duck typing def print_area(shape) { print("This shape has an area of {}".format(shape.area())); } Traits are something we can use where in Python we were using duck typing fn print_area<T: HasArea>(shape: T) { println!("This shape has an area of {}", shape.area()); } Python Rust 19
22. Implementing Traits struct Counter { count: usize, } impl Counter { fn new() -> Counter { Counter { count: 0 } } } impl Iterator for Counter { type Item = usize; fn nxt(&mut self) -> Option<Self::Item> { self.count += 1; Some(self.count) } } Traits are somewhat similar to the `collections.abc` in Python class Counter(abc.Iterator): def __init__(self): self.count = 0 def __nxt__(self): self.count += 1 return self.count with a typo in both 20
23. Implementing Traits >>> c = Counter() Traceback (most recent call last): File "<ipython-input-29-f3cf168481b9>", line 1, in <module> c = Counter() TypeError: Can't instantiate abstract class Counter with abstract methods __next__ Runtime check error[E0407]: method `nxt` is not a member of trait `Iterator` --> <anon>:11:5 | 11 | fn nxt(&mut self) -> Option<Self::Item> { | _____^ starting here... 12 | | self.count += 1; 13 | | Some(self.count) 14 | | } | |_____^ ...ending here: not a member of trait `Iterator` error[E0046]: not all trait items implemented, missing: `next` --> <anon>:9:1 | 9 | impl Iterator for Counter { | _^ starting here... 10 | | type Item = usize; 11 | | fn nxt(&mut self) -> Option<Self::Item> { 12 | | self.count += 1; 13 | | Some(self.count) 14 | | } 15 | | } | |_^ ...ending here: missing `next` in implementation | = note: `next` from trait: `fn(&mut Self) -> std::option::Option<<Self as std::iter::Iterator>::Item>` Com pile tim e check 21
24. We have barely seen Rust as a language Rust is not easy, there are a lot of concepts, one can learn step-by-step. The tooling is good The docs are good Let's use Rust! 22
25. Let's rewrite Electric Sheep Fly down! Paper on http://flam3.com/flame_draves.pdf, 41 pages 23
26. / a b \ / x \ / e \ f(x,y) = | | · | | + | | \ c d / \ y / \ f / Given 4 linear function with some magic parameters (a,b,c,d,e,f) Choose a random starting point, say (0,0) Loop: pick a random function apply the func on the last point collect the result Iterated Function System Barnsley fern 24
27. Iterated Function System F (x, y) ∼ w 1 1 F (x, y) ∼ w 2 2 F (x, y) ∼ w 3 3 F (x, y) ∼ w 4 4 p = F (p ) i+1 k i Plot the histogram Take one at random and collect the points 25 . 1
28. Python for i in range(iterations): f = np.random.choice(arange(4), p=weights) if f == 0: P = F1 * P + F1c elif f == 1: P = F2 * P + F2c elif f == 2: P = F3 * P + F3c else: P = F4 * P + F4c if i > pointsToSkip: drawPoints[frame][i-pointsToSkip] = squeeze(asarray(P)) 25 . 2
29. Rust for i in 0..iterations + 1 { let f = wc.ind_sample(&mut rng); if f == 1 { p = f1 * p + f1c; } else if f == 2 { p = f2 * p + f2c; } else if f == 3 { p = f3 * p + f3c; } else { p = f4 * p + f4c; } if i > points_to_skip { points.push((p.x, p.y)); } } 25 . 3
30. How does it work? 26 . 1
31. Build the extension $ python setup.py develop running develop running egg_info writing top-level names to ifs.egg-info/top_level.txt writing dependency_links to ifs.egg-info/dependency_links.txt writing ifs.egg-info/PKG-INFO reading manifest file 'ifs.egg-info/SOURCES.txt' writing manifest file 'ifs.egg-info/SOURCES.txt' running build_ext running build_rust cargo rustc --lib --manifest-path Cargo.toml \ --features cpython/extension-module cpython/python3-sys \ --release -- --crate-type cdylib Finished release [optimized] target(s) in 0.0 secs Creating /home/naufraghi/.../python3.5/site-packages/ifs.egg-link (link to .) ifs 1.0 is already the active version in easy-install.pth Installed /home/naufraghi/Documents/src/rusty-python-pyconotto/barnsley-fern Processing dependencies for ifs==1.0 Finished processing dependencies for ifs==1.0 26 . 2
32. Generating the fern $ python fern.py Using Python Frame 0 generated ... Frame 11 generated Fern data generation completed in 23.3s Rendered frame 0000 ... Rendered frame 0011 Converting to animated gif... Gif Generated 26 . 3
33. Generating the fern $ python fern.py rust Using Rust Frame 0 generated ... Frame 11 generated Fern data generation completed in 0.4s Rendered frame 0000 ... Rendered frame 0011 Converting to animated gif... Gif Generated 26 . 4
34. Generating the fern $ python fern.py rust Using Rust Frame 0 generated ... Frame 11 generated Fern data generation completed in 3.3s Rendered frame 0000 ... Rendered frame 0011 Converting to animated gif... Gif Generated image size 300 -> 600 iterations 50000 -> 500000 26 . 5
35. Python wrapper use cpython::{PyResult, Python}; use barnsley_fern; py_module_initializer!(_iterated_function_systems, _inititerated_function_systems, PyInit__iterated_function_systems, |py, m| { try!(m.add(py, "__doc__", "Rust iterated function systems")); try!(m.add(py, "barnsley_fern", py_fn!(py, py_barnsley_fern(iterations: usize, points_to_skip: usize, bending: f64)))); Ok(()) }); fn py_barnsley_fern(py: Python, iterations: usize, points_to_skip: usize, bending: f64) -> PyResult<Vec<(f64, f64)>> { Ok(barnsley_fern(iterations, points_to_skip, bending)) } 27
36. Python wrapper rust-cpython has an impl for most builtin types 28
37. setup.py Cargo.toml from setuptools import setup from setuptools_rust import RustExtension setup(name='ifs', version='1.0', rust_extensions=[ RustExtension('ifs._iterated_function_systems', 'Cargo.toml', debug=False)], packages=['ifs'], # rust extensions are not zip safe, just like C-extensions. zip_safe=False) [package] name = "ifs" version = "0.1.0" authors = ["Matteo Bertini <matteo@naufraghi.net>"] [lib] name = "iterated_function_systems" crate-type = ["cdylib"] [dependencies] nalgebra = "0.11.2" rand = "0.3.15" cpython = "0.1" 29
38. [tox] envlist = py27, py35 [testenv] commands = python setup.py install pip install example/ py.test deps = pytest matrix: allow_failures: os: osx include: - language: generic os: osx python: 2.7 env: - RUST_VERSION=stable - language: python python: 2.7 env: - RUST_VERSION=stable ... # Manually install python on osx before_install: - if [[ $TRAVIS_OS_NAME == 'osx' ]]; then ci/osx-install.sh; fi install: - curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUST_VERSION - export PATH="$HOME/.cargo/bin:$PATH" - rustc -V script: - sudo pip install tox - tox CI up and running 30
39. Take away Rust seems to be here to stay Rust is high level enough Integrating Rust in Python is easy We can write fast and safe Rust extensions! 31
40. Thanks! Questions? 32 . 1
41. Other choices Cython C: CPython / ffi c++: SIP (PyQt) c++: pybind11 (modern boost::python) fortran: f2py Rust: CPython / ffi 32 . 2
42. Resources https://www.rust-lang.org/ https://play.rust-lang.org/ https://doc.rust-lang.org/book/ https://github.com/dgrunwald/rust-cpython https://github.com/fafhrd91/setuptools-rust http://flam3.com/flame_draves.pdf https://github.com/marceloramires/FractalRendering 32 . 3