Rust for Python Native Extensions

Lightning talk presented at the Rust London Meetup in March 2016.

For an example of calling Rust from Python, check out https://github.com/maciejkula/python-rustlearn

Transcript

1. RUST FOR PYTHON NATIVE EXTENSIONS
2. PYTHON IS GREAT easy to learn batteries included huge package ecosystem
3. IT'S ALSO SLOW hugely dynamic interpreted
4. Which is ironic, because...
5. IT'S THE LANGUAGE FOR DATA SCIENCE/HPC most of deep learning numpy/scipy/scikit-learn pandas
6. EVERYTHING DONE THROUGH NATIVE EXTENSIONS C/C++ Fortran
7. Yes, you really need a Fortran compiler to do data science in Python.
8. THERE ARE SOME ALTERNATIVES
9. CYTHON A dialect of Python that compiles to C/C++. Has done great things for lowering barries to entry.
10. Need to care about memory allocation, dangling pointers STILL, YOU'RE WRITING C
11. ... and the generated code looks like this. static int __pyx_f_7_get_row_end( struct __pyx_obj_7 *__pyx_v_self, int __pyx_v_row) { int __pyx_r; Py_ssize_t __pyx_t_1; __pyx_t_1 = (__pyx_v_row + 1); __pyx_r = (*((int *) ( /* dim=0 */ ((char *) (((int *) __pyx_v_self->indptr.data) + __pyx_t_1)) ))); goto __pyx_L0;
12. And interfacing with Python is really easy. RUST CAN'T COME SOON ENOUGH
13. USE PYTHON CFFI import cffi ffi = cffi.FFI() ffi.cdef(""" struct Widget; struct Widget *make_widget(); int frobnicate(struct Widget* widget, float degree); """) lib = ffi.dlopen(so_path) widget = lib.make_widget() lib.frobnicate(widget, 0.5)
14. THE RUST SIDE IS JUST THE C ABI pub struct Widget { // snip } #[no_mangle] pub fn frobnicate(widget_ptr: *mut Widget, degree: f32) -> i32 { let widget = unsafe { Box::from_raw(widget_ptr) }; // snip mem::forget(widget); // If you want to keep it around 42 }
15. IT ALL WORKS BEAUTIFULLY.
16. WE HAVE A RUST ML MODEL IN PRODUCTION AT LYST Based on rustlearn, a Rust ML library Integrates seamlessly into our Python stack. We pre-build the binaries in our CI environment, so no Rust necessary on servers Runs some of our recommendation services
17. SPREAD THE WORD!
18. @Maciej_Kula THANKS FOR LISTENING