Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rust for Python Native Extensions

Rust for Python Native Extensions

Lightning talk presented at the Rust London Meetup in March 2016.

For an example of calling Rust from Python, check out https://github.com/maciejkula/python-rustlearn

F401afbfb8100568304f0caf45c79575?s=128

Maciej Kula

March 17, 2016
Tweet

Transcript

  1. RUST FOR PYTHON NATIVE EXTENSIONS

  2. PYTHON IS GREAT easy to learn batteries included huge package

    ecosystem
  3. IT'S ALSO SLOW hugely dynamic interpreted

  4. Which is ironic, because...

  5. IT'S THE LANGUAGE FOR DATA SCIENCE/HPC most of deep learning

    numpy/scipy/scikit-learn pandas
  6. EVERYTHING DONE THROUGH NATIVE EXTENSIONS C/C++ Fortran

  7. Yes, you really need a Fortran compiler to do data

    science in Python.
  8. THERE ARE SOME ALTERNATIVES

  9. CYTHON A dialect of Python that compiles to C/C++. Has

    done great things for lowering barries to entry.
  10. Need to care about memory allocation, dangling pointers STILL, YOU'RE

    WRITING C
  11. ... and the generated code looks like this. static int

    __pyx_f_7_get_row_end( struct __pyx_obj_7 *__pyx_v_self, int __pyx_v_row) { int __pyx_r; Py_ssize_t __pyx_t_1; __pyx_t_1 = (__pyx_v_row + 1); __pyx_r = (*((int *) ( /* dim=0 */ ((char *) (((int *) __pyx_v_self->indptr.data) + __pyx_t_1)) ))); goto __pyx_L0;
  12. And interfacing with Python is really easy. RUST CAN'T COME

    SOON ENOUGH
  13. USE PYTHON CFFI import cffi ffi = cffi.FFI() ffi.cdef(""" struct

    Widget; struct Widget *make_widget(); int frobnicate(struct Widget* widget, float degree); """) lib = ffi.dlopen(so_path) widget = lib.make_widget() lib.frobnicate(widget, 0.5)
  14. THE RUST SIDE IS JUST THE C ABI pub struct

    Widget { // snip } #[no_mangle] pub fn frobnicate(widget_ptr: *mut Widget, degree: f32) -> i32 { let widget = unsafe { Box::from_raw(widget_ptr) }; // snip mem::forget(widget); // If you want to keep it around 42 }
  15. IT ALL WORKS BEAUTIFULLY.

  16. WE HAVE A RUST ML MODEL IN PRODUCTION AT LYST

    Based on rustlearn, a Rust ML library Integrates seamlessly into our Python stack. We pre-build the binaries in our CI environment, so no Rust necessary on servers Runs some of our recommendation services
  17. SPREAD THE WORD!

  18. @Maciej_Kula THANKS FOR LISTENING