Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boosting Python With Rust 🚀

Boosting Python With Rust 🚀

RIIR makes much sense in the Python world. This session explores the development and distribution of Python apps written in Rust.

This point in time has seen the development of great toolings and major libraries switching to Rust runtimes. This provides concrete validation for adopting this approach.

The session also includes a case study of projects written in Rust as well as past approaches to the topic.

Abdur-Rahmaan Janhangeer

July 20, 2023
Tweet

More Decks by Abdur-Rahmaan Janhangeer

Other Decks in Programming

Transcript

  1. Boosting
    Python
    with Rust (TM)

    View full-size slide

  2. Python Mauritius Usergroup
    site
    fb
    linkedin
    mailing list
    4

    View full-size slide

  3. url
    pymug.com site
    5

    View full-size slide

  4. About me
    compileralchemy.com
    6

    View full-size slide

  5. Boosting
    Python
    with Rust (TM)
    8

    View full-size slide

  6. My relationship with Rust
    9

    View full-size slide

  7. Read the Rust book twice
    Read the Google rust training guide
    Know only about releasing scope and returning ownership
    Still don't know 3/4 of rust
    10

    View full-size slide

  8. Python / Rust friendliness
    12

    View full-size slide

  9. convention_of_methods
    readability
    emulates english
    13

    View full-size slide

  10. uses toml as default - toml invented by pypa core dev
    14

    View full-size slide

  11. build tool in py
    15

    View full-size slide

  12. iterators
    >>> x = itertools.cycle([1,2,3])
    >>> next(x)
    1
    >>> next(x)
    2
    >>> next(x)
    3
    >>> dir([1])
    [..., '__iter__', ...]
    16

    View full-size slide

  13. let nums = vec![1, 2, 3];
    for num in nums.iter() {
    println!("{}", num);
    }
    loop {
    match range.next() {
    Some(x) => {
    println!("{}", x);
    },
    None => { break }
    }
    }
    17

    View full-size slide

  14. consumers
    py: yield
    rust: ex .collect
    18

    View full-size slide

  15. string types
    b' '
    b' '
    19

    View full-size slide

  16. for in loop
    20

    View full-size slide

  17. for i, item in enumerate(bytes):
    if item == b' ':
    return i
    for (i, &item) in bytes.iter().enumerate() {
    if item == b' ' {
    return i;
    }
    }
    21

    View full-size slide

  18. Methods
    len(nums)
    nums.len()
    22

    View full-size slide

  19. -> return type
    def x() -> None: ...
    fn area(&self) -> u32 {
    self.width * self.height
    }
    23

    View full-size slide

  20. Pattern matching
    24

    View full-size slide

  21. def where_is(point):
    match point:
    case Point(x=0, y=0):
    print("Origin")
    case Point():
    print("Somewhere else")
    case _:
    print("Not a point")
    fn value_in_cents(coin: Coin) -> u8 {
    match coin {
    Coin::Penny => {
    println!("Lucky penny!");
    1
    }
    Coin::Nickel => 5,
    Coin::Dime => 10,
    Coin::Quarter => 25,
    }
    }
    25

    View full-size slide

  22. from typing import Dict, Generic, TypeVar
    T = TypeVar("T")
    class Registry(Generic[T]):
    def __init__(self) -> None:
    self._store: Dict[str, T] = {}
    fn largest(list: &[T]) -> &T
    27

    View full-size slide

  23. assert
    assert x == 1
    assert!(larger.can_hold(&smaller));
    28

    View full-size slide

  24. performance
    30

    View full-size slide

  25. threading (no GIL)
    31

    View full-size slide

  26. use rust crates
    32

    View full-size slide

  27. Py-Rust projects
    34

    View full-size slide

  28. Rustimport
    35

    View full-size slide

  29. ./
    termprint.rs
    // rustimport:pyo3
    use pyo3::prelude::*;
    #[pyfunction]
    pub fn boxaround(s: &str){
    println!("{}", "-".repeat(s.chars().count()+4));
    println!("| {} |", s);
    println!("{}", "-".repeat(s.chars().count()+4));
    }
    36

    View full-size slide

  30. $ python
    >>> import rustimport.import_hook
    >>> import termprint # This will pause for a moment to compile the module
    >>> termprint.boxaround('abc')
    -------
    | abc |
    -------
    37

    View full-size slide

  31. Protypting
    38

    View full-size slide

  32. $ python3 -m rustimport new my_single_file_extension.rs
    $ python3 -m rustimport new my_crate
    import my_crate
    39

    View full-size slide

  33. https://github.com/mityax/rustimport#usage-in-production
    40

    View full-size slide

  34. setuptools-rust
    41

    View full-size slide

  35. Poor docs
    most people no longer use it
    fit for binaries as well as modules
    see demo
    42

    View full-size slide

  36. .
    └── guessing-game
    ├── Cargo.lock
    ├── Cargo.toml
    ├── pyproject.toml
    └── src
    └── lib.rs
    44

    View full-size slide

  37. $ maturing develop
    $ maturin build
    $ maturin publish
    45

    View full-size slide

  38. https://github.com/indygreg/PyOxidizer
    Packaging python with Rust
    micro contrib
    46

    View full-size slide

  39. https://github.com/RustPython/RustPython
    Py interp in rust
    47

    View full-size slide

  40. https://github.com/PyO3/pyo3
    Rust bindings for python
    48

    View full-size slide

  41. https://pyo3.rs/v0.19.1/python_from_rust.html
    Python::with_gil(|py| {
    let fun: Py = PyModule::from_code(
    py,
    "def example(*args, **kwargs):
    if args != ():
    print('called with args', args)
    if kwargs != {}:
    print('called with kwargs', kwargs)
    if args == () and kwargs == {}:
    print('called with no arguments')",
    "",
    "",
    )?
    .getattr("example")?
    .into();
    ...
    })
    49

    View full-size slide

  42. use pyo3::prelude::*;
    Python::with_gil(|py| {
    let result = py
    .eval("[i * 10 for i in range(5)]", None, None)
    .map_err(|e| {
    e.print_and_set_sys_last_vars(py);
    })?;
    let res: Vec = result.extract().unwrap();
    assert_eq!(res, vec![0, 10, 20, 30, 40]);
    Ok(())
    })
    50

    View full-size slide

  43. Common Gotchas
    51

    View full-size slide

  44. slow
    dev mode, release
    wrong usage of rust
    52

    View full-size slide

  45. integer overflow
    53

    View full-size slide

  46. Python projects using Rust
    54

    View full-size slide

  47. Pydantic
    Rewrote v2 in Rust
    55

    View full-size slide

  48. Polars
    The fast Pandas alternative
    57

    View full-size slide

  49. Ruff
    The fastest py linter
    58

    View full-size slide

  50. Robyn
    async web server in Rust
    59

    View full-size slide

  51. compileralchemy.com
    [email protected]
    @osdotsystem
    github.com/abdur-rahmaanj
    Thank you
    60

    View full-size slide