hyperjson - the journey towards faster, safer JSON parsing in Python using Rust

hyperjson - the journey towards faster, safer JSON parsing in Python using Rust

During a recent Python Hackathon in Düsseldorf, Matthias somehow managed to sneak in some Rust code while writing a fast, safe Python module for encoding and decoding JSON. It passes (most of) the Python test suite for the json module and was unreasonably pleasant to write. Listen carefully as he tells the story of a little side-project that got out of hand and how Rust can help speed up even the most boring, highly-optimized tasks like parsing file formats in the future.

Ed9e9992fe069a0d3de05b69d8d187c3?s=128

Matthias Endler

August 01, 2018
Tweet

Transcript

  1. None
  2. hydrojson

  3. EXPECTATION... REALITY...

  4. None
  5. "Hey, I wrote a JSON encoder!"

  6. None
  7. https://www.cartoonstock.com/cartoonview.asp?catref=rdin355

  8. None
  9. http://seriot.ch/parsing_json.php

  10. https://bugs.python.org/issue21529

  11. None
  12. None
  13. Parsing JSON - Hieronymus Bosch, 1501

  14. None
  15. None
  16. None
  17. None
  18. None
  19. None
  20. ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- JSON 3 0

    0 1087568 Python 15 404 148 1331 Rust 2 63 65 468 Markdown 1 46 0 107 Bourne Shell 4 17 7 54 make 1 9 0 32 YAML 1 6 2 23 TOML 1 5 0 20 ------------------------------------------------------------------------------- SUM: 28 550 222 1089603 -------------------------------------------------------------------------------
  21. >>> import lenrs >>> len("hello") 5 >>> len(["this", "is", "a",

    "test"]) 4 >>> len(1) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'int' has no len()
  22. #![feature(proc_macro)] #![feature(proc_macro_path_invoc)] extern crate pyo3; use pyo3::prelude::*; #[py::modinit(_lenrs)] fn init(py:

    Python, m: &PyModule) -> PyResult<()> { #[pyfn(m, "len")] fn len(py: Python, obj: PyObject) -> PyResult<PyObject> { if let Ok(s) = obj.extract::<String>(py) { return Ok(s.len().to_object(py)); } if let Ok(v) = obj.extract::<Vec<String>>(py) { return Ok(v.len().to_object(py)); } Err(PyErr::new::<pyo3::exc::TypeError, _>("Not supported")) } Ok(()) } Requires nightly Rust Module name in Python Function name in Python
  23. Write your own Python Extension in Rust!
 
 https://www.youtube.com/watch?v=D9r__qxtRMQ

  24. None