Upgrade to Pro — share decks privately, control downloads, hide ads and more …

piwheels lightning talk

piwheels lightning talk

Lightning talk on my project 'piwheels' given at EuroPython 2017


Ben Nuttall

July 11, 2017


  1. A title would give away the talk contents • I’m

    Ben Nuttall • I work for the Raspberry Pi Foundation • I do a lot of Python on Raspberry Pi
  2. pip install numpy • Browse http://pypi.python.org/simple/numpy • Look for a

    wheel file with a platform tag matching the current platform – If one is found, download and install :) – If no match is found, look for source tarball/zip • If source is found, download and build from source :O • If no source is found, fail :(
  3. Which type of x86/x86_64 are you?

  4. ...I have them all

  5. Actually I’m linux_armv7l

  6. What? I’ve never heard of it. Here’s the source, build

    it yourself! • PyPI doesn’t support uploading ARM wheels :( • “pip install numpy” takes: – ~20 mins on Pi 3 (1.2GHz quad-core) – ~2.5 hours on Pi 1 (700MHz single-core)
  7. Fine! I’ll build my own package repository... • cd Projects

    • mkdir piwheels • cd piwheels • git init
  8. piwheels v1 • Pi 3 in my living room •

    Build the latest version of every package (106k packages) • Log output into postgres database • Host a package repository on the same Pi • On GitHub but not reproducable
  9. piwheels v1 • It took 10 days to complete the

    build run • 76% build success rate • Repository live at piwheels.bennuttall.com • “pip install numpy ­i http://piwheels.bennuttall.com” works and takes 6 seconds :)
  10. piwheels v2 • Pi 3 in a data centre (Thanks

    Mythic Beasts) • Build every version of every package (740k releases) • Host a package repository on the same Pi • Repository live at www.piwheels.hostedpi.com • Test suite, installation instructions & developer documentation on GitHub: – github.com/bennuttall/piwheels – Help! Contribute!
  11. In the future • Complete the build backlog (in progress)

    • Continue to build all new releases • Try to fix failed builds (missing dependencies, etc) • Rebuild Python 3.4 wheels for 3.5 & 3.6 • Add the piwheels server to pip config in Raspbian (our distro) as an additional index – Users get wheels for free without needing to know about it
  12. pypi.org • Next generation PyPI project • A Google developer

    came across piwheels and filed an issue with the pypi.org project (warehouse) – github.com/pypa/warehouse/issues/2003 • pypi.org now supports uploading ARM wheels :) – Thanks @kpayson64 and @dstufft • Package maintainers can upload wheels built by piwheels to pypi.org and they appear on pypi.python.org – \o/