Upgrade to Pro — share decks privately, control downloads, hide ads and more …

piwheels lightning talk

piwheels lightning talk

Lightning talk on my project 'piwheels' given at EuroPython 2017

Ben Nuttall

July 11, 2017
Tweet

More Decks by Ben Nuttall

Other Decks in Programming

Transcript

  1. A title would give away the talk contents • I’m

    Ben Nuttall • I work for the Raspberry Pi Foundation • I do a lot of Python on Raspberry Pi
  2. pip install numpy • Browse http://pypi.python.org/simple/numpy • Look for a

    wheel file with a platform tag matching the current platform – If one is found, download and install :) – If no match is found, look for source tarball/zip • If source is found, download and build from source :O • If no source is found, fail :(
  3. Which type of x86/x86_64 are you?

  4. ...I have them all

  5. Actually I’m linux_armv7l

  6. What? I’ve never heard of it. Here’s the source, build

    it yourself! • PyPI doesn’t support uploading ARM wheels :( • “pip install numpy” takes: – ~20 mins on Pi 3 (1.2GHz quad-core) – ~2.5 hours on Pi 1 (700MHz single-core)
  7. Fine! I’ll build my own package repository... • cd Projects

    • mkdir piwheels • cd piwheels • git init
  8. piwheels v1 • Pi 3 in my living room •

    Build the latest version of every package (106k packages) • Log output into postgres database • Host a package repository on the same Pi • On GitHub but not reproducable
  9. piwheels v1 • It took 10 days to complete the

    build run • 76% build success rate • Repository live at piwheels.bennuttall.com • “pip install numpy ­i http://piwheels.bennuttall.com” works and takes 6 seconds :)
  10. piwheels v2 • Pi 3 in a data centre (Thanks

    Mythic Beasts) • Build every version of every package (740k releases) • Host a package repository on the same Pi • Repository live at www.piwheels.hostedpi.com • Test suite, installation instructions & developer documentation on GitHub: – github.com/bennuttall/piwheels – Help! Contribute!
  11. In the future • Complete the build backlog (in progress)

    • Continue to build all new releases • Try to fix failed builds (missing dependencies, etc) • Rebuild Python 3.4 wheels for 3.5 & 3.6 • Add the piwheels server to pip config in Raspbian (our distro) as an additional index – Users get wheels for free without needing to know about it
  12. pypi.org • Next generation PyPI project • A Google developer

    came across piwheels and filed an issue with the pypi.org project (warehouse) – github.com/pypa/warehouse/issues/2003 • pypi.org now supports uploading ARM wheels :) – Thanks @kpayson64 and @dstufft • Package maintainers can upload wheels built by piwheels to pypi.org and they appear on pypi.python.org – \o/