piwheels: a Python package repository for Raspberry Pi

piwheels: a Python package repository for Raspberry Pi

Lightning talk given at EuroPython 2020 (online)

A1995c9abe48450ce2f82d93ca0b863f?s=128

Ben Nuttall

July 23, 2020
Tweet

Transcript

  1. piwheels: a Python package repository for Raspberry Pi Ben Nuttall

    twitter.com/ben_nuttall
  2. @ben_nuttall Ben Nuttall • Software engineer at BBC News Labs

    • Formerly at Raspberry Pi Foundation • Creator of gpiozero, piwheels and pyjokes • Opensource.com correspondent • twitter.com/ben_nuttall • github.com/bennuttall
  3. @ben_nuttall Why piwheels? • Before piwheels, “pip install” on Raspberry

    Pi would take ages • People don’t generally build Arm platform wheels • Just x86/x86_64 for Linux, Mac & Windows • I wondered if it would be possible to build a repository to host Arm wheels • It is • So I did
  4. @ben_nuttall Uploading Pure Python package to PyPI Python version 2/3

    Universal Python wheel Source (sdist)
  5. @ben_nuttall Uploading Python package with C extensions to PyPI Python

    version 2.7 3.5 3.6 3.7 3.8 Windows (win32) Windows (amd64) Mac OS (x86_64) Linux (i686) Linux (x86_64) Source (sdist)
  6. @ben_nuttall What is piwheels? • Open source project maintained by:

    • Ben Nuttall • Dave Jones • Tooling to automate building wheels of everything on PyPI • Wheels are pre-compiled binary distributions of Python packages • Tailored towards the Arm platform • linux_armv6l and linux_armv7l platform tags
  7. @ben_nuttall What is piwheels.org? • Python package repository (like PyPI)

    • Hosts Arm wheels of all packages on PyPI • Website comprises: • Simple index • Project pages • Blog & info pages
  8. @ben_nuttall What is piwheels.org? Python version 3.4 3.5 3.7 linux_armv6l

    linux_armv7l
  9. @ben_nuttall I assume you cross-compile? Nope

  10. @ben_nuttall Mythic Beasts Pi cloud

  11. @ben_nuttall The stack • Master • Distributes jobs, web server

    hosts index and wheel files • Postgres database • Monitors PyPI, generates build queue • Logs build attempts, logs downloads • Builders • Get given build jobs • 3 × Raspbian Jessie (Python 3.4) • 2 × Raspbian Stretch (Python 3.5) • 2 × Raspbian Buster (Python 3.7) • Easy to scale up/down
  12. @ben_nuttall The stack • Master • Distributes jobs, web server

    hosts index and wheel files • Postgres database • Monitors PyPI, generates build queue • Logs build attempts, logs downloads • Builders • Get given build jobs • 3 × Raspbian Jessie (Python 3.4) • 2 × Raspbian Stretch (Python 3.5) • 2 × Raspbian Buster (Python 3.7) • Easy to scale up/down
  13. @ben_nuttall piwheels builds

  14. @ben_nuttall Using piwheels • Raspbian is pre-configured to use piwheels.org

    as an additional index to PyPI • Users get platform wheels without knowing it’s there • Tutorials that say "warning: this step will take ages" are now wrong
  15. @ben_nuttall Using piwheels • Raspbian is pre-configured to use piwheels.org

    as an additional index to PyPI • Users get platform wheels without knowing it’s there • Tutorials that say "warning: this step will take ages" are now wrong
  16. @ben_nuttall piwheels.org/simple/scipy/

  17. @ben_nuttall piwheels.org/project/scipy/

  18. @ben_nuttall 2019 stats • 9 million downloads • 128 years

    saved
  19. @ben_nuttall piwheels • piwheels.org • github.com/piwheels • twitter.com/piwheels • blog.piwheels.org

    • Ben: twitter.com/ben_nuttall • Dave: twitter.com/waveform80