$30 off During Our Annual Pro Sale. View Details »

piwheels: a Python package repository for Raspberry Pi

piwheels: a Python package repository for Raspberry Pi

Lightning talk given at EuroPython 2020 (online)

Ben Nuttall

July 23, 2020
Tweet

More Decks by Ben Nuttall

Other Decks in Programming

Transcript

  1. piwheels: a Python package repository for Raspberry Pi
    Ben Nuttall
    twitter.com/ben_nuttall

    View Slide

  2. @ben_nuttall
    Ben Nuttall

    Software engineer at BBC News Labs

    Formerly at Raspberry Pi Foundation

    Creator of gpiozero, piwheels and pyjokes

    Opensource.com correspondent

    twitter.com/ben_nuttall

    github.com/bennuttall

    View Slide

  3. @ben_nuttall
    Why piwheels?

    Before piwheels, “pip install” on Raspberry Pi
    would take ages

    People don’t generally build Arm platform
    wheels

    Just x86/x86_64 for Linux, Mac & Windows

    I wondered if it would be possible to build a
    repository to host Arm wheels

    It is

    So I did

    View Slide

  4. @ben_nuttall
    Uploading Pure Python package to PyPI
    Python version 2/3
    Universal
    Python wheel
    Source (sdist)

    View Slide

  5. @ben_nuttall
    Uploading Python package with C extensions to PyPI
    Python version 2.7 3.5 3.6 3.7 3.8
    Windows (win32)
    Windows (amd64)
    Mac OS (x86_64)
    Linux (i686)
    Linux (x86_64)
    Source (sdist)

    View Slide

  6. @ben_nuttall
    What is piwheels?

    Open source project maintained by:

    Ben Nuttall

    Dave Jones

    Tooling to automate building wheels of
    everything on PyPI

    Wheels are pre-compiled binary distributions of
    Python packages

    Tailored towards the Arm platform

    linux_armv6l and linux_armv7l platform tags

    View Slide

  7. @ben_nuttall
    What is piwheels.org?

    Python package repository (like PyPI)

    Hosts Arm wheels of all packages on PyPI

    Website comprises:

    Simple index

    Project pages

    Blog & info pages

    View Slide

  8. @ben_nuttall
    What is piwheels.org?
    Python version 3.4 3.5 3.7
    linux_armv6l
    linux_armv7l

    View Slide

  9. @ben_nuttall
    I assume you cross-compile?
    Nope

    View Slide

  10. @ben_nuttall
    Mythic Beasts Pi cloud

    View Slide

  11. @ben_nuttall
    The stack

    Master

    Distributes jobs, web server hosts index and wheel files

    Postgres database

    Monitors PyPI, generates build queue

    Logs build attempts, logs downloads

    Builders

    Get given build jobs

    3 × Raspbian Jessie (Python 3.4)

    2 × Raspbian Stretch (Python 3.5)

    2 × Raspbian Buster (Python 3.7)

    Easy to scale up/down

    View Slide

  12. @ben_nuttall
    The stack

    Master

    Distributes jobs, web server hosts index and wheel files

    Postgres database

    Monitors PyPI, generates build queue

    Logs build attempts, logs downloads

    Builders

    Get given build jobs

    3 × Raspbian Jessie (Python 3.4)

    2 × Raspbian Stretch (Python 3.5)

    2 × Raspbian Buster (Python 3.7)

    Easy to scale up/down

    View Slide

  13. @ben_nuttall
    piwheels builds

    View Slide

  14. @ben_nuttall
    Using piwheels

    Raspbian is pre-configured to use
    piwheels.org as an additional index to
    PyPI

    Users get platform wheels without
    knowing it’s there

    Tutorials that say "warning: this step
    will take ages" are now wrong

    View Slide

  15. @ben_nuttall
    Using piwheels

    Raspbian is pre-configured to use
    piwheels.org as an additional index to
    PyPI

    Users get platform wheels without
    knowing it’s there

    Tutorials that say "warning: this step
    will take ages" are now wrong

    View Slide

  16. @ben_nuttall
    piwheels.org/simple/scipy/

    View Slide

  17. @ben_nuttall
    piwheels.org/project/scipy/

    View Slide

  18. @ben_nuttall
    2019 stats

    9 million downloads

    128 years saved

    View Slide

  19. @ben_nuttall
    piwheels

    piwheels.org

    github.com/piwheels

    twitter.com/piwheels

    blog.piwheels.org

    Ben: twitter.com/ben_nuttall

    Dave: twitter.com/waveform80

    View Slide