piwheels: a Python package repository for Raspberry Pi

Lightning talk given at EuroPython 2020 (online)

Transcript

1. piwheels: a Python package repository for Raspberry Pi
   Ben Nuttall
   twitter.com/ben_nuttall
   

   View Slide

2. @ben_nuttall
   Ben Nuttall
   ●
   Software engineer at BBC News Labs
   ●
   Formerly at Raspberry Pi Foundation
   ●
   Creator of gpiozero, piwheels and pyjokes
   ●
   Opensource.com correspondent
   ●
   twitter.com/ben_nuttall
   ●
   github.com/bennuttall
   

   View Slide

3. @ben_nuttall
   Why piwheels?
   ●
   Before piwheels, “pip install” on Raspberry Pi
   would take ages
   ●
   People don’t generally build Arm platform
   wheels
   ●
   Just x86/x86_64 for Linux, Mac & Windows
   ●
   I wondered if it would be possible to build a
   repository to host Arm wheels
   ●
   It is
   ●
   So I did
   

   View Slide

4. @ben_nuttall
   Uploading Pure Python package to PyPI
   Python version 2/3
   Universal
   Python wheel
   Source (sdist)
   

   View Slide

5. @ben_nuttall
   Uploading Python package with C extensions to PyPI
   Python version 2.7 3.5 3.6 3.7 3.8
   Windows (win32)
   Windows (amd64)
   Mac OS (x86_64)
   Linux (i686)
   Linux (x86_64)
   Source (sdist)
   

   View Slide

6. @ben_nuttall
   What is piwheels?
   ●
   Open source project maintained by:
   ●
   Ben Nuttall
   ●
   Dave Jones
   ●
   Tooling to automate building wheels of
   everything on PyPI
   ●
   Wheels are pre-compiled binary distributions of
   Python packages
   ●
   Tailored towards the Arm platform
   ●
   linux_armv6l and linux_armv7l platform tags
   

   View Slide

7. @ben_nuttall
   What is piwheels.org?
   ●
   Python package repository (like PyPI)
   ●
   Hosts Arm wheels of all packages on PyPI
   ●
   Website comprises:
   ●
   Simple index
   ●
   Project pages
   ●
   Blog & info pages
   

   View Slide

8. @ben_nuttall
   What is piwheels.org?
   Python version 3.4 3.5 3.7
   linux_armv6l
   linux_armv7l
   

   View Slide

9. @ben_nuttall
   I assume you cross-compile?
   Nope
   

   View Slide

10. @ben_nuttall
    Mythic Beasts Pi cloud
    

    View Slide

11. @ben_nuttall
    The stack
    ●
    Master
    ●
    Distributes jobs, web server hosts index and wheel files
    ●
    Postgres database
    ●
    Monitors PyPI, generates build queue
    ●
    Logs build attempts, logs downloads
    ●
    Builders
    ●
    Get given build jobs
    ●
    3 × Raspbian Jessie (Python 3.4)
    ●
    2 × Raspbian Stretch (Python 3.5)
    ●
    2 × Raspbian Buster (Python 3.7)
    ●
    Easy to scale up/down
    

    View Slide

12. @ben_nuttall
    The stack
    ●
    Master
    ●
    Distributes jobs, web server hosts index and wheel files
    ●
    Postgres database
    ●
    Monitors PyPI, generates build queue
    ●
    Logs build attempts, logs downloads
    ●
    Builders
    ●
    Get given build jobs
    ●
    3 × Raspbian Jessie (Python 3.4)
    ●
    2 × Raspbian Stretch (Python 3.5)
    ●
    2 × Raspbian Buster (Python 3.7)
    ●
    Easy to scale up/down
    

    View Slide

13. @ben_nuttall
    piwheels builds
    

    View Slide

14. @ben_nuttall
    Using piwheels
    ●
    Raspbian is pre-configured to use
    piwheels.org as an additional index to
    PyPI
    ●
    Users get platform wheels without
    knowing it’s there
    ●
    Tutorials that say "warning: this step
    will take ages" are now wrong
    

    View Slide

15. @ben_nuttall
    Using piwheels
    ●
    Raspbian is pre-configured to use
    piwheels.org as an additional index to
    PyPI
    ●
    Users get platform wheels without
    knowing it’s there
    ●
    Tutorials that say "warning: this step
    will take ages" are now wrong
    

    View Slide

16. @ben_nuttall
    piwheels.org/simple/scipy/
    

    View Slide

17. @ben_nuttall
    piwheels.org/project/scipy/
    

    View Slide

18. @ben_nuttall
    2019 stats
    ●
    9 million downloads
    ●
    128 years saved
    

    View Slide

19. @ben_nuttall
    piwheels
    ●
    piwheels.org
    ●
    github.com/piwheels
    ●
    twitter.com/piwheels
    ●
    blog.piwheels.org
    ●
    Ben: twitter.com/ben_nuttall
    ●
    Dave: twitter.com/waveform80
    

    View Slide