Slide 1

Slide 1 text

piwheels: a Python package repository for Raspberry Pi Ben Nuttall twitter.com/ben_nuttall

Slide 2

Slide 2 text

@ben_nuttall Ben Nuttall ● Software engineer at BBC News Labs ● Formerly at Raspberry Pi Foundation ● Creator of gpiozero, piwheels and pyjokes ● Opensource.com correspondent ● twitter.com/ben_nuttall ● github.com/bennuttall

Slide 3

Slide 3 text

@ben_nuttall Why piwheels? ● Before piwheels, “pip install” on Raspberry Pi would take ages ● People don’t generally build Arm platform wheels ● Just x86/x86_64 for Linux, Mac & Windows ● I wondered if it would be possible to build a repository to host Arm wheels ● It is ● So I did

Slide 4

Slide 4 text

@ben_nuttall Uploading Pure Python package to PyPI Python version 2/3 Universal Python wheel Source (sdist)

Slide 5

Slide 5 text

@ben_nuttall Uploading Python package with C extensions to PyPI Python version 2.7 3.5 3.6 3.7 3.8 Windows (win32) Windows (amd64) Mac OS (x86_64) Linux (i686) Linux (x86_64) Source (sdist)

Slide 6

Slide 6 text

@ben_nuttall What is piwheels? ● Open source project maintained by: ● Ben Nuttall ● Dave Jones ● Tooling to automate building wheels of everything on PyPI ● Wheels are pre-compiled binary distributions of Python packages ● Tailored towards the Arm platform ● linux_armv6l and linux_armv7l platform tags

Slide 7

Slide 7 text

@ben_nuttall What is piwheels.org? ● Python package repository (like PyPI) ● Hosts Arm wheels of all packages on PyPI ● Website comprises: ● Simple index ● Project pages ● Blog & info pages

Slide 8

Slide 8 text

@ben_nuttall What is piwheels.org? Python version 3.4 3.5 3.7 linux_armv6l linux_armv7l

Slide 9

Slide 9 text

@ben_nuttall I assume you cross-compile? Nope

Slide 10

Slide 10 text

@ben_nuttall Mythic Beasts Pi cloud

Slide 11

Slide 11 text

@ben_nuttall The stack ● Master ● Distributes jobs, web server hosts index and wheel files ● Postgres database ● Monitors PyPI, generates build queue ● Logs build attempts, logs downloads ● Builders ● Get given build jobs ● 3 × Raspbian Jessie (Python 3.4) ● 2 × Raspbian Stretch (Python 3.5) ● 2 × Raspbian Buster (Python 3.7) ● Easy to scale up/down

Slide 12

Slide 12 text

@ben_nuttall The stack ● Master ● Distributes jobs, web server hosts index and wheel files ● Postgres database ● Monitors PyPI, generates build queue ● Logs build attempts, logs downloads ● Builders ● Get given build jobs ● 3 × Raspbian Jessie (Python 3.4) ● 2 × Raspbian Stretch (Python 3.5) ● 2 × Raspbian Buster (Python 3.7) ● Easy to scale up/down

Slide 13

Slide 13 text

@ben_nuttall piwheels builds

Slide 14

Slide 14 text

@ben_nuttall Using piwheels ● Raspbian is pre-configured to use piwheels.org as an additional index to PyPI ● Users get platform wheels without knowing it’s there ● Tutorials that say "warning: this step will take ages" are now wrong

Slide 15

Slide 15 text

@ben_nuttall Using piwheels ● Raspbian is pre-configured to use piwheels.org as an additional index to PyPI ● Users get platform wheels without knowing it’s there ● Tutorials that say "warning: this step will take ages" are now wrong

Slide 16

Slide 16 text

@ben_nuttall piwheels.org/simple/scipy/

Slide 17

Slide 17 text

@ben_nuttall piwheels.org/project/scipy/

Slide 18

Slide 18 text

@ben_nuttall How many downloads? ● 2018: 5 million ● 2019: 9 million ● 2020 so far: 4 million ● To date: 19 million ● March 2020: 1.2 million ● Yesterday: 40,000

Slide 19

Slide 19 text

@ben_nuttall How much time saved? ● 2018: 42 years ● 2019: 128 years ● 2020 so far: 76 years ● To date: 250 years ● March 2020: 22 years ● Yesterday: 262 days

Slide 20

Slide 20 text

@ben_nuttall Dependency calculation

Slide 21

Slide 21 text

@ben_nuttall Dependency calculation

Slide 22

Slide 22 text

@ben_nuttall What’s next for piwheels? ● Implement new PEPs ● Requires-Python ● Yank ● Deploy JSON API ● Package information at piwheels.org/project//json ● Keep fixing broken builds ● Get ready for Debian Bullseye (Python ~3.9) in 2021

Slide 23

Slide 23 text

@ben_nuttall piwheels ● piwheels.org ● github.com/piwheels ● twitter.com/piwheels ● blog.piwheels.org ● Ben: twitter.com/ben_nuttall ● Dave: twitter.com/waveform80