Share
Embed
Transcript
14 slides
-
PyPI – the Python package index • Install Python libraries
and applications with “pip install” • Python packages can be implemented in Python or in C – gpiozero is implemented in Python – numpy is implemented in C • Packages implemented in Python do not require building: – Download source – Install • Packages implemented in C require building: – Download source – Build – Install -
The problem • Packages implemented in C can take a
long time to build • “pip install numpy” takes: – 30 mins on Pi 3 – 2.5 hours on Pi 1 -
The general solution: Python wheels • A built distribution format
called “wheels” exists • Wheels can be uploaded to PyPI alongside source distributions • Wheels are architecture-specific – e.g. win32, win_amd64, macosx, linux_i686, linux_x86_64 • A recent addition allowed “manylinux” wheels to be uploaded • Most people don't bother uploading wheels -
Useless! • None of these wheels are of any use
to a Pi user • No architecture match • Fallback: build from source! -
ARM Wheels • Wheels built on a Raspberry Pi are
specific to the ARM architecture, tagged with the ARM version: – numpy-1.12.1-cp34-cp34m-linux_armv6l.whl – numpy-1.12.1-cp34-cp34m-linux_armv7l.whl • PyPI does not currently support uploading ARM wheels – The next generation of PyPI does have support for ARM wheels (due to my project) -
Why use pip not apt? • pip provides newer versions
of packages • virtualenv users install dependencies using pip – This includes pre-installed system packages – “pip install sense-hat” inside a virtalenv includes installing numpy and other dependencies -
The solution: piwheels • There were 106, 589 packages on
PyPI (at the time) • I attempted to build them all on a Pi 3 – It took 10 days – 76% built successfully – Build output logged in a database • The same Pi 3 hosts a package repository – “pip install numpy -i http://piwheels.bennuttall.com” works and takes 6 seconds :) – Browse http://piwheels.bennuttall.com/package to see build output -
Benchmarks Package Pi 1 install time (PyPI) Pi 3 install
time (PyPI) Pi 1 install time (piwheels) Pi 3 install time (piwheels) numpy sense-hat gpiozero matplotlib PIL scipy pandas django flask -
What next? • Make reproducable project to welcome open-source contributions
• Move to a Mythic Beasts hosted Pi • Automate rebuilding wheels for new releases • Manually inspect failed builds, try to resolve issues • Get a domain name and SSL cert • Add server URL to Raspbian's pip config • Build for multiple Python versions (2.7, 3.4, 3.5, 3.6) • Build for ARMv6, ARMv7, ARMv8