PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Slide 1

Slide 1 text

PyPI In a box: Using a Raspberry Pi as a portable PyPI server Vuyisile Ndlovu PyConNA 2020 @terrameijar

Slide 2

Slide 2 text

What I will discuss ● What PyPI is and what that has to do with pip ● Why you would want to make a portable PyPI server ● How to clone PyPI ● How to setup a Raspberry Pi as a PyPI server

Slide 3

Slide 3 text

PiPY

Slide 4

Slide 4 text

PyPI ● PyPI helps you find and install software developed and shared by the Python community. ● Hosts 200k+ packages ● Pip uses PyPI as its default source of packages and their dependencies ● Not to be confused with PyPy ● Package authors use PyPI to distribute their software

Slide 5

Slide 5 text

pip

Slide 6

Slide 6 text

Pip is similar to npm in JS, gem in Ruby, NuGet in .NET All modern Python version ship with pip

Slide 7

Slide 7 text

pip --help Usage: pip [options] Commands: install Install packages. download Download packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. check Verify installed packages have compatible dependencies.

Slide 8

Slide 8 text

How does it work?

Slide 9

Slide 9 text

pip install requests

Slide 10

Slide 10 text

pip install requests Collecting requests Collecting certifi>=2017.4.17 (from requests) Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) Collecting chardet<3.1.0,>=3.0.2 (from requests) Collecting idna<2.9,>=2.5 (from requests) Installing collected packages: certifi, urllib3, chardet, idna, requests Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.8

Slide 11

Slide 11 text

Why you may want to clone PyPI

Slide 12

Slide 12 text

● Expensive or unreliable Internet ● Poor/limited access to power ● Security reasons (corporate firewalls etc)

Slide 13

Slide 13 text

Creating your own PyPI Server

Slide 14

Slide 14 text

Raspberry Pi Small computer the size of a bank card ● Lightweight ● Affordable ● Portable ● Low power

Slide 15

Slide 15 text

● minirepo. Allows you to download a selective clone, e.g download Python3 packages only ● pypiserver --> Creates an index from the downloaded packages ● nginx --> Web server use this as a reverse proxy to pypiserver Clone & Serve up PyPI content

Slide 16

Slide 16 text

Turning a Raspberry Pi into a wireless access point

Slide 17

Slide 17 text

● Setup WiFi Access Point ● Setup DNS & DHCP

Slide 18

Slide 18 text

● Hostapd ● Dnsmasq

Slide 19

Slide 19 text

Static IP # /etc/dhcpcd.conf interface wlan0 static ip_address=192.168.4.1/24 nohook wpa_supplicant

Slide 20

Slide 20 text

DHCP # /etc/dnsmasq.conf interface=wlan0 listen-address=192.168.4.1 dhcp-range=192.168.4.2,192.168.4.30,255.255.255.0,24h address=/raspberrypi.local/192.168.4.1

Slide 21

Slide 21 text

# /etc/hostapd/hostapd.conf interface=wlan0 driver=nl80211 ssid=NameOfNetwork hw_mode=g channel=7 wmm_enabled=0 macaddr_acl=0 auth_algs=1 ignore_broadcast_ssid=0 wpa=2 wpa_key_mgmt=WPA-PSK wpa_pairwise=TKIP rsn_pairwise=CCMP wpa_passphrase=YourNetworkPassword Access Point

Slide 22

Slide 22 text

Serving up the Python packages

Slide 23

Slide 23 text

● Download packages using minirepo ● Serve them up using pypiserver ● Setup nginx as a reverse proxy to pypiserver

Slide 24

Slide 24 text

Nginx

Slide 25

Slide 25 text

proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=pypiserver_cache:10m max_size=10g inactive=120m use_temp_path=off; upstream pypi { server 127.0.0.1:8080; } server { listen 80; server_name cheeseshop.com; autoindex on; location / { proxy_set_header Host $host:$server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Real-IP $remote_addr; proxy_cache pypiserver_cache; proxy_pass http://pypi; } }