PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Things beyond our control can limit your access to PyPI and cloning it means you always have access. This talk will show you how.

Transcript

1. PyPI In a box: Using a Raspberry Pi as a portable PyPI server Vuyisile Ndlovu PyConNA 2020 @terrameijar
2. What I will discuss • What PyPI is and what that has to do with pip • Why you would want to make a portable PyPI server • How to clone PyPI • How to setup a Raspberry Pi as a PyPI server
3. PiPY
4. PyPI • PyPI helps you find and install software developed and shared by the Python community. • Hosts 200k+ packages • Pip uses PyPI as its default source of packages and their dependencies • Not to be confused with PyPy • Package authors use PyPI to distribute their software
5. pip
6. Pip is similar to npm in JS, gem in Ruby, NuGet in .NET All modern Python version ship with pip
7. pip --help Usage: pip <command> [options] Commands: install Install packages. download Download packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. check Verify installed packages have compatible dependencies.
8. How does it work?
9. pip install requests
10. pip install requests Collecting requests Collecting certifi>=2017.4.17 (from requests) Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) Collecting chardet<3.1.0,>=3.0.2 (from requests) Collecting idna<2.9,>=2.5 (from requests) Installing collected packages: certifi, urllib3, chardet, idna, requests Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.8
11. Why you may want to clone PyPI
12. • Expensive or unreliable Internet • Poor/limited access to power • Security reasons (corporate firewalls etc)
13. Creating your own PyPI Server
14. Raspberry Pi Small computer the size of a bank card • Lightweight • Affordable • Portable • Low power
15. • minirepo. Allows you to download a selective clone, e.g download Python3 packages only • pypiserver --> Creates an index from the downloaded packages • nginx --> Web server use this as a reverse proxy to pypiserver Clone & Serve up PyPI content
16. Turning a Raspberry Pi into a wireless access point
17. • Setup WiFi Access Point • Setup DNS & DHCP
18. • Hostapd • Dnsmasq
19. Static IP # /etc/dhcpcd.conf interface wlan0 static ip_address=192.168.4.1/24 nohook wpa_supplicant
20. DHCP # /etc/dnsmasq.conf interface=wlan0 listen-address=192.168.4.1 dhcp-range=192.168.4.2,192.168.4.30,255.255.255.0,24h address=/raspberrypi.local/192.168.4.1
21. # /etc/hostapd/hostapd.conf interface=wlan0 driver=nl80211 ssid=NameOfNetwork hw_mode=g channel=7 wmm_enabled=0 macaddr_acl=0 auth_algs=1 ignore_broadcast_ssid=0 wpa=2 wpa_key_mgmt=WPA-PSK wpa_pairwise=TKIP rsn_pairwise=CCMP wpa_passphrase=YourNetworkPassword Access Point
22. Serving up the Python packages
23. • Download packages using minirepo • Serve them up using pypiserver • Setup nginx as a reverse proxy to pypiserver
24. Nginx
25. proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=pypiserver_cache:10m max_size=10g inactive=120m use_temp_path=off; upstream pypi { server 127.0.0.1:8080; } server { listen 80; server_name cheeseshop.com; autoindex on; location / { proxy_set_header Host $host:$server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Real-IP $remote_addr; proxy_cache pypiserver_cache; proxy_pass http://pypi; } }
26. Demo
27. WiFi Name: PyPIServer Password: PythonRocks
28. Vuyisile Ndlovu @terrameijar vuyisile.com/blog http://bit.ly/pypi_in_box
29. Questions?