Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Things beyond our control can limit your access to PyPI and cloning it means you always have access. This talk will show you how.

Vuyisile Ndlovu

February 20, 2020

Other Decks in Programming


  1. PyPI In a box: Using a Raspberry Pi as a

    portable PyPI server Vuyisile Ndlovu PyConNA 2020 @terrameijar
  2. What I will discuss • What PyPI is and what

    that has to do with pip • Why you would want to make a portable PyPI server • How to clone PyPI • How to setup a Raspberry Pi as a PyPI server
  3. PiPY

  4. PyPI • PyPI helps you find and install software developed

    and shared by the Python community. • Hosts 200k+ packages • Pip uses PyPI as its default source of packages and their dependencies • Not to be confused with PyPy • Package authors use PyPI to distribute their software
  5. pip

  6. Pip is similar to npm in JS, gem in Ruby,

    NuGet in .NET All modern Python version ship with pip
  7. pip --help Usage: pip <command> [options] Commands: install Install packages.

    download Download packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. check Verify installed packages have compatible dependencies.
  8. How does it work?

  9. pip install requests

  10. pip install requests Collecting requests Collecting certifi>=2017.4.17 (from requests) Collecting

    urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) Collecting chardet<3.1.0,>=3.0.2 (from requests) Collecting idna<2.9,>=2.5 (from requests) Installing collected packages: certifi, urllib3, chardet, idna, requests Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.8
  11. Why you may want to clone PyPI

  12. • Expensive or unreliable Internet • Poor/limited access to power

    • Security reasons (corporate firewalls etc)
  13. Creating your own PyPI Server

  14. Raspberry Pi Small computer the size of a bank card

    • Lightweight • Affordable • Portable • Low power
  15. • minirepo. Allows you to download a selective clone, e.g

    download Python3 packages only • pypiserver --> Creates an index from the downloaded packages • nginx --> Web server use this as a reverse proxy to pypiserver Clone & Serve up PyPI content
  16. Turning a Raspberry Pi into a wireless access point

  17. • Setup WiFi Access Point • Setup DNS & DHCP

  18. • Hostapd • Dnsmasq

  19. Static IP # /etc/dhcpcd.conf interface wlan0 static ip_address= nohook wpa_supplicant

  20. DHCP # /etc/dnsmasq.conf interface=wlan0 listen-address= dhcp-range=,,,24h address=/raspberrypi.local/

  21. # /etc/hostapd/hostapd.conf interface=wlan0 driver=nl80211 ssid=NameOfNetwork hw_mode=g channel=7 wmm_enabled=0 macaddr_acl=0 auth_algs=1

    ignore_broadcast_ssid=0 wpa=2 wpa_key_mgmt=WPA-PSK wpa_pairwise=TKIP rsn_pairwise=CCMP wpa_passphrase=YourNetworkPassword Access Point
  22. Serving up the Python packages

  23. • Download packages using minirepo • Serve them up using

    pypiserver • Setup nginx as a reverse proxy to pypiserver
  24. Nginx

  25. proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=pypiserver_cache:10m max_size=10g inactive=120m use_temp_path=off; upstream pypi {

    server; } server { listen 80; server_name cheeseshop.com; autoindex on; location / { proxy_set_header Host $host:$server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Real-IP $remote_addr; proxy_cache pypiserver_cache; proxy_pass http://pypi; } }
  26. Demo

  27. WiFi Name: PyPIServer Password: PythonRocks

  28. Vuyisile Ndlovu @terrameijar vuyisile.com/blog http://bit.ly/pypi_in_box

  29. Questions?