PyPI In a box: Using a Raspberry Pi as a portable PyPI server

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Things beyond our control can limit your access to PyPI and cloning it means you always have access. This talk will show you how.

19acb83cfe29a28b06767a0e5d7330d4?s=128

Vuyisile Ndlovu

February 20, 2020
Tweet

Transcript

  1. 1.

    PyPI In a box: Using a Raspberry Pi as a

    portable PyPI server Vuyisile Ndlovu PyConNA 2020 @terrameijar
  2. 2.

    What I will discuss • What PyPI is and what

    that has to do with pip • Why you would want to make a portable PyPI server • How to clone PyPI • How to setup a Raspberry Pi as a PyPI server
  3. 3.
  4. 4.

    PyPI • PyPI helps you find and install software developed

    and shared by the Python community. • Hosts 200k+ packages • Pip uses PyPI as its default source of packages and their dependencies • Not to be confused with PyPy • Package authors use PyPI to distribute their software
  5. 5.

    pip

  6. 6.

    Pip is similar to npm in JS, gem in Ruby,

    NuGet in .NET All modern Python version ship with pip
  7. 7.

    pip --help Usage: pip <command> [options] Commands: install Install packages.

    download Download packages. uninstall Uninstall packages. freeze Output installed packages in requirements format. list List installed packages. show Show information about installed packages. check Verify installed packages have compatible dependencies.
  8. 10.

    pip install requests Collecting requests Collecting certifi>=2017.4.17 (from requests) Collecting

    urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) Collecting chardet<3.1.0,>=3.0.2 (from requests) Collecting idna<2.9,>=2.5 (from requests) Installing collected packages: certifi, urllib3, chardet, idna, requests Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.8
  9. 12.

    • Expensive or unreliable Internet • Poor/limited access to power

    • Security reasons (corporate firewalls etc)
  10. 14.

    Raspberry Pi Small computer the size of a bank card

    • Lightweight • Affordable • Portable • Low power
  11. 15.

    • minirepo. Allows you to download a selective clone, e.g

    download Python3 packages only • pypiserver --> Creates an index from the downloaded packages • nginx --> Web server use this as a reverse proxy to pypiserver Clone & Serve up PyPI content
  12. 21.

    # /etc/hostapd/hostapd.conf interface=wlan0 driver=nl80211 ssid=NameOfNetwork hw_mode=g channel=7 wmm_enabled=0 macaddr_acl=0 auth_algs=1

    ignore_broadcast_ssid=0 wpa=2 wpa_key_mgmt=WPA-PSK wpa_pairwise=TKIP rsn_pairwise=CCMP wpa_passphrase=YourNetworkPassword Access Point
  13. 23.

    • Download packages using minirepo • Serve them up using

    pypiserver • Setup nginx as a reverse proxy to pypiserver
  14. 24.
  15. 25.

    proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=pypiserver_cache:10m max_size=10g inactive=120m use_temp_path=off; upstream pypi {

    server 127.0.0.1:8080; } server { listen 80; server_name cheeseshop.com; autoindex on; location / { proxy_set_header Host $host:$server_port; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Real-IP $remote_addr; proxy_cache pypiserver_cache; proxy_pass http://pypi; } }
  16. 26.