Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Things beyond our control can limit your access to PyPI and cloning it means you always have access. This talk will show you how.

Vuyisile Ndlovu

February 20, 2020
Tweet

Other Decks in Programming

Transcript

  1. PyPI In a box: Using a Raspberry Pi as a
    portable PyPI server
    Vuyisile Ndlovu
    PyConNA 2020
    @terrameijar

    View Slide

  2. What I will discuss
    ● What PyPI is and what that has to do with pip
    ● Why you would want to make a portable PyPI server
    ● How to clone PyPI
    ● How to setup a Raspberry Pi as a PyPI server

    View Slide

  3. PiPY

    View Slide

  4. PyPI
    ● PyPI helps you find and install software developed and shared by the Python
    community.
    ● Hosts 200k+ packages
    ● Pip uses PyPI as its default source of packages and their dependencies
    ● Not to be confused with PyPy
    ● Package authors use PyPI to distribute their software

    View Slide

  5. pip

    View Slide

  6. Pip is similar to npm in JS, gem in Ruby,
    NuGet in .NET
    All modern Python version ship with pip

    View Slide

  7. pip --help
    Usage:
    pip [options]
    Commands:
    install Install packages.
    download Download packages.
    uninstall Uninstall packages.
    freeze Output installed packages in
    requirements format.
    list List installed packages.
    show Show information about installed
    packages.
    check Verify installed packages have
    compatible dependencies.

    View Slide

  8. How does it work?

    View Slide

  9. pip install requests

    View Slide

  10. pip install requests
    Collecting requests
    Collecting certifi>=2017.4.17 (from requests)
    Collecting
    urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from
    requests)
    Collecting chardet<3.1.0,>=3.0.2 (from requests)
    Collecting idna<2.9,>=2.5 (from requests)
    Installing collected packages: certifi, urllib3,
    chardet, idna, requests
    Successfully installed certifi-2019.11.28
    chardet-3.0.4 idna-2.8 requests-2.22.0
    urllib3-1.25.8

    View Slide

  11. Why you may
    want to clone PyPI

    View Slide

  12. ● Expensive or unreliable
    Internet
    ● Poor/limited access to power
    ● Security reasons (corporate
    firewalls etc)

    View Slide

  13. Creating your own
    PyPI Server

    View Slide

  14. Raspberry Pi
    Small computer the size of a bank
    card
    ● Lightweight
    ● Affordable
    ● Portable
    ● Low power

    View Slide

  15. ● minirepo. Allows you to download a selective clone, e.g download
    Python3 packages only
    ● pypiserver --> Creates an index from the downloaded packages
    ● nginx --> Web server use this as a reverse proxy to pypiserver
    Clone & Serve up PyPI content

    View Slide

  16. Turning a Raspberry
    Pi into a wireless
    access point

    View Slide

  17. ● Setup WiFi
    Access Point
    ● Setup DNS &
    DHCP

    View Slide

  18. ● Hostapd
    ● Dnsmasq

    View Slide

  19. Static IP
    # /etc/dhcpcd.conf
    interface wlan0
    static ip_address=192.168.4.1/24
    nohook wpa_supplicant

    View Slide

  20. DHCP
    # /etc/dnsmasq.conf
    interface=wlan0
    listen-address=192.168.4.1
    dhcp-range=192.168.4.2,192.168.4.30,255.255.255.0,24h
    address=/raspberrypi.local/192.168.4.1

    View Slide

  21. # /etc/hostapd/hostapd.conf
    interface=wlan0
    driver=nl80211
    ssid=NameOfNetwork
    hw_mode=g
    channel=7
    wmm_enabled=0
    macaddr_acl=0
    auth_algs=1
    ignore_broadcast_ssid=0
    wpa=2
    wpa_key_mgmt=WPA-PSK
    wpa_pairwise=TKIP
    rsn_pairwise=CCMP
    wpa_passphrase=YourNetworkPassword
    Access Point

    View Slide

  22. Serving up the Python
    packages

    View Slide

  23. ● Download packages using
    minirepo
    ● Serve them up using
    pypiserver
    ● Setup nginx as a reverse
    proxy to pypiserver

    View Slide

  24. Nginx

    View Slide

  25. proxy_cache_path /data/nginx/cache
    levels=1:2
    keys_zone=pypiserver_cache:10m
    max_size=10g
    inactive=120m
    use_temp_path=off;
    upstream pypi {
    server 127.0.0.1:8080;
    }
    server {
    listen 80;
    server_name cheeseshop.com;
    autoindex on;
    location / {
    proxy_set_header Host $host:$server_port;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_cache pypiserver_cache;
    proxy_pass http://pypi;
    }
    }

    View Slide

  26. Demo

    View Slide

  27. WiFi Name: PyPIServer
    Password: PythonRocks

    View Slide

  28. Vuyisile Ndlovu
    @terrameijar
    vuyisile.com/blog
    http://bit.ly/pypi_in_box

    View Slide

  29. Questions?

    View Slide