Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Things beyond our control can limit your access to PyPI and cloning it means you always have access. This talk will show you how.

Vuyisile Ndlovu

February 20, 2020
Tweet

More Decks by Vuyisile Ndlovu

Other Decks in Programming

Transcript

  1. PyPI In a box: Using a Raspberry Pi as a
    portable PyPI server
    Vuyisile Ndlovu
    PyConNA 2020
    @terrameijar

    View full-size slide

  2. What I will discuss
    ● What PyPI is and what that has to do with pip
    ● Why you would want to make a portable PyPI server
    ● How to clone PyPI
    ● How to setup a Raspberry Pi as a PyPI server

    View full-size slide

  3. PyPI
    ● PyPI helps you find and install software developed and shared by the Python
    community.
    ● Hosts 200k+ packages
    ● Pip uses PyPI as its default source of packages and their dependencies
    ● Not to be confused with PyPy
    ● Package authors use PyPI to distribute their software

    View full-size slide

  4. Pip is similar to npm in JS, gem in Ruby,
    NuGet in .NET
    All modern Python version ship with pip

    View full-size slide

  5. pip --help
    Usage:
    pip [options]
    Commands:
    install Install packages.
    download Download packages.
    uninstall Uninstall packages.
    freeze Output installed packages in
    requirements format.
    list List installed packages.
    show Show information about installed
    packages.
    check Verify installed packages have
    compatible dependencies.

    View full-size slide

  6. How does it work?

    View full-size slide

  7. pip install requests

    View full-size slide

  8. pip install requests
    Collecting requests
    Collecting certifi>=2017.4.17 (from requests)
    Collecting
    urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from
    requests)
    Collecting chardet<3.1.0,>=3.0.2 (from requests)
    Collecting idna<2.9,>=2.5 (from requests)
    Installing collected packages: certifi, urllib3,
    chardet, idna, requests
    Successfully installed certifi-2019.11.28
    chardet-3.0.4 idna-2.8 requests-2.22.0
    urllib3-1.25.8

    View full-size slide

  9. Why you may
    want to clone PyPI

    View full-size slide

  10. ● Expensive or unreliable
    Internet
    ● Poor/limited access to power
    ● Security reasons (corporate
    firewalls etc)

    View full-size slide

  11. Creating your own
    PyPI Server

    View full-size slide

  12. Raspberry Pi
    Small computer the size of a bank
    card
    ● Lightweight
    ● Affordable
    ● Portable
    ● Low power

    View full-size slide

  13. ● minirepo. Allows you to download a selective clone, e.g download
    Python3 packages only
    ● pypiserver --> Creates an index from the downloaded packages
    ● nginx --> Web server use this as a reverse proxy to pypiserver
    Clone & Serve up PyPI content

    View full-size slide

  14. Turning a Raspberry
    Pi into a wireless
    access point

    View full-size slide

  15. ● Setup WiFi
    Access Point
    ● Setup DNS &
    DHCP

    View full-size slide

  16. ● Hostapd
    ● Dnsmasq

    View full-size slide

  17. Static IP
    # /etc/dhcpcd.conf
    interface wlan0
    static ip_address=192.168.4.1/24
    nohook wpa_supplicant

    View full-size slide

  18. DHCP
    # /etc/dnsmasq.conf
    interface=wlan0
    listen-address=192.168.4.1
    dhcp-range=192.168.4.2,192.168.4.30,255.255.255.0,24h
    address=/raspberrypi.local/192.168.4.1

    View full-size slide

  19. # /etc/hostapd/hostapd.conf
    interface=wlan0
    driver=nl80211
    ssid=NameOfNetwork
    hw_mode=g
    channel=7
    wmm_enabled=0
    macaddr_acl=0
    auth_algs=1
    ignore_broadcast_ssid=0
    wpa=2
    wpa_key_mgmt=WPA-PSK
    wpa_pairwise=TKIP
    rsn_pairwise=CCMP
    wpa_passphrase=YourNetworkPassword
    Access Point

    View full-size slide

  20. Serving up the Python
    packages

    View full-size slide

  21. ● Download packages using
    minirepo
    ● Serve them up using
    pypiserver
    ● Setup nginx as a reverse
    proxy to pypiserver

    View full-size slide

  22. proxy_cache_path /data/nginx/cache
    levels=1:2
    keys_zone=pypiserver_cache:10m
    max_size=10g
    inactive=120m
    use_temp_path=off;
    upstream pypi {
    server 127.0.0.1:8080;
    }
    server {
    listen 80;
    server_name cheeseshop.com;
    autoindex on;
    location / {
    proxy_set_header Host $host:$server_port;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_cache pypiserver_cache;
    proxy_pass http://pypi;
    }
    }

    View full-size slide

  23. WiFi Name: PyPIServer
    Password: PythonRocks

    View full-size slide

  24. Vuyisile Ndlovu
    @terrameijar
    vuyisile.com/blog
    http://bit.ly/pypi_in_box

    View full-size slide