PyPI In a box: Using a Raspberry Pi as a portable PyPI server

Things beyond our control can limit your access to PyPI and cloning it means you always have access. This talk will show you how.

Transcript

1. PyPI In a box: Using a Raspberry Pi as a
   portable PyPI server
   Vuyisile Ndlovu
   PyConNA 2020
   @terrameijar
   

   View Slide

2. What I will discuss
   ● What PyPI is and what that has to do with pip
   ● Why you would want to make a portable PyPI server
   ● How to clone PyPI
   ● How to setup a Raspberry Pi as a PyPI server
   

   View Slide

3. PiPY
   

   View Slide

4. PyPI
   ● PyPI helps you find and install software developed and shared by the Python
   community.
   ● Hosts 200k+ packages
   ● Pip uses PyPI as its default source of packages and their dependencies
   ● Not to be confused with PyPy
   ● Package authors use PyPI to distribute their software
   

   View Slide

5. pip
   

   View Slide

6. Pip is similar to npm in JS, gem in Ruby,
   NuGet in .NET
   All modern Python version ship with pip
   

   View Slide

7. pip --help
   Usage:
   pip [options]
   Commands:
   install Install packages.
   download Download packages.
   uninstall Uninstall packages.
   freeze Output installed packages in
   requirements format.
   list List installed packages.
   show Show information about installed
   packages.
   check Verify installed packages have
   compatible dependencies.
   

   View Slide

8. How does it work?
   

   View Slide

9. pip install requests
   

   View Slide

10. pip install requests
    Collecting requests
    Collecting certifi>=2017.4.17 (from requests)
    Collecting
    urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from
    requests)
    Collecting chardet<3.1.0,>=3.0.2 (from requests)
    Collecting idna<2.9,>=2.5 (from requests)
    Installing collected packages: certifi, urllib3,
    chardet, idna, requests
    Successfully installed certifi-2019.11.28
    chardet-3.0.4 idna-2.8 requests-2.22.0
    urllib3-1.25.8
    

    View Slide

11. Why you may
    want to clone PyPI
    

    View Slide

12. ● Expensive or unreliable
    Internet
    ● Poor/limited access to power
    ● Security reasons (corporate
    firewalls etc)
    

    View Slide

13. Creating your own
    PyPI Server
    

    View Slide

14. Raspberry Pi
    Small computer the size of a bank
    card
    ● Lightweight
    ● Affordable
    ● Portable
    ● Low power
    

    View Slide

15. ● minirepo. Allows you to download a selective clone, e.g download
    Python3 packages only
    ● pypiserver --> Creates an index from the downloaded packages
    ● nginx --> Web server use this as a reverse proxy to pypiserver
    Clone & Serve up PyPI content
    

    View Slide

16. Turning a Raspberry
    Pi into a wireless
    access point
    

    View Slide

17. ● Setup WiFi
    Access Point
    ● Setup DNS &
    DHCP
    

    View Slide

18. ● Hostapd
    ● Dnsmasq
    

    View Slide

19. Static IP
    # /etc/dhcpcd.conf
    interface wlan0
    static ip_address=192.168.4.1/24
    nohook wpa_supplicant
    

    View Slide

20. DHCP
    # /etc/dnsmasq.conf
    interface=wlan0
    listen-address=192.168.4.1
    dhcp-range=192.168.4.2,192.168.4.30,255.255.255.0,24h
    address=/raspberrypi.local/192.168.4.1
    

    View Slide

21. # /etc/hostapd/hostapd.conf
    interface=wlan0
    driver=nl80211
    ssid=NameOfNetwork
    hw_mode=g
    channel=7
    wmm_enabled=0
    macaddr_acl=0
    auth_algs=1
    ignore_broadcast_ssid=0
    wpa=2
    wpa_key_mgmt=WPA-PSK
    wpa_pairwise=TKIP
    rsn_pairwise=CCMP
    wpa_passphrase=YourNetworkPassword
    Access Point
    

    View Slide

22. Serving up the Python
    packages
    

    View Slide

23. ● Download packages using
    minirepo
    ● Serve them up using
    pypiserver
    ● Setup nginx as a reverse
    proxy to pypiserver
    

    View Slide

24. Nginx
    

    View Slide

25. proxy_cache_path /data/nginx/cache
    levels=1:2
    keys_zone=pypiserver_cache:10m
    max_size=10g
    inactive=120m
    use_temp_path=off;
    upstream pypi {
    server 127.0.0.1:8080;
    }
    server {
    listen 80;
    server_name cheeseshop.com;
    autoindex on;
    location / {
    proxy_set_header Host $host:$server_port;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_cache pypiserver_cache;
    proxy_pass http://pypi;
    }
    }
    

    View Slide

26. Demo
    

    View Slide

27. WiFi Name: PyPIServer
    Password: PythonRocks
    

    View Slide

28. Vuyisile Ndlovu
    @terrameijar
    vuyisile.com/blog
    http://bit.ly/pypi_in_box
    

    View Slide

29. Questions?
    

    View Slide