redistributable archives • A distribution is an archive of zero or more packages • Motivational case studies: PIL & Pillow. PyCrypto(dome). • Built by setuptools through setup.py ◦ Example: sdist ◦ Simple, source-only .tar.gz 11 Packages are not proper packages.
team in 2011 • 30+ midtier apps, services, and batch jobs ◦ Max single service volume: 1.2 billion reqs/day (2016) ◦ Max single service throughput: 10,000 reqs/sec/worker (2016) • Multiprotocol, service-focused, gevent-based framework • Hundreds of users, almost all grassroots 26 http://github.com/paypal/support Wonderfully, spontaneously Python.
binary libraries) / 5 team members = ∞ static builds. 27 PayPal Python Environment Support Matrix (2014) Operating System Architecture Python Version Linux 32-bit/64-bit 2.6/2.7 Mac 64-bit 2.7 Solaris 32-bit 2.7 Windows 32-bit 2.7
conda install --download-only 2. Miniconda + .tar.bz2 archives into the RPM 3. Run a tiny installer script in the RPM postinstall section Ready to test and deploy! https://www.paypal-engineering.com/2016/09/07/python-packaging-at-paypal/ https://github.com/paypal/support/blob/master/examples/miniconda 32 One way to box a snake:
in its own partition! • Literally v1 was ISO9660 • E.g., AppImage / kdenlive 36 Even trendier than selfies! Anaconda meets old autorun CDs https://github.com/AppImage/AppImageKit https://github.com/appimage-packages/kdenlive
into an images for local, stage, and prod • Library dependencies from: yum, conda, pip, and sky (git repo) • Service deps specified in terms of docker images • Plugins + core shipped as PEX to local developers • Open-sourced just for you: https://github.com/shopkick/sky 40
Bring your own Miniconda • Custom installer script that calls conda ◦ docker build instrumentation severely lacking. Layer proliferation. • Tips: ◦ Run after yum, before pip ◦ Size is always an issue: nomkl unless you really need the speed, conda clean. ◦ --no-channel-priority useful for mixing and matching 41