The Future of Linux Packaging​

What's the future of Linux distributions? Containers, npm, linuxbrew, snaps join their friends container-optimized Linux and minimal distro images and make the future of APT and RPM a little bit... cloudy. How is open source software delivery and package management changing, and where are distributions going and what it means for Linux system administrators?

Originally presented at Texas LinuxFest 2019 in Irving, TX

Transcript

1. The Future of Linux Packaging José Miguel Parrella (@bureado) Texas LinuxFest 2019
2. Agenda Beyond distributing software Tensions are more than cosmetics High expectations on package managers An assessment framework for package managers Attributes of a post-modern package manager
4. From distributing software... Data model Application Configuration Libraries Utilities Kernel & firmware Managed DB Docker Hub Ansible nix snapd apt + fwupd Image + ephemeral partition curl | sh Mix of packages and layers, multiple cadences
5. ...to distributing state Data model Configuration Application Libraries Utilities Kernel • Linux ended up everywhere • New forms of Linux • The network became faster and more reliable • We changed how we look at distributed systems • It became harder to always represent state as a file • Pets gave way to cattle Data model Configuration Application Libraries Utilities Kernel Data model Configuration Application Libraries Utilities Kernel Data model Configuration Application Libraries Utilities Kernel Data model Configuration Application Libraries Utilities Kernel & firmware
6. It's been getting specialized! • Library managers • From CPAN and PyPI to NPM and Golang packages • Next-gen package management • Container image specification, Helm charts/CNAB & registries • Use cases where provenance is controlled by final distributor (embedded, firmware) Ecosystem Debian Upstream Coverage Ruby 1100 9300 11.83% Perl 3700 31000 11.94% Python 3700 118000 3.14% Node.js 1300 350000 0.37% All-up libs 30K 2.8M 1.07% Docker Hub ? 2.3M N/A Helm Charts CNAB Bundles Portable Services ~0 ? ~0 Source: libraries.io, APT lists, Docker Hub (2018)
13. What problems are we trying to solve? • Push the packaging responsibility upstream? • Be able to distribute non-free software more effectively and/or monetize? • Provide additional container and application security capabilities? • Reduce the size of a Linux distribution? • Make it easier to package for Linux by removing dependency tracking? • Immutable, composable and reproducible systems? • Support cloud distribution models? • Make the software experience easier and better for Linux desktop users?
14. Package managers Associated distros (if applicable) • Atomic unit of software distribution (snap, bundle, etc.) • What the unit actually is (tarred source, squashfs, etc.) • What the unit metadata describes (dependencies, origin, checksums, etc.) • Where the unit comes from (repository equivalent) • Core repository concepts (e.g., channels, governance, login, proprietary software, etc.) • How are updates delivered? • What’s the isolation/sandboxing story? • Universe (size) and type of apps • How packages are built (developer tooling) • Source vs. binary, binary caches, etc. • Any components of the system not managed as a unit? • Upgrade/rollback strategy (e.g., dual partitions for CoreOS) • What software is available (e.g., in bundles) and what for? • What versions of said software? How is upstream tracked? • Is it a rolling release? • How are end users expected to bring their applications? • How system state is described (e.g., version hashes, all-up system release numbers) • Coexistence with other packaging systems • How is package provenance validated? What type of software is available? How is it distributed? Who seems to be gravitating towards it?
16. What else do we expect from package managers? • Rich dependencies/metadata • Ability to automate operations (particularly upgrades) • Ability to compose from multiple repositories • Signatures and other mechanisms to prevent tampering o Mixing multiple release cadences o Distributing multiple kinds of artifacts oSource provenance/supply chain security
17. Attributes of the post- modern package manager SW supply chain security Reproduciibilityand trusted builds Identity and fingerprinting Smarter repos Client-aware Flighting Beyond CDN System-wide observability Package manager coexistence System-wide indices, metadata
18. Thank you! • Slides posted to Speaker Deck • Find me on Twitter and GitHub: @bureado • Web: www.jmp.soy • See you tonight at Lightning Talks!
19. Framework (sample) Name​ Atomic unit​ Unit source​ Univers e​ Isolation​ Runtimes​ Core value prop​ Use case focus​ Related​ Depends​ Snap​ Snap(squashfs)​ Stores​ 1,500 ( *)​ AppArmor*,offers classicmode Core snap​ Autoupdates and bundling​ Proprietary apps and IoT likely​ UbuntuCo re Systemd Flatpak Package(OSTre e,OCI)​ Flathub 290 Sandbox plusbubblewrap Runtimes(GNOM E,etc.)​ Cross- distroportability Desktopapplications OSTree Atomic​ AppStrea m​ OSTree Systemd bubblewr ap Nix​ Paths​ Nixpkgs 6,500 None (other than the Nixstorepath)​ N/A​ Atomic upgrades and multi-versioning Universal,declarativesy stems NixOS N/A​ Guix Paths​ Hydra​ 7,660 None (other than the GNUstorepath)​ N/A​ Ease of use​ Transactions​ Buildreproducibility Easy packaging? Universal​ GuixSD Guile​ AppImag e AppImage Distribute d​ 150+(* ) to 380 Optional via firejail N/A​ 1 app = 1 file​ Desktopapplications Firejail N/A​ swupd Bundles​ Repos​ 180 N/A​ A few (e.g.,perl,python)​ No packages,binary deltasonly Deterministicupgrades in the cloud​ Clear Linux​ N/A ​Helm Charts​ Hub 280 Kubernetes pods​ N/A Complex, multi-node apps​ ​Data, Dev Tools ​CNAB K8S​