$30 off During Our Annual Pro Sale. View Details »

The Future of Linux Packaging​

The Future of Linux Packaging​

What's the future of Linux distributions? Containers, npm, linuxbrew, snaps join their friends container-optimized Linux and minimal distro images and make the future of APT and RPM a little bit... cloudy. How is open source software delivery and package management changing, and where are distributions going and what it means for Linux system administrators?

Originally presented at Texas LinuxFest 2019 in Irving, TX

José Miguel Parrella

June 01, 2019
Tweet

More Decks by José Miguel Parrella

Other Decks in Technology

Transcript

  1. The Future of
    Linux Packaging
    José Miguel Parrella
    (@bureado)
    Texas LinuxFest
    2019

    View Slide

  2. Agenda
    Beyond
    distributing
    software
    Tensions are more
    than cosmetics
    High expectations
    on package
    managers
    An assessment
    framework for
    package managers
    Attributes of a
    post-modern
    package manager

    View Slide

  3. View Slide

  4. From distributing software...
    Data model
    Application
    Configuration
    Libraries
    Utilities
    Kernel & firmware
    Managed DB
    Docker Hub
    Ansible
    nix
    snapd
    apt + fwupd
    Image +
    ephemeral
    partition
    curl | sh
    Mix of packages
    and layers,
    multiple cadences

    View Slide

  5. ...to distributing state
    Data model
    Configuration
    Application
    Libraries
    Utilities
    Kernel
    • Linux ended up everywhere
    • New forms of Linux
    • The network became faster and more
    reliable
    • We changed how we look at distributed
    systems
    • It became harder to always represent
    state as a file
    • Pets gave way to cattle
    Data model
    Configuration
    Application
    Libraries
    Utilities
    Kernel
    Data model
    Configuration
    Application
    Libraries
    Utilities
    Kernel
    Data model
    Configuration
    Application
    Libraries
    Utilities
    Kernel
    Data model
    Configuration
    Application
    Libraries
    Utilities
    Kernel & firmware

    View Slide

  6. It's been getting specialized!
    • Library managers
    • From CPAN and PyPI to NPM and
    Golang packages
    • Next-gen package management
    • Container image specification,
    Helm charts/CNAB & registries
    • Use cases where provenance
    is controlled by final distributor
    (embedded, firmware)
    Ecosystem Debian Upstream Coverage
    Ruby 1100 9300 11.83%
    Perl 3700 31000 11.94%
    Python 3700 118000 3.14%
    Node.js 1300 350000 0.37%
    All-up libs 30K 2.8M 1.07%
    Docker Hub ? 2.3M N/A
    Helm Charts
    CNAB Bundles
    Portable
    Services
    ~0 ? ~0
    Source: libraries.io, APT lists, Docker Hub (2018)

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. What
    problems are
    we trying to
    solve?
    • Push the packaging responsibility upstream?
    • Be able to distribute non-free software more
    effectively and/or monetize?
    • Provide additional container and application
    security capabilities?
    • Reduce the size of a Linux distribution?
    • Make it easier to package for Linux by removing
    dependency tracking?
    • Immutable, composable and reproducible
    systems?
    • Support cloud distribution models?
    • Make the software experience easier and
    better for Linux desktop users?

    View Slide

  14. Package managers Associated distros (if applicable)
    • Atomic unit of software distribution (snap,
    bundle, etc.)
    • What the unit actually is (tarred source,
    squashfs, etc.)
    • What the unit metadata describes
    (dependencies, origin, checksums, etc.)
    • Where the unit comes from (repository
    equivalent)
    • Core repository concepts (e.g., channels,
    governance, login, proprietary software, etc.)
    • How are updates delivered?
    • What’s the isolation/sandboxing story?
    • Universe (size) and type of apps
    • How packages are built (developer tooling)
    • Source vs. binary, binary caches, etc.
    • Any components of the system not managed as a
    unit?
    • Upgrade/rollback strategy (e.g., dual partitions
    for CoreOS)
    • What software is available (e.g., in bundles) and
    what for?
    • What versions of said software? How is
    upstream tracked?
    • Is it a rolling release?
    • How are end users expected to bring their
    applications?
    • How system state is described (e.g., version
    hashes, all-up system release numbers)
    • Coexistence with other packaging systems
    • How is package provenance validated?
    What type of software is available? How is it distributed? Who seems to be gravitating towards it?

    View Slide

  15. View Slide

  16. What else do
    we expect
    from package
    managers?
    • Rich dependencies/metadata
    • Ability to automate operations (particularly
    upgrades)
    • Ability to compose from multiple repositories
    • Signatures and other mechanisms to prevent
    tampering
    o Mixing multiple release cadences
    o Distributing multiple kinds of artifacts
    oSource provenance/supply chain security

    View Slide

  17. Attributes of
    the post-
    modern
    package
    manager
    SW supply chain
    security
    Reproduciibilityand trusted
    builds
    Identity and fingerprinting
    Smarter repos
    Client-aware
    Flighting
    Beyond CDN
    System-wide
    observability
    Package manager
    coexistence
    System-wide indices,
    metadata

    View Slide

  18. Thank you!
    • Slides posted to Speaker Deck
    • Find me on Twitter and GitHub: @bureado
    • Web: www.jmp.soy
    • See you tonight at Lightning Talks!

    View Slide

  19. Framework (sample)
    Name​ Atomic unit​
    Unit
    source​
    Univers
    e​
    Isolation​ Runtimes​ Core value prop​ Use case focus​ Related​ Depends​
    Snap​ Snap(squashfs)​ Stores​
    1,500 (
    *)​
    AppArmor*,offers
    classicmode
    Core snap​
    Autoupdates
    and bundling​
    Proprietary apps and
    IoT likely​
    UbuntuCo
    re
    Systemd
    Flatpak
    Package(OSTre
    e,OCI)​
    Flathub 290
    Sandbox
    plusbubblewrap
    Runtimes(GNOM
    E,etc.)​
    Cross-
    distroportability
    Desktopapplications
    OSTree
    Atomic​
    AppStrea
    m​
    OSTree
    Systemd
    bubblewr
    ap
    Nix​ Paths​ Nixpkgs 6,500
    None (other than
    the Nixstorepath)​
    N/A​
    Atomic upgrades and
    multi-versioning
    Universal,declarativesy
    stems
    NixOS N/A​
    Guix Paths​ Hydra​ 7,660
    None (other than
    the GNUstorepath)​
    N/A​
    Ease of use​
    Transactions​
    Buildreproducibility
    Easy packaging?
    Universal​ GuixSD Guile​
    AppImag
    e
    AppImage Distribute
    d​
    150+(*
    ) to 380
    Optional via firejail N/A​ 1 app = 1 file​ Desktopapplications Firejail N/A​
    swupd Bundles​ Repos​ 180 N/A​
    A few
    (e.g.,perl,python)​
    No packages,binary
    deltasonly
    Deterministicupgrades
    in the cloud​
    Clear
    Linux​
    N/A
    ​Helm Charts​ Hub 280 Kubernetes pods​ N/A
    Complex, multi-node
    apps​
    ​Data, Dev Tools ​CNAB K8S​

    View Slide