$30 off During Our Annual Pro Sale. View Details »

Entangled In Dependencies: Pipenv&Poetry

Entangled In Dependencies: Pipenv&Poetry

Python dependencies management: Pipenv and Poetry.

Alex Khaerov

March 26, 2019
Tweet

More Decks by Alex Khaerov

Other Decks in Technology

Transcript

  1. Pipenv & Poetry
    Entangled In Dependencies:
    by Alex Khaerov
    @hayorov

    View Slide

  2. @hayorov
    Hi!

    View Slide

  3. @hayorov
    we ❤
    company
    who I am

    View Slide

  4. @hayorov
    Alex Khaerov
    we ❤
    company
    who I am
    @hayorov

    View Slide

  5. @hayorov
    Chainstack
    Multi-cloud and multi-blockchain Platform as a Service
    Alex Khaerov
    we ❤
    company
    who I am
    @hayorov

    View Slide

  6. @hayorov
    Chainstack
    Multi-cloud and multi-blockchain Platform as a Service
    Alex Khaerov
    we ❤
    company
    who I am
    @hayorov

    View Slide

  7. @hayorov
    Chainstack
    Multi-cloud and multi-blockchain Platform as a Service
    Alex Khaerov
    we ❤
    company
    who I am
    @hayorov
    We are hiring:
    careers.chainstack.com

    View Slide

  8. @hayorov
    easy_install
    requrements.txt
    pip
    virtualenv
    setup.py
    sdist
    wheels
    virtualenvwrapper
    pyenv
    conan
    flit
    bento
    hatch
    pants
    conda
    pipenv
    pip-tools
    poetry

    View Slide

  9. @hayorov
    easy_install
    requrements.txt
    pip
    virtualenv
    setup.py
    sdist
    wheels
    virtualenvwrapper
    pyenv
    conan
    flit
    bento
    hatch
    pants
    conda
    pipenv
    pip-tools
    poetry
    $

    View Slide

  10. @hayorov
    Who's
    That
    Pokémon?
    Who's
    That
    Pokémon?
    ???
    ???

    View Slide

  11. @hayorov
    Dependency Manager for

    View Slide

  12. @hayorov
    Dependency Manager for
    composer

    View Slide

  13. @hayorov
    Dependency Manager for
    composer

    View Slide

  14. @hayorov
    Dependency Manager for
    composer
    conductor

    View Slide

  15. @hayorov
    Agenda
    The history
    Problem statement
    Modern era
    Hello Pipenv
    Pipenv Poetry
    So now what?

    View Slide

  16. @hayorov
    The Old Days

    View Slide

  17. @hayorov
    The Old Days
    Hey, what’s wrong with curl,
    tar and setup.py install?

    View Slide

  18. @hayorov
    Problems With This
    • “The Cheeseshop” (PyPi) was merely an index of packages,
    not a sole package host.
    • Packages were often hosted elsewhere.
    • Ran on a single server in
    &, 

    while serving the entire Python community.
    • Its use wasn’t a fraction of what it is today, 

    so it wasn’t a problem.

    View Slide

  19. @hayorov
    More Obvious Issues
    • Very manual process - not good for
    automation.
    • Globally installed packages - impossible to have two
    versions of the same library installed.
    • People often just copied things into site-packages, manually.
    • Poor user experience.

    View Slide

  20. @hayorov
    Next Iteration

    View Slide

  21. @hayorov
    Improvements!
    • Much better user experience for installation.
    • Most packages were installed from PyPi.
    • Easier to automate programatically.
    • , no easy_uninstall.

    View Slide

  22. @hayorov
    2010 ->
    • Pip became the de-facto replacement for easy_install.
    • Virtualenv became common practice.
    • Pinned requirements.txt file passed around.

    View Slide

  23. @hayorov
    2010 ->
    • Pip became the de-facto replacement for easy_install.
    • Virtualenv became common practice.
    • Pinned requirements.txt file passed around.
    Pip + Virtualenv + requirements.txt =

    View Slide

  24. @hayorov
    Other Communities
    • Node.js yarn || npm (lockfile)
    • PHP: Composer (lockfile)
    • Rust: Cargo (lockfile)
    • Ruby: Bundler (lockfile)
    • Python: Pip + Virtualenv (no lockfile?)

    View Slide

  25. @hayorov
    ✋ … requirements.txt
    • $ pip freeze > requirements.txt
    • Mismatch: “What you want installed” vs “what you need”
    installed.
    • A pre-flattened dependency tree is required in order to
    establish deterministic builds.
    • Tools like pip-tools were created to ease this pain.

    View Slide

  26. @hayorov
    Two Requirements Files
    $ cat req-to-freeze.txt
    Flask
    $ cat requirements.txt
    click ==6.7
    Flask=0.12.2
    itsdangerous ==0.24
    Jinja2=2.10
    MarkupSafe=1.0
    Werkzeug=0.14.1

    View Slide

  27. @hayorov
    Pipfile: New Standard
    • Pipfile replaces requirements.txt.
    • TOML format, so easy to read/write manually.
    • Two groups: [packages] and [dev-packages].
    • Will eventually land in Pip 19 has support for pyproject.toml

    View Slide

  28. @hayorov
    Resulting Pipfile.lock
    • JSON, so easily machine-parsable
    • Contains all transitive dependancies, pinned, with all
    acceptable hashes for each release.
    • Two groups: {

    “default”: …

    “develop”: …

    }

    View Slide

  29. @hayorov
    Pipenv Sales Pitch
    • Officially recommended tool from python.org.
    • Features: Pipfile/pipfile.lock today.
    • Automates away vrtualenv entirely.
    • Ensures deterministic build (inc. hash check verification).
    • Other useful tools: e.g. $ pipenv graph.

    View Slide

  30. @hayorov
    Downsides
    • Stability! Stability! Stability!
    • Pipenv is probably not the tool for building a library.
    • Pipenv update is really slow.

    • The dependency resolution will fail even if there is a solution.
    • Pipenv remote will not delete dependencies, only the
    package.

    View Slide

  31. @hayorov
    Poetry
    • v0.1.0 released in Feb, 2018 as the reaction to Pipenv issues.
    • Comes with exhaustive and fast dependency resolver.
    • Emphasis on semantic versioning and constraint specification
    (version “^1.4” instead of Pipenv “*”).
    • Easily build and package your projects: poetry publish.

    View Slide

  32. @hayorov
    Poetry
    • v0.1.0 released in Feb, 2018 as the reaction to Pipenv issues.
    • Comes with exhaustive and fast dependency resolver.
    • Emphasis on semantic versioning and constraint specification
    (version “^1.4” instead of Pipenv “*”).
    • Easily build and package your projects: poetry publish.

    View Slide

  33. @hayorov
    So Now What?
    • Start a new project with Poetry
    • Already with Pipenv? Stay on it!
    • Be patient and wait for native support in Pip and Python

    PEP518 Specifying Minimum Build System Requirements for Python Projects

    PEP517 A build-system independent format for source trees

    View Slide

  34. Thank you
    questions…

    @hayorov

    View Slide

  35. Thank you
    questions…

    Alex Khaerov
    @hayorov

    View Slide