"Making Sense of Python Virtual Environment Tools and Workflows" by Daniel Holmes

7b0645f018c0bddc8ce3900ccc3ba70c?s=47 Pycon ZA
October 10, 2019

"Making Sense of Python Virtual Environment Tools and Workflows" by Daniel Holmes

One of the challenges developers at any level face is managing their dependencies and development environments. Python is no exception. In the Python ecosystem, there are a number of tools for managing dependencies and environments. However, many of these tools have similar names with slightly differing functions and workflows (e.g: virtualenv, pyenv, venv, virtualenvwrapper, pipenv, and so on...) Furthermore, a number of new tools in this space continue to surface each year. This often creates confusion and appears to contradict the adage: “There should be one – and preferably only one – obvious way to do it.” -- The Zen of Python

In the absence of "one obvious way", the aim of this talk is to help developers at any level better understand the current tools available, what each achieves, and how they can potentially make the developer's life easier. It will cover Python dependency management and the importance of using virtual environments. An explanation and comparison of these different tools, including newer ones, will be also be presented.


Pycon ZA

October 10, 2019


  1. Making Sense of Python Virtual Environment Tools and Workflows Daniel

    Holmes PyConZA 2019 1
  2. Agenda • Python dependency management • Virtual environments • Comparison

    of tools Daniel Holmes | PyConZA 2019 2
  3. “Dependency management is like your city’s sewage system. When it’s

    working well, it’s easy to forget that it even exists. The only time you’ll remember it is when you experience the agony induced by its failure.” – Paul Kernfeld 3 Daniel Holmes | PyConZA 2019
  4. Daniel Holmes | PyConZA 2019 4

  5. pip – pip installs packages 5 Daniel Holmes | PyConZA

  6. Daniel Holmes | PyConZA 2019 6

  7. 7 Daniel Holmes | PyConZA 2019

  8. Daniel Holmes | PyConZA 2019 8 Virtual Environment 1 Python

    2.7 NumPy SciPy Pandas 0.20.3 Virtual Environment 2 Python 3.6 Requests BeautifulSoup Pandas 0.24.4 Virtual Environment 3 Python 3.7 Django Celery Pandas 0.25.1 Virtual Environment Tool pip
  9. “There should be one-- and preferably only one --obvious way

    to do it.” - The Zen of Python 9 Daniel Holmes | PyConZA 2019
  10. (py)?(v|virtual|pip)?env 10 Daniel Holmes | PyConZA 2019 virtualenv pipenv pyenv

    venv pyvenv
  11. 11 Daniel Holmes | PyConZA 2019 Hatch Poetry Conda Miniconda

  12. Java is to JavaScript as Car is to Carpet 12

    Daniel Holmes | PyConZA 2019
  13. 13 Daniel Holmes | PyConZA 2019

  14. 14 Daniel Holmes | PyConZA 2019

  15. 15 Daniel Holmes | PyConZA 2019

  16. virtualenv Most popular Works with Python 2 and 3 Copies

    Python binaries 16 Daniel Holmes | PyConZA 2019
  17. venv Standard library since Python 3.3 Similar to virtualenv, but

    done right Does not copy Python binaries 17 Daniel Holmes | PyConZA 2019
  18. pipenv Recommended by the PyPA* High level tool Deterministic builds

    of dependencies 18 Daniel Holmes | PyConZA 2019 * not the Python core team!
  19. pyenv Manage multiple versions of Python Has a plugin for

    virtualenv 19 Daniel Holmes | PyConZA 2019
  20. pyvenv Came with Python 3... but deprecated in Python 3.6

    20 Daniel Holmes | PyConZA 2019
  21. conda Cross platform, language agnostic Used mainly for data science

    Can replace virtualenv 21 Daniel Holmes | PyConZA 2019
  22. Which tool should I use? It depends... 22 Daniel Holmes

    | PyConZA 2019
  23. Tool Use case virtualenv General Python development, Python 2 and

    3 pipenv Project workflows, web development pyenv Manage multiple Python versions venv General Python development, Python 3.3+ Conda Multi-language projects, data science, cross platform Hatch Packaging support Poetry Packaging support, better dependency resolution Daniel Holmes | PyConZA 2019 23
  24. 24 Daniel Holmes | PyConZA 2019 Aleks Hughes: https://flexibleframework.xyz/2019/04/python-virtual-env-shootout/

  25. Summary • Standard Python dependency resolution isn’t great • We

    need virtual environments • Many tools with similar names creates confusion • 3 best tools to choose from: • venv • pipenv • conda Daniel Holmes | PyConZA 2019 25
  26. Tool Pros Cons virtualenv Most widely used, works in Python

    2 and 3 Copies around Python binaries pipenv Does a lot of work for you Not great working outside the project root. Can be slow. No packaging. pyenv Can use with virtualenv Need to install separate packages venv Standard library in Python 3, does not copy Python binaries Only Python 3.3+ pyvenv Still available < Python 3.6 Deprecated in Python 3.6 Conda Cross platform, supports multi- language projects Need full Anaconda to share packages across environments Hatch Takes care of everything in packaging process Doesn’t manage dependencies Poetry Support for packaging, tries to overcome pipenv shortcomings Doesn’t do things in a standardised way Daniel Holmes | PyConZA 2019 26
  27. Thank You 27 Daniel Holmes | PyConZA 2019