"Making Sense of Python Virtual Environment Tools and Workflows" by Daniel Holmes

One of the challenges developers at any level face is managing their dependencies and development environments. Python is no exception. In the Python ecosystem, there are a number of tools for managing dependencies and environments. However, many of these tools have similar names with slightly differing functions and workflows (e.g: virtualenv, pyenv, venv, virtualenvwrapper, pipenv, and so on...) Furthermore, a number of new tools in this space continue to surface each year. This often creates confusion and appears to contradict the adage: “There should be one – and preferably only one – obvious way to do it.” -- The Zen of Python

In the absence of "one obvious way", the aim of this talk is to help developers at any level better understand the current tools available, what each achieves, and how they can potentially make the developer's life easier. It will cover Python dependency management and the importance of using virtual environments. An explanation and comparison of these different tools, including newer ones, will be also be presented.

Transcript

1. Making Sense of Python Virtual Environment Tools and Workflows Daniel Holmes PyConZA 2019 1
2. Agenda • Python dependency management • Virtual environments • Comparison of tools Daniel Holmes | PyConZA 2019 2
3. “Dependency management is like your city’s sewage system. When it’s working well, it’s easy to forget that it even exists. The only time you’ll remember it is when you experience the agony induced by its failure.” – Paul Kernfeld 3 Daniel Holmes | PyConZA 2019
4. Daniel Holmes | PyConZA 2019 4
5. pip – pip installs packages 5 Daniel Holmes | PyConZA 2019
6. Daniel Holmes | PyConZA 2019 6
7. 7 Daniel Holmes | PyConZA 2019
8. Daniel Holmes | PyConZA 2019 8 Virtual Environment 1 Python 2.7 NumPy SciPy Pandas 0.20.3 Virtual Environment 2 Python 3.6 Requests BeautifulSoup Pandas 0.24.4 Virtual Environment 3 Python 3.7 Django Celery Pandas 0.25.1 Virtual Environment Tool pip
9. “There should be one-- and preferably only one --obvious way to do it.” - The Zen of Python 9 Daniel Holmes | PyConZA 2019
10. (py)?(v|virtual|pip)?env 10 Daniel Holmes | PyConZA 2019 virtualenv pipenv pyenv venv pyvenv
11. 11 Daniel Holmes | PyConZA 2019 Hatch Poetry Conda Miniconda Anaconda
12. Java is to JavaScript as Car is to Carpet 12 Daniel Holmes | PyConZA 2019
13. 13 Daniel Holmes | PyConZA 2019
14. 14 Daniel Holmes | PyConZA 2019
15. 15 Daniel Holmes | PyConZA 2019
16. virtualenv Most popular Works with Python 2 and 3 Copies Python binaries 16 Daniel Holmes | PyConZA 2019
17. venv Standard library since Python 3.3 Similar to virtualenv, but done right Does not copy Python binaries 17 Daniel Holmes | PyConZA 2019
18. pipenv Recommended by the PyPA* High level tool Deterministic builds of dependencies 18 Daniel Holmes | PyConZA 2019 * not the Python core team!
19. pyenv Manage multiple versions of Python Has a plugin for virtualenv 19 Daniel Holmes | PyConZA 2019
20. pyvenv Came with Python 3... but deprecated in Python 3.6 20 Daniel Holmes | PyConZA 2019
21. conda Cross platform, language agnostic Used mainly for data science Can replace virtualenv 21 Daniel Holmes | PyConZA 2019
22. Which tool should I use? It depends... 22 Daniel Holmes | PyConZA 2019
23. Tool Use case virtualenv General Python development, Python 2 and 3 pipenv Project workflows, web development pyenv Manage multiple Python versions venv General Python development, Python 3.3+ Conda Multi-language projects, data science, cross platform Hatch Packaging support Poetry Packaging support, better dependency resolution Daniel Holmes | PyConZA 2019 23
24. 24 Daniel Holmes | PyConZA 2019 Aleks Hughes: https://flexibleframework.xyz/2019/04/python-virtual-env-shootout/
25. Summary • Standard Python dependency resolution isn’t great • We need virtual environments • Many tools with similar names creates confusion • 3 best tools to choose from: • venv • pipenv • conda Daniel Holmes | PyConZA 2019 25
26. Tool Pros Cons virtualenv Most widely used, works in Python 2 and 3 Copies around Python binaries pipenv Does a lot of work for you Not great working outside the project root. Can be slow. No packaging. pyenv Can use with virtualenv Need to install separate packages venv Standard library in Python 3, does not copy Python binaries Only Python 3.3+ pyvenv Still available < Python 3.6 Deprecated in Python 3.6 Conda Cross platform, supports multi- language projects Need full Anaconda to share packages across environments Hatch Takes care of everything in packaging process Doesn’t manage dependencies Poetry Support for packaging, tries to overcome pipenv shortcomings Doesn’t do things in a standardised way Daniel Holmes | PyConZA 2019 26
27. Thank You 27 Daniel Holmes | PyConZA 2019