Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bye, bye Virtual Environments?

Bye, bye Virtual Environments?

Alex Khaerov

June 24, 2019
Tweet

More Decks by Alex Khaerov

Other Decks in Programming

Transcript

  1. @hayorov Alex Khaerov company who I am Development Lead doing

    software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov
  2. @hayorov Chainstack multi-cloud and multi-blockchain platform as a service based

    in Singapore # and hiring Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov
  3. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent

    format for source trees setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
  4. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent

    format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
  5. @hayorov easy_install requrements.txt pip virtualenv & PEP 517 A build-system

    independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects PEP 582 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
  6. @hayorov @hayorov Modules, libraries = requirements Where VirtualEnv coming from?

    Modern era (PEP 517/518) Why Pipenv/Poetry is not enough Welcome PEP 582 Concerns So now what? Agenda
  7. @hayorov Problems With This • “The Cheeseshop” (PyPi) was merely

    an index of packages, not a sole package host. • Packages were often hosted elsewhere. • Ran on a single server in ', 
 while serving the entire Python community. • Its use wasn’t a fraction of what it is today, 
 so it wasn’t a problem.
  8. @hayorov More Obvious Issues • Very manual process - not

    good for • Globally installed packages - impossible to have two versions of the same library installed. • People often just copied things into site-packages, manually. • Poor user experience.
  9. @hayorov Improvements! • Much better user experience for installation. •

    Most packages were installed from PyPi. • Easier to automate programatically. • , no easy_uninstall.
  10. @hayorov 2010 -> … • Pip became the de-facto replacement

    for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice.
  11. @hayorov 2010 -> … • Pip became the de-facto replacement

    for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice. Pip + Virtualenv + requirements.txt =
  12. @hayorov Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper,

    pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv
  13. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv
  14. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Oct 17, 2005 virtual-python.py is added to EasyInstall.
  15. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall.
  16. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3.
  17. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool
  18. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool • Virtualenv is a directory containing a Python interpreter, a special pyvenv.cfg • The “standard tool for creating virtual environments” is venv.
  19. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. 2019
  20. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. All the same 2019
  21. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. All the same 2019
  22. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning All the same 2019
  23. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019
  24. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019
  25. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating
  26. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating • Cognitive overhead
 Setting up, remembering installation location, 
 activating/deactivating
  27. @hayorov PEP 582 - Python local packages directory “This PEP

    proposes to add to Python a mechanism to automatically recognize a directory and prefer importing packages installed in this location over user or global site-packages." __pypackages __
  28. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py
  29. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder
  30. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder
  31. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder Full structure
  32. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7'
  33. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages'
  34. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages' __pypackages __
  35. @hayorov Try it Today reference CPython implementation need to build

    the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages
  36. @hayorov Try it Today reference CPython implementation latest commit accfcb3

    on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages
  37. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc
  38. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pip install --user pythonloc
  39. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc
  40. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc
  41. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc
  42. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)?
  43. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib
  44. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ?

    -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib
  45. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ?

    -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib What about bloat?
 Yes - nodejs
  46. @hayorov So Now What? • Start a new project with

    Poetry • Already with Pipenv? Stay on it! • Be patient and wait for native support of PEP 582 • Wanna try? cs01/pythonloc