Bye, bye Virtual Environments?

Transcript

1. @hayorov @hayorov Привет ✋
2. Virtual Environments? Прощай, hayorov Alex Khaerov
3. @hayorov Alex Khaerov company who I am Development Lead @hayorov
4. @hayorov Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov
5. @hayorov Chainstack multi-cloud and multi-blockchain platform as a service based in Singapore # and hiring Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov
6. @hayorov
7. @hayorov easy_install requrements.txt pip virtualenv
8. @hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv
9. @hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
10. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent format for source trees setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
11. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
12. @hayorov easy_install requrements.txt pip virtualenv & PEP 517 A build-system independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects PEP 582 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
13. @hayorov KEEP CALM
14. @hayorov Who's That Pokémon? Who's That Pokémon? ??? ???
15. @hayorov Who's That Pokémon? Who's That Pokémon? ??? ???
16. @hayorov Dependency Manager for
17. @hayorov Dependency Manager for composer
18. @hayorov Dependency Manager for composer
19. @hayorov Dependency Manager for composer conductor
20. @hayorov @hayorov Modules, libraries = requirements Where VirtualEnv coming from? Modern era (PEP 517/518) Why Pipenv/Poetry is not enough Welcome PEP 582 Concerns So now what? Agenda
21. @hayorov
22. @hayorov DON’T REPEAT YOURSELF
23. @hayorov The Old Days
24. @hayorov The Old Days Hey, what’s wrong with curl, tar and setup.py install?
25. @hayorov Problems With This • “The Cheeseshop” (PyPi) was merely an index of packages, not a sole package host. • Packages were often hosted elsewhere. • Ran on a single server in ', 
 while serving the entire Python community. • Its use wasn’t a fraction of what it is today, 
 so it wasn’t a problem.
26. @hayorov More Obvious Issues • Very manual process - not good for • Globally installed packages - impossible to have two versions of the same library installed. • People often just copied things into site-packages, manually. • Poor user experience.
27. @hayorov Next Iteration
28. @hayorov Improvements! • Much better user experience for installation. • Most packages were installed from PyPi. • Easier to automate programatically. • , no easy_uninstall.
29. @hayorov 2010 -> … • Pip became the de-facto replacement for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice.
30. @hayorov 2010 -> … • Pip became the de-facto replacement for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice. Pip + Virtualenv + requirements.txt =
31. @hayorov Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv
32. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv
33. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Oct 17, 2005 virtual-python.py is added to EasyInstall.
34. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall.
35. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3.
36. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool
37. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool • Virtualenv is a directory containing a Python interpreter, a special pyvenv.cfg • The “standard tool for creating virtual environments” is venv.
38. 2019
39. Pipenv Poetry 2019
40. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock 2019
41. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely 2019
42. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build 2019
43. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. 2019
44. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. All the same 2019
45. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. All the same 2019
46. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning All the same 2019
47. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019
48. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019
49. @hayorov Pipenv/Poetry is not enough
50. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining “virtual environments” is hard
 

51. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating
52. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating • Cognitive overhead
 Setting up, remembering installation location, 
 activating/deactivating
53. @hayorov @hayorov
54. @hayorov __pypackages __
55. @hayorov PEP 582 - Python local packages directory “This PEP proposes to add to Python a mechanism to automatically recognize a directory and prefer importing packages installed in this location over user or global site-packages." __pypackages __
56. @hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py
57. @hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py
58. @hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder
59. @hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder
60. @hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder Full structure
61. @hayorov
62. @hayorov Python binary is executed
63. @hayorov Python binary is executed Determine its prefix (as stored in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7'
64. @hayorov Python binary is executed Determine its prefix (as stored in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages'
65. @hayorov Python binary is executed Determine its prefix (as stored in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages' __pypackages __
66. @hayorov Try it Today
67. @hayorov Try it Today reference CPython implementation https://github.com/kushaldas/cpython/tree/pypackages
68. @hayorov Try it Today reference CPython implementation need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages
69. @hayorov Try it Today reference CPython implementation latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages
70. @hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc
71. @hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pip install --user pythonloc
72. @hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc
73. @hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc
74. @hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc
75. @hayorov Concerns
76. @hayorov Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not.
77. @hayorov Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)?
78. @hayorov Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib
79. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ? -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib
80. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ? -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib What about bloat?
 Yes - nodejs
81. @hayorov So Now What? • Start a new project with Poetry • Already with Pipenv? Stay on it! • Be patient and wait for native support of PEP 582 • Wanna try? cs01/pythonloc
82. Thank you questions… Alex Khaerov t.me/hayorov http://bit.ly/xxx