Bye, bye Virtual Environments?

Transcript

1. @hayorov @hayorov Привет ✋

2. Virtual Environments? Прощай, hayorov Alex Khaerov

3. @hayorov Alex Khaerov company who I am Development Lead @hayorov

4. @hayorov Alex Khaerov company who I am Development Lead doing

   software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov

5. @hayorov Chainstack multi-cloud and multi-blockchain platform as a service based

   in Singapore # and hiring Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov

6. @hayorov

7. @hayorov easy_install requrements.txt pip virtualenv

8. @hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv

9. @hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv

   conan flit bento hatch pants conda pipenv pip-tools poetry

10. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent

    format for source trees setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

11. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent

    format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

12. @hayorov easy_install requrements.txt pip virtualenv & PEP 517 A build-system

    independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects PEP 582 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

13. @hayorov KEEP CALM

14. @hayorov Who's That Pokémon? Who's That Pokémon? ??? ???

15. @hayorov Who's That Pokémon? Who's That Pokémon? ??? ???

16. @hayorov Dependency Manager for

17. @hayorov Dependency Manager for composer

18. @hayorov Dependency Manager for composer

19. @hayorov Dependency Manager for composer conductor

20. @hayorov @hayorov Modules, libraries = requirements Where VirtualEnv coming from?

    Modern era (PEP 517/518) Why Pipenv/Poetry is not enough Welcome PEP 582 Concerns So now what? Agenda

21. @hayorov

22. @hayorov DON’T REPEAT YOURSELF

23. @hayorov The Old Days

24. @hayorov The Old Days Hey, what’s wrong with curl, tar

    and setup.py install?

25. @hayorov Problems With This • “The Cheeseshop” (PyPi) was merely

    an index of packages, not a sole package host. • Packages were often hosted elsewhere. • Ran on a single server in ', 
 while serving the entire Python community. • Its use wasn’t a fraction of what it is today, 
 so it wasn’t a problem.

26. @hayorov More Obvious Issues • Very manual process - not

    good for • Globally installed packages - impossible to have two versions of the same library installed. • People often just copied things into site-packages, manually. • Poor user experience.

27. @hayorov Next Iteration

28. @hayorov Improvements! • Much better user experience for installation. •

    Most packages were installed from PyPi. • Easier to automate programatically. • , no easy_uninstall.

29. @hayorov 2010 -> … • Pip became the de-facto replacement

    for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice.

30. @hayorov 2010 -> … • Pip became the de-facto replacement

    for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice. Pip + Virtualenv + requirements.txt =

31. @hayorov Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper,

    pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv

32. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv

33. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Oct 17, 2005 virtual-python.py is added to EasyInstall.

34. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall.

35. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3.

36. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool

37. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool • Virtualenv is a directory containing a Python interpreter, a special pyvenv.cfg • The “standard tool for creating virtual environments” is venv.

38. 2019

39. Pipenv Poetry 2019

40. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock 2019

41. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely 2019

42. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build 2019

43. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. 2019

44. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. All the same 2019

45. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. All the same 2019

46. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning All the same 2019

47. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019

48. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019

49. @hayorov Pipenv/Poetry is not enough

50. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 


51. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating

52. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating • Cognitive overhead
 Setting up, remembering installation location, 
 activating/deactivating

53. @hayorov @hayorov

54. @hayorov __pypackages __

55. @hayorov PEP 582 - Python local packages directory “This PEP

    proposes to add to Python a mechanism to automatically recognize a directory and prefer importing packages installed in this location over user or global site-packages." __pypackages __

56. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py

57. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py

58. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder

59. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder

60. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder Full structure

61. @hayorov

62. @hayorov Python binary is executed

63. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7'

64. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages'

65. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages' __pypackages __

66. @hayorov Try it Today

67. @hayorov Try it Today reference CPython implementation https://github.com/kushaldas/cpython/tree/pypackages

68. @hayorov Try it Today reference CPython implementation need to build

    the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages

69. @hayorov Try it Today reference CPython implementation latest commit accfcb3

    on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages

70. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc

71. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pip install --user pythonloc

72. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc

73. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc

74. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc

75. @hayorov Concerns

76. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not.

77. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)?

78. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib

79. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ?

    -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib

80. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ?

    -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib What about bloat?
 Yes - nodejs

81. @hayorov So Now What? • Start a new project with

    Poetry • Already with Pipenv? Stay on it! • Be patient and wait for native support of PEP 582 • Wanna try? cs01/pythonloc

82. Thank you questions… Alex Khaerov t.me/hayorov http://bit.ly/xxx