Bye, bye Virtual Environments?

Bye, bye Virtual Environments?

320f3b558c773592bba16c976d1b28d1?s=128

Alex Khaerov

June 24, 2019
Tweet

Transcript

  1. @hayorov @hayorov Привет ✋

  2. Virtual Environments? Прощай, hayorov Alex Khaerov

  3. @hayorov Alex Khaerov company who I am Development Lead @hayorov

  4. @hayorov Alex Khaerov company who I am Development Lead doing

    software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov
  5. @hayorov Chainstack multi-cloud and multi-blockchain platform as a service based

    in Singapore # and hiring Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov
  6. @hayorov

  7. @hayorov easy_install requrements.txt pip virtualenv

  8. @hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv

  9. @hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv

    conan flit bento hatch pants conda pipenv pip-tools poetry
  10. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent

    format for source trees setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
  11. @hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent

    format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
  12. @hayorov easy_install requrements.txt pip virtualenv & PEP 517 A build-system

    independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects PEP 582 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry
  13. @hayorov KEEP CALM

  14. @hayorov Who's That Pokémon? Who's That Pokémon? ??? ???

  15. @hayorov Who's That Pokémon? Who's That Pokémon? ??? ???

  16. @hayorov Dependency Manager for

  17. @hayorov Dependency Manager for composer

  18. @hayorov Dependency Manager for composer

  19. @hayorov Dependency Manager for composer conductor

  20. @hayorov @hayorov Modules, libraries = requirements Where VirtualEnv coming from?

    Modern era (PEP 517/518) Why Pipenv/Poetry is not enough Welcome PEP 582 Concerns So now what? Agenda
  21. @hayorov

  22. @hayorov DON’T REPEAT YOURSELF

  23. @hayorov The Old Days

  24. @hayorov The Old Days Hey, what’s wrong with curl, tar

    and setup.py install?
  25. @hayorov Problems With This • “The Cheeseshop” (PyPi) was merely

    an index of packages, not a sole package host. • Packages were often hosted elsewhere. • Ran on a single server in ', 
 while serving the entire Python community. • Its use wasn’t a fraction of what it is today, 
 so it wasn’t a problem.
  26. @hayorov More Obvious Issues • Very manual process - not

    good for • Globally installed packages - impossible to have two versions of the same library installed. • People often just copied things into site-packages, manually. • Poor user experience.
  27. @hayorov Next Iteration

  28. @hayorov Improvements! • Much better user experience for installation. •

    Most packages were installed from PyPi. • Easier to automate programatically. • , no easy_uninstall.
  29. @hayorov 2010 -> … • Pip became the de-facto replacement

    for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice.
  30. @hayorov 2010 -> … • Pip became the de-facto replacement

    for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice. Pip + Virtualenv + requirements.txt =
  31. @hayorov Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper,

    pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv
  32. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv
  33. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Oct 17, 2005 virtual-python.py is added to EasyInstall.
  34. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall.
  35. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3.
  36. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool
  37. @hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv,

    pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool • Virtualenv is a directory containing a Python interpreter, a special pyvenv.cfg • The “standard tool for creating virtual environments” is venv.
  38. 2019

  39. Pipenv Poetry 2019

  40. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock 2019

  41. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely 2019
  42. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build 2019
  43. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. 2019
  44. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. All the same 2019
  45. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. All the same 2019
  46. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning All the same 2019
  47. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019
  48. Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away

    Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019
  49. @hayorov Pipenv/Poetry is not enough

  50. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 

  51. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating
  52. @hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining

    “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating • Cognitive overhead
 Setting up, remembering installation location, 
 activating/deactivating
  53. @hayorov @hayorov

  54. @hayorov __pypackages __

  55. @hayorov PEP 582 - Python local packages directory “This PEP

    proposes to add to Python a mechanism to automatically recognize a directory and prefer importing packages installed in this location over user or global site-packages." __pypackages __
  56. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py
  57. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py
  58. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder
  59. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder
  60. @hayorov foo | |- myscript.py
 | |- my_lib | |_

    __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder Full structure
  61. @hayorov

  62. @hayorov Python binary is executed

  63. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7'
  64. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages'
  65. @hayorov Python binary is executed Determine its prefix (as stored

    in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages' __pypackages __
  66. @hayorov Try it Today

  67. @hayorov Try it Today reference CPython implementation https://github.com/kushaldas/cpython/tree/pypackages

  68. @hayorov Try it Today reference CPython implementation need to build

    the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages
  69. @hayorov Try it Today reference CPython implementation latest commit accfcb3

    on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages
  70. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc
  71. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pip install --user pythonloc
  72. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc
  73. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc
  74. @hayorov Try it Today reference CPython implementation proof of concept

    wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc
  75. @hayorov Concerns

  76. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not.
  77. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)?
  78. @hayorov Concerns What about entry points?
 pip and virtualenv associate

    them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib
  79. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ?

    -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib
  80. @hayorov What about the fallback to site-packages?
 __pypackages__ -> ?

    -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib What about bloat?
 Yes - nodejs
  81. @hayorov So Now What? • Start a new project with

    Poetry • Already with Pipenv? Stay on it! • Be patient and wait for native support of PEP 582 • Wanna try? cs01/pythonloc
  82. Thank you questions… Alex Khaerov t.me/hayorov http://bit.ly/xxx