Tweet
Tweet

Slide 1

Slide 1 text

@hayorov @hayorov Привет ✋

Slide 2

Slide 2 text

Virtual Environments? Прощай, hayorov Alex Khaerov

Slide 3

Slide 3 text

@hayorov Alex Khaerov company who I am Development Lead @hayorov

Slide 4

Slide 4 text

@hayorov Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov

Slide 5

Slide 5 text

@hayorov Chainstack multi-cloud and multi-blockchain platform as a service based in Singapore # and hiring Alex Khaerov company who I am Development Lead doing software development in the recent decade junior speaker - Python, Kubernetes committee member (Moscow Python, Helm Summit) a huge fan of laptop stickers and newbie @hayorov

Slide 6

Slide 6 text

@hayorov

Slide 7

Slide 7 text

@hayorov easy_install requrements.txt pip virtualenv

Slide 8

Slide 8 text

@hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv

Slide 9

Slide 9 text

@hayorov easy_install requrements.txt pip virtualenv setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

Slide 10

Slide 10 text

@hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent format for source trees setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

Slide 11

Slide 11 text

@hayorov easy_install requrements.txt pip virtualenv PEP 517 A build-system independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

Slide 12

Slide 12 text

@hayorov easy_install requrements.txt pip virtualenv & PEP 517 A build-system independent format for source trees PEP 518 Specifying Minimum Build System Requirements for Python Projects PEP 582 Specifying Minimum Build System Requirements for Python Projects setup.py sdist wheels virtualenvwrapper pyenv conan flit bento hatch pants conda pipenv pip-tools poetry

Slide 13

Slide 13 text

@hayorov KEEP CALM

Slide 14

Slide 14 text

@hayorov Who's That Pokémon? Who's That Pokémon? ??? ???

Slide 15

Slide 15 text

@hayorov Who's That Pokémon? Who's That Pokémon? ??? ???

Slide 16

Slide 16 text

@hayorov Dependency Manager for

Slide 17

Slide 17 text

@hayorov Dependency Manager for composer

Slide 18

Slide 18 text

@hayorov Dependency Manager for composer

Slide 19

Slide 19 text

@hayorov Dependency Manager for composer conductor

Slide 20

Slide 20 text

@hayorov @hayorov Modules, libraries = requirements Where VirtualEnv coming from? Modern era (PEP 517/518) Why Pipenv/Poetry is not enough Welcome PEP 582 Concerns So now what? Agenda

Slide 21

Slide 21 text

@hayorov

Slide 22

Slide 22 text

@hayorov DON’T REPEAT YOURSELF

Slide 23

Slide 23 text

@hayorov The Old Days

Slide 24

Slide 24 text

@hayorov The Old Days Hey, what’s wrong with curl, tar and setup.py install?

Slide 25

Slide 25 text

@hayorov Problems With This • “The Cheeseshop” (PyPi) was merely an index of packages, not a sole package host. • Packages were often hosted elsewhere. • Ran on a single server in ', 
 while serving the entire Python community. • Its use wasn’t a fraction of what it is today, 
 so it wasn’t a problem.

Slide 26

Slide 26 text

@hayorov More Obvious Issues • Very manual process - not good for • Globally installed packages - impossible to have two versions of the same library installed. • People often just copied things into site-packages, manually. • Poor user experience.

Slide 27

Slide 27 text

@hayorov Next Iteration

Slide 28

Slide 28 text

@hayorov Improvements! • Much better user experience for installation. • Most packages were installed from PyPi. • Easier to automate programatically. • , no easy_uninstall.

Slide 29

Slide 29 text

@hayorov 2010 -> … • Pip became the de-facto replacement for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice.

Slide 30

Slide 30 text

@hayorov 2010 -> … • Pip became the de-facto replacement for easy_install. • Pinned requirements.txt file passed around. • Virtualenv became common practice. Pip + Virtualenv + requirements.txt =

Slide 31

Slide 31 text

@hayorov Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv

Slide 32

Slide 32 text

@hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv

Slide 33

Slide 33 text

@hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Oct 17, 2005 virtual-python.py is added to EasyInstall.

Slide 34

Slide 34 text

@hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall.

Slide 35

Slide 35 text

@hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3.

Slide 36

Slide 36 text

@hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool

Slide 37

Slide 37 text

@hayorov Virtualenv Hatch, VirtualEnvManager, autoenv, fades, inve, pew, pipenv, pyenv-virtualenv, pyenv-virtualenvwrapper, pyenv, pyvenv, rvirtualenv, tox, v, venv, vex, virtual-python, virtualenv-burrito, virtualenv-mv, virtualenv, virtualenvwrapper-win, virtualenvwrapper, workingenv Jun 13, 2011 PEP 405 (Python Virtual Environments) is created. Oct 17, 2005 virtual-python.py is added to EasyInstall. May 25, 2012 PEP 405 is accepted for inclusion in Python 3.3. Sep 13, 2015 Python 3.5 is released and venv is recommended tool • Virtualenv is a directory containing a Python interpreter, a special pyvenv.cfg • The “standard tool for creating virtual environments” is venv.

Slide 38

Slide 38 text

2019

Slide 39

Slide 39 text

Pipenv Poetry 2019

Slide 40

Slide 40 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock 2019

Slide 41

Slide 41 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely 2019

Slide 42

Slide 42 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build 2019

Slide 43

Slide 43 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. 2019

Slide 44

Slide 44 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. All the same 2019

Slide 45

Slide 45 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. All the same 2019

Slide 46

Slide 46 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning All the same 2019

Slide 47

Slide 47 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019

Slide 48

Slide 48 text

Pipenv Poetry Was first with pyproject.toml- styled Pipfile/pipfile.lock Automates away Virtualenv entirely Ensures deterministic build Other useful tools e.g. $ pipenv graph. Exhaustive and fast dependency resolver. Emphasis on semantic versioning Easily build and package $ poetry publish All the same 2019

Slide 49

Slide 49 text

@hayorov Pipenv/Poetry is not enough

Slide 50

Slide 50 text

@hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining “virtual environments” is hard
 


Slide 51

Slide 51 text

@hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating

Slide 52

Slide 52 text

@hayorov Pipenv/Poetry is not enough • Learning curve 
 Explaining “virtual environments” is hard
 
 • Terminal isolation
 Activating/deactivating • Cognitive overhead
 Setting up, remembering installation location, 
 activating/deactivating

Slide 53

Slide 53 text

@hayorov @hayorov

Slide 54

Slide 54 text

@hayorov __pypackages __

Slide 55

Slide 55 text

@hayorov PEP 582 - Python local packages directory “This PEP proposes to add to Python a mechanism to automatically recognize a directory and prefer importing packages installed in this location over user or global site-packages." __pypackages __

Slide 56

Slide 56 text

@hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py

Slide 57

Slide 57 text

@hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py

Slide 58

Slide 58 text

@hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder

Slide 59

Slide 59 text

@hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder

Slide 60

Slide 60 text

@hayorov foo | |- myscript.py
 | |- my_lib | |_ __init __.py | |_my_lib.py | |_ __pypackages __ |_ 3.8 |- requests | |_ __init __.py |_ django |_ __init __.py Magic folder PyVersion folder Full structure

Slide 61

Slide 61 text

@hayorov

Slide 62

Slide 62 text

@hayorov Python binary is executed

Slide 63

Slide 63 text

@hayorov Python binary is executed Determine its prefix (as stored in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7'

Slide 64

Slide 64 text

@hayorov Python binary is executed Determine its prefix (as stored in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages'

Slide 65

Slide 65 text

@hayorov Python binary is executed Determine its prefix (as stored in sys.prefix) In [2]: sys.prefix Out[2]: '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7' find the standard library and other key files, and determine site-package In [3]: sys.path Out[3]: ['/usr/local/bin', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload' '', ‘/usr/local/lib/python3.7/site-packages' __pypackages __

Slide 66

Slide 66 text

@hayorov Try it Today

Slide 67

Slide 67 text

@hayorov Try it Today reference CPython implementation https://github.com/kushaldas/cpython/tree/pypackages

Slide 68

Slide 68 text

@hayorov Try it Today reference CPython implementation need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages

Slide 69

Slide 69 text

@hayorov Try it Today reference CPython implementation latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages

Slide 70

Slide 70 text

@hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc

Slide 71

Slide 71 text

@hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pip install --user pythonloc

Slide 72

Slide 72 text

@hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc

Slide 73

Slide 73 text

@hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc

Slide 74

Slide 74 text

@hayorov Try it Today reference CPython implementation proof of concept wrapper pythonloc latest commit accfcb3 on 14 Sep 2018 need to build the binaries yourself https://github.com/kushaldas/cpython/tree/pypackages https://github.com/cs01/pythonloc pythonloc my_script.py pip install --user pythonloc piploc / pipfreezeloc

Slide 75

Slide 75 text

@hayorov Concerns

Slide 76

Slide 76 text

@hayorov Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not.

Slide 77

Slide 77 text

@hayorov Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)?

Slide 78

Slide 78 text

@hayorov Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib

Slide 79

Slide 79 text

@hayorov What about the fallback to site-packages?
 __pypackages__ -> ? -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib

Slide 80

Slide 80 text

@hayorov What about the fallback to site-packages?
 __pypackages__ -> ? -> site-packages Concerns What about entry points?
 pip and virtualenv associate them automatically, PEP582 - not. What about Pipenv (or poetry, or pip)? What about cross platform?
 __pypackages__/windows/3.6/lib or __pypackages__/unix/3.6/lib What about bloat?
 Yes - nodejs

Slide 81

Slide 81 text

@hayorov So Now What? • Start a new project with Poetry • Already with Pipenv? Stay on it! • Be patient and wait for native support of PEP 582 • Wanna try? cs01/pythonloc

Slide 82

Slide 82 text

Thank you questions… Alex Khaerov t.me/hayorov http://bit.ly/xxx