Developer Ergonomics

Keynote for PyCaribbean

Transcript

1. Problem
2. You'll eventually want to install third party packages...
3. Enter easy_install
4. Enter easy_install
5. Enter pip
6. $ pip install Django==1.8
7. You just installed Django globally.
8. $ pip install Django==1.10
9. You just installed Django globally.
10. Developer Ergonomics
11. Package Managers and Environments
12. José Padilla
13. Work Training Open Source
14. Work Training Open Source
15. Work Training Open Source
16. Backstory
17. Lets begin...
18. Node.js + NPM
19. Node Package Manager
20. Installing packages with npm is easy.
21. $ npm install express
22. └─┬ express@4.14.1 ├─┬ accepts@1.3.3 │ ├─┬ mime-types@2.1.14 │ │ └── mime-db@1.26.0 │ └── negotiator@0.6.1 ├── array-flatten@1.1.1 ├── content-disposition@0.5.2 ├── content-type@1.0.2 ├── cookie@0.3.1 ├── cookie-signature@1.0.6 ├─┬ debug@2.2.0 │ └── ms@0.7.1 ├── depd@1.1.0 ├── encodeurl@1.0.1 ├── escape-html@1.0.3 ├── etag@1.7.0 ├─┬ finalhandler@0.5.1 │ ├── statuses@1.3.1 │ └── unpipe@1.0.0 ├── fresh@0.3.0 ├── merge-descriptors@1.0.1 ├── methods@1.1.2 ├─┬ on-finished@2.3.0 │ └── ee-first@1.1.1 ├── parseurl@1.3.1 ├── path-to-regexp@0.1.7 ├─┬ proxy-addr@1.1.3 │ ├── forwarded@0.1.0 │ └── ipaddr.js@1.2.0 ├── qs@6.2.0 ├── range-parser@1.2.0 ├─┬ send@0.14.2 │ ├── destroy@1.0.4 │ ├─┬ http-errors@1.5.1 │ │ ├── inherits@2.0.3 │ │ └── setprototypeof@1.0.2 │ ├── mime@1.3.4 │ └── ms@0.7.2 ├── serve-static@1.11.2 ├─┬ type-is@1.6.14 │ └── media-typer@0.3.0 ├── utils-merge@1.0.0 └── vary@1.1.0 └─┬ express@4.14.1 ├─┬ accepts@1.3.3 │ ├─┬ mime-types@2.1.14 │ │ └── mime-db@1.26.0 │ └── negotiator@0.6.1 ├── array-flatten@1.1.1 ├── content-disposition@0.5.2 ├── content-type@1.0.2 ├── cookie@0.3.1 ├── cookie-signature@1.0.6 ├─┬ debug@2.2.0 │ └── ms@0.7.1 ├── depd@1.1.0 ├── encodeurl@1.0.1 ├── escape-html@1.0.3
23. $ npm install express --save
24. { "name": "pycaribbean", "version": "1.0.0", "dependencies": { "express": "^4.14.1" } }
25. $ npm uninstall express --save
26. { "name": "pycaribbean", "version": "1.0.0", "dependencies": {} }
27. The default behavior of npm is to install packages locally to the current directory.
28. global = ?
29. global = CLI tools
30. $ npm install eslint --global
31. Dev / Prod
32. $ npm install supertest --save-dev
33. { "name": "nodejs", "version": "1.0.0", "dependencies": {}, "devDependencies": { "supertest": "^3.0.0" } }
34. $ npm install
35. $ npm install --production
36. Dependency Conflicts
37. $ npm install package-a --save
38. └─┬ package-a@1.0.0 └── request@1.0.0
39. $ npm install package-b --save
40. └─┬ package-b@1.0.0 └── request@2.0.0
41. ├─┬ package-a@1.0.0 │ └── request@1.0.0 └─┬ package-b@1.0.0 └── request@2.0.0
42. Determinism
44. Determinism
45. $ npm shrinkwrap
46. { "name": "A", "version": "1.1.0", "dependencies": { "B": { "version": "1.0.1", "from": "B@^1.0.0", "resolved": "..." } } }
47. Node.js + Yarn
48. Yarn
49. Fast
50. Reliable
51. Secure
52. Installing packages with yarn is easy.
53. $ yarn
54. $ yarn add express
55. $ yarn add --dev mocha
56. $ yarn global add eslint
57. Determinism
58. # THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY. # yarn lockfile v1 package-1@^1.0.0: version "1.0.3" resolved "https://registry.npmjs.org/package-1/-/ package-1-1.0.3.tgz#c3ca7434938648c3e0d9c1e328dd68b622c2 84ca" package-2@^2.0.0: version "2.0.1" resolved "https://registry.npmjs.org/package-2/-/ package-2-2.0.1.tgz#9a5f699051b1e7073328f2a008968b64ea29 55d2" dependencies: package-4 "^4.0.0"
59. Rust + Cargo
60. Cargo
61. Cross-pollination at its best.
62. Cargo.toml
63. [package] name = "package_a" version = "0.1.0" authors = ["root"] [dependencies] package_b = "0.0.1"
64. Cargo.lock
65. [root] name = "package_a" version = "0.0.1" dependencies = [ "package_b 0.0.1 (...)", ] [[package]] name = "package_b" version = "0.0.1" source = "..." [metadata] "checksum package_b 0.0.1 (...)" = "7507624b29483431c0ba2d82aece"
66. Python + PIP
67. PIP
68. Installing packages with pip is easy.
69. $ pip install flask
70. $ pip install flask $ pip freeze > requirements.txt
71. click==6.7 Flask==0.12 itsdangerous==0.24 Jinja2==2.9.5 MarkupSafe==0.23 Werkzeug==0.11.15
72. $ pip uninstall flask $ pip freeze > requirements.txt
73. click==6.7 itsdangerous==0.24 Jinja2==2.9.5 MarkupSafe==0.23 Werkzeug==0.11.15
74. Dev / Prod
75. ├── requirements │ ├── common.txt │ ├── dev.txt │ └── prod.txt └── requirements.txt
76. $ cat requirements.txt -r requirements/prod.txt
77. $ cat requirements/prod.txt -r common.txt gunicorn==19.6.0 newrelic==2.74.0.54
78. $ cat requirements/common.txt Django==1.10.5
79. $ cat requirements/dev.txt pytest==3.0.6
80. $ pip install -r requirements.txt
81. $ pip install -r requirements/dev.txt
82. The default behavior of pip is to install packages globally.
83. Installing local dependencies
84. Python 2
85. virtualenv
86. $ pip install virtualenv
87. $ virtualenv .venv
88. $ source .venv/bin/activate
89. $ .venv/bin/python Python 2.7.9 (default, Dec 21 2016, 01:15:20) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>
90. (.venv) $ which python /path/to/.venv/bin/python
91. (.venv) $ pip install flask
92. (.venv) $ deactivate
93. $ which python /usr/bin/python
94. Python 3
95. venv
96. $ python3 -m venv .venv
97. $ source .venv/bin/activate
98. (.venv) $ pip install flask
99. (.venv) $ deactivate
100. Dependency Conflicts
101. package-a Depends on requests==1.0.0
102. package-b Depends on requests==2.12.4
103. $ pip install package-a $ pip install package-b $ pip freeze
104. package-a==0.1 package-b==0.1 requests==2.12.4
105. Packages and dependencies are flat.
106. $ pip check package-a has requirement requests==1.0.0, but you have requests 2.12.4.
107. $ pip install flask && pip check
108. function pip-install() { pip install $@ && pip check }
109. $ pip-install flask
110. Going forward
111. Changing the defaults of tools like pip without creating chaos is hard.
112. Installing packages in Python could be easier.
113. "One of the hurdles that new Python developers have to get over is understanding the Python packaging ecosystem." – Jamie Matthews (@j4mie)
114. "Variables in Python are local by default, so why aren't packages? You have to go out of your way to make a global variable. You should also have to go out of your way to install a package globally." – Trey Hunner (@treyhunner)
115. My ideal scenario
116. Default to installing packages locally.
117. Improved dependency resolving.
118. Fully specified and deterministic.
119. Secure.
120. $ pip init
121. $ pip install flask
122. $ pip install --dev pytest
123. $ pip install
124. How does it work?
125. IDK
126. "There's someone already working on a similar idea"
127. Kenneth Reitz open sourced pipenv on January 19 2017
128. Python + Pipenv
129. More than 15 hours on Hacker News' front page.
130. More than 2500 stars on GitHub.
131. Dogfooding kennethreitz/requests
132. Pipenv
133. Sacred Marriage of Pipfile, Pip, & Virtualenv
134. Cross-pollination at its best.
135. Pipfile
136. Requirements 2.0
137. Pipfile will be superior to requirements.txt file in a number of ways...
138. TOML syntax for declaring all types of Python dependencies.
139. One Pipfile, as opposed to multiple requirements.txt files.
140. [dev-packages] pytest = "*" [packages] flask = "*"
141. Pipfile.lock
142. { "default": { "MarkupSafe": { "version": "==0.23", "hash": "sha256:a4ec1aff59b95a1..." }, "Jinja2": { "version": "==2.9.5", "hash": "sha256:a7b7438120dbe76..." }, "Werkzeug": { "version": "==0.11.15", "hash": "sha256:c6f6f89124df051..." }, "flask": { "version": "==0.12", "hash": "sha256:7f03bb2c2554524..." }, "itsdangerous": { "version": "==0.24", "hash": "sha256:cbb3fcf8d3e33df..." }, "click": { "version": "==6.7", "hash": "sha256:29f99fc6125fbc9..." } }, "develop": { "packaging": { "version": "==16.8", "hash": "sha256:99276dc6e3a7851..." }, "pytest": { "version": "==3.0.6", "hash": "sha256:da0ab50c7eec068..." }, "setuptools": { "version": "==34.1.1", "hash": "sha256:5f74aabe68c441b..." }, "pyparsing": { "version": "==2.1.10", "hash": "sha256:67101d7acee6929..." }, "py": { "version": "==1.4.32", "hash": "sha256:2d4bba2e25fff58..." }, "six": { "version": "==1.10.0", "hash": "sha256:0ff78c403d9bccf..." }, "appdirs": { "version": "==1.4.0", "hash": "sha256:85e58578db8f295..." } }, "_meta": { "sources": [ { "url": "https://pypi.python.org/simple", "verify_ssl": true } ], "requires": {}, "hash": { "sha256": "c755a9d5787ce2fdc70f7..." } } } { "default": { "MarkupSafe": { "version": "==0.23", "hash": "sha256:a4ec1aff59b95a1..." }, "Jinja2": { "version": "==2.9.5", "hash": "sha256:a7b7438120dbe76..." }, "Werkzeug": { "version": "==0.11.15", "hash": "sha256:c6f6f89124df051..." }, "flask": { "version": "==0.12", "hash": "sha256:7f03bb2c2554524..." }, "itsdangerous": { "version": "==0.24", "hash": "sha256:cbb3fcf8d3e33df..." }, "click": { "version": "==6.7", "hash": "sha256:29f99fc6125fbc9..." } }, "develop": { "packaging": { "version": "==16.8", "hash": "sha256:99276dc6e3a7851..." },
143. pypa/pipfile
144. Pipenv
145. Enables truly deterministic builds, while easily specifying what you want.
146. Automatically generates and checks file hashes for locked dependencies.
147. Automatically finds your project home, recursively, by looking for a Pipfile.
148. Automatically generates a Pipfile, if one doesn’t exist.
149. Automatically generates a Pipfile.lock, if one doesn’t exist.
150. Automatically creates a virtualenv in a standard location.
151. Automatically adds packages to a Pipfile when they are installed.
152. Automatically removes packages from a Pipfile when they are un-installed.
153. Also automatically updates pip.
154. Installing packages with Pipenv is easy.
155. $ pipenv install flask
156. $ pipenv install pytest --dev
157. $ pipenv install
158. $ pipenv shell
159. Current drawbacks?
160. $ pipenv install flask $ pipenv lock
161. $ pipenv install --lock flask
162. pipenv.org
163. Conclusion
164. Its our job to build the best software possible using the best tools for it.
165. We go out there and learn from other communities to make ours better.
166. Python is not only for those that've been using it for the past 10 years...
167. ...its also for those that will begin using it for the next 10.
168. There are smart people already working on how to improve this.
169. Awesome work already done.
170. Thank you to everyone helping make Python and it's ecosystem better.
171. From The Zen of Python
172. Beautiful is better than ugly.
173. Simple is better than complex.
174. There should be one and preferably only one obvious way to do it.
175. Now is better than never.
176. Thank you