Developer Ergonomics

Becd166a81dc51c0009f602d175d0cc8?s=47 José Padilla
February 18, 2017

Developer Ergonomics

Keynote for PyCaribbean

Becd166a81dc51c0009f602d175d0cc8?s=128

José Padilla

February 18, 2017
Tweet

Transcript

  1. Problem

  2. You'll eventually want to install third party packages...

  3. Enter easy_install

  4. Enter easy_install

  5. Enter pip

  6. $ pip install Django==1.8

  7. You just installed Django globally.

  8. $ pip install Django==1.10

  9. You just installed Django globally.

  10. Developer Ergonomics

  11. Package Managers and Environments

  12. José Padilla

  13. Work Training Open Source

  14. Work Training Open Source

  15. Work Training Open Source

  16. Backstory

  17. Lets begin...

  18. Node.js + NPM

  19. Node Package Manager

  20. Installing packages with npm is easy.

  21. $ npm install express

  22. └─┬ express@4.14.1 ├─┬ accepts@1.3.3 │ ├─┬ mime-types@2.1.14 │ │ └──

    mime-db@1.26.0 │ └── negotiator@0.6.1 ├── array-flatten@1.1.1 ├── content-disposition@0.5.2 ├── content-type@1.0.2 ├── cookie@0.3.1 ├── cookie-signature@1.0.6 ├─┬ debug@2.2.0 │ └── ms@0.7.1 ├── depd@1.1.0 ├── encodeurl@1.0.1 ├── escape-html@1.0.3 ├── etag@1.7.0 ├─┬ finalhandler@0.5.1 │ ├── statuses@1.3.1 │ └── unpipe@1.0.0 ├── fresh@0.3.0 ├── merge-descriptors@1.0.1 ├── methods@1.1.2 ├─┬ on-finished@2.3.0 │ └── ee-first@1.1.1 ├── parseurl@1.3.1 ├── path-to-regexp@0.1.7 ├─┬ proxy-addr@1.1.3 │ ├── forwarded@0.1.0 │ └── ipaddr.js@1.2.0 ├── qs@6.2.0 ├── range-parser@1.2.0 ├─┬ send@0.14.2 │ ├── destroy@1.0.4 │ ├─┬ http-errors@1.5.1 │ │ ├── inherits@2.0.3 │ │ └── setprototypeof@1.0.2 │ ├── mime@1.3.4 │ └── ms@0.7.2 ├── serve-static@1.11.2 ├─┬ type-is@1.6.14 │ └── media-typer@0.3.0 ├── utils-merge@1.0.0 └── vary@1.1.0 └─┬ express@4.14.1 ├─┬ accepts@1.3.3 │ ├─┬ mime-types@2.1.14 │ │ └── mime-db@1.26.0 │ └── negotiator@0.6.1 ├── array-flatten@1.1.1 ├── content-disposition@0.5.2 ├── content-type@1.0.2 ├── cookie@0.3.1 ├── cookie-signature@1.0.6 ├─┬ debug@2.2.0 │ └── ms@0.7.1 ├── depd@1.1.0 ├── encodeurl@1.0.1 ├── escape-html@1.0.3
  23. $ npm install express --save

  24. { "name": "pycaribbean", "version": "1.0.0", "dependencies": { "express": "^4.14.1" }

    }
  25. $ npm uninstall express --save

  26. { "name": "pycaribbean", "version": "1.0.0", "dependencies": {} }

  27. The default behavior of npm is to install packages locally

    to the current directory.
  28. global = ?

  29. global = CLI tools

  30. $ npm install eslint --global

  31. Dev / Prod

  32. $ npm install supertest --save-dev

  33. { "name": "nodejs", "version": "1.0.0", "dependencies": {}, "devDependencies": { "supertest":

    "^3.0.0" } }
  34. $ npm install

  35. $ npm install --production

  36. Dependency Conflicts

  37. $ npm install package-a --save

  38. └─┬ package-a@1.0.0 └── request@1.0.0

  39. $ npm install package-b --save

  40. └─┬ package-b@1.0.0 └── request@2.0.0

  41. ├─┬ package-a@1.0.0 │ └── request@1.0.0 └─┬ package-b@1.0.0 └── request@2.0.0

  42. Determinism

  43. None
  44. Determinism

  45. $ npm shrinkwrap

  46. { "name": "A", "version": "1.1.0", "dependencies": { "B": { "version":

    "1.0.1", "from": "B@^1.0.0", "resolved": "..." } } }
  47. Node.js + Yarn

  48. Yarn

  49. Fast

  50. Reliable

  51. Secure

  52. Installing packages with yarn is easy.

  53. $ yarn

  54. $ yarn add express

  55. $ yarn add --dev mocha

  56. $ yarn global add eslint

  57. Determinism

  58. # THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS

    FILE DIRECTLY. # yarn lockfile v1 package-1@^1.0.0: version "1.0.3" resolved "https://registry.npmjs.org/package-1/-/ package-1-1.0.3.tgz#c3ca7434938648c3e0d9c1e328dd68b622c2 84ca" package-2@^2.0.0: version "2.0.1" resolved "https://registry.npmjs.org/package-2/-/ package-2-2.0.1.tgz#9a5f699051b1e7073328f2a008968b64ea29 55d2" dependencies: package-4 "^4.0.0"
  59. Rust + Cargo

  60. Cargo

  61. Cross-pollination at its best.

  62. Cargo.toml

  63. [package] name = "package_a" version = "0.1.0" authors = ["root"]

    [dependencies] package_b = "0.0.1"
  64. Cargo.lock

  65. [root] name = "package_a" version = "0.0.1" dependencies = [

    "package_b 0.0.1 (...)", ] [[package]] name = "package_b" version = "0.0.1" source = "..." [metadata] "checksum package_b 0.0.1 (...)" = "7507624b29483431c0ba2d82aece"
  66. Python + PIP

  67. PIP

  68. Installing packages with pip is easy.

  69. $ pip install flask

  70. $ pip install flask $ pip freeze > requirements.txt

  71. click==6.7 Flask==0.12 itsdangerous==0.24 Jinja2==2.9.5 MarkupSafe==0.23 Werkzeug==0.11.15

  72. $ pip uninstall flask $ pip freeze > requirements.txt

  73. click==6.7 itsdangerous==0.24 Jinja2==2.9.5 MarkupSafe==0.23 Werkzeug==0.11.15

  74. Dev / Prod

  75. ├── requirements │ ├── common.txt │ ├── dev.txt │ └──

    prod.txt └── requirements.txt
  76. $ cat requirements.txt -r requirements/prod.txt

  77. $ cat requirements/prod.txt -r common.txt gunicorn==19.6.0 newrelic==2.74.0.54

  78. $ cat requirements/common.txt Django==1.10.5

  79. $ cat requirements/dev.txt pytest==3.0.6

  80. $ pip install -r requirements.txt

  81. $ pip install -r requirements/dev.txt

  82. The default behavior of pip is to install packages globally.

  83. Installing local dependencies

  84. Python 2

  85. virtualenv

  86. $ pip install virtualenv

  87. $ virtualenv .venv

  88. $ source .venv/bin/activate

  89. $ .venv/bin/python Python 2.7.9 (default, Dec 21 2016, 01:15:20) [GCC

    4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>
  90. (.venv) $ which python /path/to/.venv/bin/python

  91. (.venv) $ pip install flask

  92. (.venv) $ deactivate

  93. $ which python /usr/bin/python

  94. Python 3

  95. venv

  96. $ python3 -m venv .venv

  97. $ source .venv/bin/activate

  98. (.venv) $ pip install flask

  99. (.venv) $ deactivate

  100. Dependency Conflicts

  101. package-a Depends on requests==1.0.0

  102. package-b Depends on requests==2.12.4

  103. $ pip install package-a $ pip install package-b $ pip

    freeze
  104. package-a==0.1 package-b==0.1 requests==2.12.4

  105. Packages and dependencies are flat.

  106. $ pip check package-a has requirement requests==1.0.0, but you have

    requests 2.12.4.
  107. $ pip install flask && pip check

  108. function pip-install() { pip install $@ && pip check }

  109. $ pip-install flask

  110. Going forward

  111. Changing the defaults of tools like pip without creating chaos

    is hard.
  112. Installing packages in Python could be easier.

  113. "One of the hurdles that new Python developers have to

    get over is understanding the Python packaging ecosystem." – Jamie Matthews (@j4mie)
  114. "Variables in Python are local by default, so why aren't

    packages? You have to go out of your way to make a global variable. You should also have to go out of your way to install a package globally." – Trey Hunner (@treyhunner)
  115. My ideal scenario

  116. Default to installing packages locally.

  117. Improved dependency resolving.

  118. Fully specified and deterministic.

  119. Secure.

  120. $ pip init

  121. $ pip install flask

  122. $ pip install --dev pytest

  123. $ pip install

  124. How does it work?

  125. IDK

  126. "There's someone already working on a similar idea"

  127. Kenneth Reitz open sourced pipenv on January 19 2017

  128. Python + Pipenv

  129. More than 15 hours on Hacker News' front page.

  130. More than 2500 stars on GitHub.

  131. Dogfooding kennethreitz/requests

  132. Pipenv

  133. Sacred Marriage of Pipfile, Pip, & Virtualenv

  134. Cross-pollination at its best.

  135. Pipfile

  136. Requirements 2.0

  137. Pipfile will be superior to requirements.txt file in a number

    of ways...
  138. TOML syntax for declaring all types of Python dependencies.

  139. One Pipfile, as opposed to multiple requirements.txt files.

  140. [dev-packages] pytest = "*" [packages] flask = "*"

  141. Pipfile.lock

  142. { "default": { "MarkupSafe": { "version": "==0.23", "hash": "sha256:a4ec1aff59b95a1..." },

    "Jinja2": { "version": "==2.9.5", "hash": "sha256:a7b7438120dbe76..." }, "Werkzeug": { "version": "==0.11.15", "hash": "sha256:c6f6f89124df051..." }, "flask": { "version": "==0.12", "hash": "sha256:7f03bb2c2554524..." }, "itsdangerous": { "version": "==0.24", "hash": "sha256:cbb3fcf8d3e33df..." }, "click": { "version": "==6.7", "hash": "sha256:29f99fc6125fbc9..." } }, "develop": { "packaging": { "version": "==16.8", "hash": "sha256:99276dc6e3a7851..." }, "pytest": { "version": "==3.0.6", "hash": "sha256:da0ab50c7eec068..." }, "setuptools": { "version": "==34.1.1", "hash": "sha256:5f74aabe68c441b..." }, "pyparsing": { "version": "==2.1.10", "hash": "sha256:67101d7acee6929..." }, "py": { "version": "==1.4.32", "hash": "sha256:2d4bba2e25fff58..." }, "six": { "version": "==1.10.0", "hash": "sha256:0ff78c403d9bccf..." }, "appdirs": { "version": "==1.4.0", "hash": "sha256:85e58578db8f295..." } }, "_meta": { "sources": [ { "url": "https://pypi.python.org/simple", "verify_ssl": true } ], "requires": {}, "hash": { "sha256": "c755a9d5787ce2fdc70f7..." } } } { "default": { "MarkupSafe": { "version": "==0.23", "hash": "sha256:a4ec1aff59b95a1..." }, "Jinja2": { "version": "==2.9.5", "hash": "sha256:a7b7438120dbe76..." }, "Werkzeug": { "version": "==0.11.15", "hash": "sha256:c6f6f89124df051..." }, "flask": { "version": "==0.12", "hash": "sha256:7f03bb2c2554524..." }, "itsdangerous": { "version": "==0.24", "hash": "sha256:cbb3fcf8d3e33df..." }, "click": { "version": "==6.7", "hash": "sha256:29f99fc6125fbc9..." } }, "develop": { "packaging": { "version": "==16.8", "hash": "sha256:99276dc6e3a7851..." },
  143. pypa/pipfile

  144. Pipenv

  145. Enables truly deterministic builds, while easily specifying what you want.

  146. Automatically generates and checks file hashes for locked dependencies.

  147. Automatically finds your project home, recursively, by looking for a

    Pipfile.
  148. Automatically generates a Pipfile, if one doesn’t exist.

  149. Automatically generates a Pipfile.lock, if one doesn’t exist.

  150. Automatically creates a virtualenv in a standard location.

  151. Automatically adds packages to a Pipfile when they are installed.

  152. Automatically removes packages from a Pipfile when they are un-installed.

  153. Also automatically updates pip.

  154. Installing packages with Pipenv is easy.

  155. $ pipenv install flask

  156. $ pipenv install pytest --dev

  157. $ pipenv install

  158. $ pipenv shell

  159. Current drawbacks?

  160. $ pipenv install flask $ pipenv lock

  161. $ pipenv install --lock flask

  162. pipenv.org

  163. Conclusion

  164. Its our job to build the best software possible using

    the best tools for it.
  165. We go out there and learn from other communities to

    make ours better.
  166. Python is not only for those that've been using it

    for the past 10 years...
  167. ...its also for those that will begin using it for

    the next 10.
  168. There are smart people already working on how to improve

    this.
  169. Awesome work already done.

  170. Thank you to everyone helping make Python and it's ecosystem

    better.
  171. From The Zen of Python

  172. Beautiful is better than ugly.

  173. Simple is better than complex.

  174. There should be one and preferably only one obvious way

    to do it.
  175. Now is better than never.

  176. Thank you