Developer Ergonomics - Speaker Deck

Slide 1

Slide 1 text

Problem

Slide 2

Slide 2 text

You'll eventually want to install third party packages...

Slide 3

Slide 3 text

Enter easy_install

Slide 4

Slide 4 text

Enter easy_install

Slide 5

Slide 5 text

Enter pip

Slide 6

Slide 6 text

$ pip install Django==1.8

Slide 7

Slide 7 text

You just installed Django globally.

Slide 8

Slide 8 text

$ pip install Django==1.10

Slide 9

Slide 9 text

You just installed Django globally.

Slide 10

Slide 10 text

Developer Ergonomics

Slide 11

Slide 11 text

Package Managers and Environments

Slide 12

Slide 12 text

José Padilla

Slide 13

Slide 13 text

Work Training Open Source

Slide 14

Slide 14 text

Work Training Open Source

Slide 15

Slide 15 text

Work Training Open Source

Slide 16

Slide 16 text

Backstory

Slide 17

Slide 17 text

Lets begin...

Slide 18

Slide 18 text

Node.js + NPM

Slide 19

Slide 19 text

Node Package Manager

Slide 20

Slide 20 text

Installing packages with npm is easy.

Slide 21

Slide 21 text

$ npm install express

Slide 22

Slide 22 text

└─┬ express@4.14.1 ├─┬ accepts@1.3.3 │ ├─┬ mime-types@2.1.14 │ │ └── mime-db@1.26.0 │ └── negotiator@0.6.1 ├── array-flatten@1.1.1 ├── content-disposition@0.5.2 ├── content-type@1.0.2 ├── cookie@0.3.1 ├── cookie-signature@1.0.6 ├─┬ debug@2.2.0 │ └── ms@0.7.1 ├── depd@1.1.0 ├── encodeurl@1.0.1 ├── escape-html@1.0.3 ├── etag@1.7.0 ├─┬ finalhandler@0.5.1 │ ├── statuses@1.3.1 │ └── unpipe@1.0.0 ├── fresh@0.3.0 ├── merge-descriptors@1.0.1 ├── methods@1.1.2 ├─┬ on-finished@2.3.0 │ └── ee-first@1.1.1 ├── parseurl@1.3.1 ├── path-to-regexp@0.1.7 ├─┬ proxy-addr@1.1.3 │ ├── forwarded@0.1.0 │ └── ipaddr.js@1.2.0 ├── qs@6.2.0 ├── range-parser@1.2.0 ├─┬ send@0.14.2 │ ├── destroy@1.0.4 │ ├─┬ http-errors@1.5.1 │ │ ├── inherits@2.0.3 │ │ └── setprototypeof@1.0.2 │ ├── mime@1.3.4 │ └── ms@0.7.2 ├── serve-static@1.11.2 ├─┬ type-is@1.6.14 │ └── media-typer@0.3.0 ├── utils-merge@1.0.0 └── vary@1.1.0 └─┬ express@4.14.1 ├─┬ accepts@1.3.3 │ ├─┬ mime-types@2.1.14 │ │ └── mime-db@1.26.0 │ └── negotiator@0.6.1 ├── array-flatten@1.1.1 ├── content-disposition@0.5.2 ├── content-type@1.0.2 ├── cookie@0.3.1 ├── cookie-signature@1.0.6 ├─┬ debug@2.2.0 │ └── ms@0.7.1 ├── depd@1.1.0 ├── encodeurl@1.0.1 ├── escape-html@1.0.3

Slide 23

Slide 23 text

$ npm install express --save

Slide 24

Slide 24 text

{ "name": "pycaribbean", "version": "1.0.0", "dependencies": { "express": "^4.14.1" } }

Slide 25

Slide 25 text

$ npm uninstall express --save

Slide 26

Slide 26 text

{ "name": "pycaribbean", "version": "1.0.0", "dependencies": {} }

Slide 27

Slide 27 text

The default behavior of npm is to install packages locally to the current directory.

Slide 28

Slide 28 text

global = ?

Slide 29

Slide 29 text

global = CLI tools

Slide 30

Slide 30 text

$ npm install eslint --global

Slide 31

Slide 31 text

Dev / Prod

Slide 32

Slide 32 text

$ npm install supertest --save-dev

Slide 33

Slide 33 text

{ "name": "nodejs", "version": "1.0.0", "dependencies": {}, "devDependencies": { "supertest": "^3.0.0" } }

Slide 34

Slide 34 text

$ npm install

Slide 35

Slide 35 text

$ npm install --production

Slide 36

Slide 36 text

Dependency Conﬂicts

Slide 37

Slide 37 text

$ npm install package-a --save

Slide 38

Slide 38 text

└─┬ package-a@1.0.0 └── request@1.0.0

Slide 39

Slide 39 text

$ npm install package-b --save

Slide 40

Slide 40 text

└─┬ package-b@1.0.0 └── request@2.0.0

Slide 41

Slide 41 text

├─┬ package-a@1.0.0 │ └── request@1.0.0 └─┬ package-b@1.0.0 └── request@2.0.0

Slide 42

Slide 42 text

Determinism

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Determinism

Slide 45

Slide 45 text

$ npm shrinkwrap

Slide 46

Slide 46 text

{ "name": "A", "version": "1.1.0", "dependencies": { "B": { "version": "1.0.1", "from": "B@^1.0.0", "resolved": "..." } } }

Slide 47

Slide 47 text

Node.js + Yarn

Slide 48

Slide 48 text

Yarn

Slide 49

Slide 49 text

Fast

Slide 50

Slide 50 text

Reliable

Slide 51

Slide 51 text

Secure

Slide 52

Slide 52 text

Installing packages with yarn is easy.

Slide 53

Slide 53 text

$ yarn

Slide 54

Slide 54 text

$ yarn add express

Slide 55

Slide 55 text

$ yarn add --dev mocha

Slide 56

Slide 56 text

$ yarn global add eslint

Slide 57

Slide 57 text

Determinism

Slide 58

Slide 58 text

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY. # yarn lockfile v1 package-1@^1.0.0: version "1.0.3" resolved "https://registry.npmjs.org/package-1/-/ package-1-1.0.3.tgz#c3ca7434938648c3e0d9c1e328dd68b622c2 84ca" package-2@^2.0.0: version "2.0.1" resolved "https://registry.npmjs.org/package-2/-/ package-2-2.0.1.tgz#9a5f699051b1e7073328f2a008968b64ea29 55d2" dependencies: package-4 "^4.0.0"

Slide 59

Slide 59 text

Rust + Cargo

Slide 60

Slide 60 text

Cargo

Slide 61

Slide 61 text

Cross-pollination at its best.

Slide 62

Slide 62 text

Cargo.toml

Slide 63

Slide 63 text

[package] name = "package_a" version = "0.1.0" authors = ["root"] [dependencies] package_b = "0.0.1"

Slide 64

Slide 64 text

Cargo.lock

Slide 65

Slide 65 text

[root] name = "package_a" version = "0.0.1" dependencies = [ "package_b 0.0.1 (...)", ] [[package]] name = "package_b" version = "0.0.1" source = "..." [metadata] "checksum package_b 0.0.1 (...)" = "7507624b29483431c0ba2d82aece"

Slide 66

Slide 66 text

Python + PIP

Slide 67

Slide 67 text

PIP

Slide 68

Slide 68 text

Installing packages with pip is easy.

Slide 69

Slide 69 text

$ pip install ﬂask

Slide 70

Slide 70 text

$ pip install ﬂask $ pip freeze > requirements.txt

Slide 71

Slide 71 text

click==6.7 Flask==0.12 itsdangerous==0.24 Jinja2==2.9.5 MarkupSafe==0.23 Werkzeug==0.11.15

Slide 72

Slide 72 text

$ pip uninstall ﬂask $ pip freeze > requirements.txt

Slide 73

Slide 73 text

click==6.7 itsdangerous==0.24 Jinja2==2.9.5 MarkupSafe==0.23 Werkzeug==0.11.15

Slide 74

Slide 74 text

Dev / Prod

Slide 75

Slide 75 text

├── requirements │ ├── common.txt │ ├── dev.txt │ └── prod.txt └── requirements.txt

Slide 76

Slide 76 text

$ cat requirements.txt -r requirements/prod.txt

Slide 77

Slide 77 text

$ cat requirements/prod.txt -r common.txt gunicorn==19.6.0 newrelic==2.74.0.54

Slide 78

Slide 78 text

$ cat requirements/common.txt Django==1.10.5

Slide 79

Slide 79 text

$ cat requirements/dev.txt pytest==3.0.6

Slide 80

Slide 80 text

$ pip install -r requirements.txt

Slide 81

Slide 81 text

$ pip install -r requirements/dev.txt

Slide 82

Slide 82 text

The default behavior of pip is to install packages globally.

Slide 83

Slide 83 text

Installing local dependencies

Slide 84

Slide 84 text

Python 2

Slide 85

Slide 85 text

virtualenv

Slide 86

Slide 86 text

$ pip install virtualenv

Slide 87

Slide 87 text

$ virtualenv .venv

Slide 88

Slide 88 text

$ source .venv/bin/activate

Slide 89

Slide 89 text

$ .venv/bin/python Python 2.7.9 (default, Dec 21 2016, 01:15:20) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>

Slide 90

Slide 90 text

(.venv) $ which python /path/to/.venv/bin/python

Slide 91

Slide 91 text

(.venv) $ pip install ﬂask

Slide 92

Slide 92 text

(.venv) $ deactivate

Slide 93

Slide 93 text

$ which python /usr/bin/python

Slide 94

Slide 94 text

Python 3

Slide 95

Slide 95 text

venv

Slide 96

Slide 96 text

$ python3 -m venv .venv

Slide 97

Slide 97 text

$ source .venv/bin/activate

Slide 98

Slide 98 text

(.venv) $ pip install ﬂask

Slide 99

Slide 99 text

(.venv) $ deactivate

Slide 100

Slide 100 text

Dependency Conﬂicts

Slide 101

Slide 101 text

package-a Depends on requests==1.0.0

Slide 102

Slide 102 text

package-b Depends on requests==2.12.4

Slide 103

Slide 103 text

$ pip install package-a $ pip install package-b $ pip freeze

Slide 104

Slide 104 text

package-a==0.1 package-b==0.1 requests==2.12.4

Slide 105

Slide 105 text

Packages and dependencies are ﬂat.

Slide 106

Slide 106 text

$ pip check package-a has requirement requests==1.0.0, but you have requests 2.12.4.

Slide 107

Slide 107 text

$ pip install ﬂask && pip check

Slide 108

Slide 108 text

function pip-install() { pip install $@ && pip check }

Slide 109

Slide 109 text

$ pip-install ﬂask

Slide 110

Slide 110 text

Going forward

Slide 111

Slide 111 text

Changing the defaults of tools like pip without creating chaos is hard.

Slide 112

Slide 112 text

Installing packages in Python could be easier.

Slide 113

Slide 113 text

"One of the hurdles that new Python developers have to get over is understanding the Python packaging ecosystem." – Jamie Matthews (@j4mie)

Slide 114

Slide 114 text

"Variables in Python are local by default, so why aren't packages? You have to go out of your way to make a global variable. You should also have to go out of your way to install a package globally." – Trey Hunner (@treyhunner)

Slide 115

Slide 115 text

My ideal scenario

Slide 116

Slide 116 text

Default to installing packages locally.

Slide 117

Slide 117 text

Improved dependency resolving.

Slide 118

Slide 118 text

Fully speciﬁed and deterministic.

Slide 119

Slide 119 text

Secure.

Slide 120

Slide 120 text

$ pip init

Slide 121

Slide 121 text

$ pip install ﬂask

Slide 122

Slide 122 text

$ pip install --dev pytest

Slide 123

Slide 123 text

$ pip install

Slide 124

Slide 124 text

How does it work?

Slide 125

Slide 125 text

IDK

Slide 126

Slide 126 text

"There's someone already working on a similar idea"

Slide 127

Slide 127 text

Kenneth Reitz open sourced pipenv on January 19 2017

Slide 128

Slide 128 text

Python + Pipenv

Slide 129

Slide 129 text

More than 15 hours on Hacker News' front page.

Slide 130

Slide 130 text

More than 2500 stars on GitHub.

Slide 131

Slide 131 text

Dogfooding kennethreitz/requests

Slide 132

Slide 132 text

Pipenv

Slide 133

Slide 133 text

Sacred Marriage of Pipﬁle, Pip, & Virtualenv

Slide 134

Slide 134 text

Cross-pollination at its best.

Slide 135

Slide 135 text

Pipﬁle

Slide 136

Slide 136 text

Requirements 2.0

Slide 137

Slide 137 text

Pipﬁle will be superior to requirements.txt ﬁle in a number of ways...

Slide 138

Slide 138 text

TOML syntax for declaring all types of Python dependencies.

Slide 139

Slide 139 text

One Pipﬁle, as opposed to multiple requirements.txt ﬁles.

Slide 140

Slide 140 text

[dev-packages] pytest = "*" [packages] flask = "*"

Slide 141

Slide 141 text

Pipﬁle.lock

Slide 142

Slide 142 text

{ "default": { "MarkupSafe": { "version": "==0.23", "hash": "sha256:a4ec1aff59b95a1..." }, "Jinja2": { "version": "==2.9.5", "hash": "sha256:a7b7438120dbe76..." }, "Werkzeug": { "version": "==0.11.15", "hash": "sha256:c6f6f89124df051..." }, "flask": { "version": "==0.12", "hash": "sha256:7f03bb2c2554524..." }, "itsdangerous": { "version": "==0.24", "hash": "sha256:cbb3fcf8d3e33df..." }, "click": { "version": "==6.7", "hash": "sha256:29f99fc6125fbc9..." } }, "develop": { "packaging": { "version": "==16.8", "hash": "sha256:99276dc6e3a7851..." }, "pytest": { "version": "==3.0.6", "hash": "sha256:da0ab50c7eec068..." }, "setuptools": { "version": "==34.1.1", "hash": "sha256:5f74aabe68c441b..." }, "pyparsing": { "version": "==2.1.10", "hash": "sha256:67101d7acee6929..." }, "py": { "version": "==1.4.32", "hash": "sha256:2d4bba2e25fff58..." }, "six": { "version": "==1.10.0", "hash": "sha256:0ff78c403d9bccf..." }, "appdirs": { "version": "==1.4.0", "hash": "sha256:85e58578db8f295..." } }, "_meta": { "sources": [ { "url": "https://pypi.python.org/simple", "verify_ssl": true } ], "requires": {}, "hash": { "sha256": "c755a9d5787ce2fdc70f7..." } } } { "default": { "MarkupSafe": { "version": "==0.23", "hash": "sha256:a4ec1aff59b95a1..." }, "Jinja2": { "version": "==2.9.5", "hash": "sha256:a7b7438120dbe76..." }, "Werkzeug": { "version": "==0.11.15", "hash": "sha256:c6f6f89124df051..." }, "flask": { "version": "==0.12", "hash": "sha256:7f03bb2c2554524..." }, "itsdangerous": { "version": "==0.24", "hash": "sha256:cbb3fcf8d3e33df..." }, "click": { "version": "==6.7", "hash": "sha256:29f99fc6125fbc9..." } }, "develop": { "packaging": { "version": "==16.8", "hash": "sha256:99276dc6e3a7851..." },

Slide 143

Slide 143 text

pypa/pipﬁle

Slide 144

Slide 144 text

Pipenv

Slide 145

Slide 145 text

Enables truly deterministic builds, while easily specifying what you want.

Slide 146

Slide 146 text

Automatically generates and checks ﬁle hashes for locked dependencies.

Slide 147

Slide 147 text

Automatically ﬁnds your project home, recursively, by looking for a Pipﬁle.

Slide 148

Slide 148 text

Automatically generates a Pipﬁle, if one doesn’t exist.

Slide 149

Slide 149 text

Automatically generates a Pipﬁle.lock, if one doesn’t exist.

Slide 150

Slide 150 text

Automatically creates a virtualenv in a standard location.

Slide 151

Slide 151 text

Automatically adds packages to a Pipﬁle when they are installed.

Slide 152

Slide 152 text

Automatically removes packages from a Pipﬁle when they are un-installed.

Slide 153

Slide 153 text

Also automatically updates pip.

Slide 154

Slide 154 text

Installing packages with Pipenv is easy.

Slide 155

Slide 155 text

$ pipenv install ﬂask

Slide 156

Slide 156 text

$ pipenv install pytest --dev

Slide 157

Slide 157 text

$ pipenv install

Slide 158

Slide 158 text

$ pipenv shell

Slide 159

Slide 159 text

Current drawbacks?

Slide 160

Slide 160 text

$ pipenv install ﬂask $ pipenv lock

Slide 161

Slide 161 text

$ pipenv install --lock ﬂask

Slide 162

Slide 162 text

pipenv.org

Slide 163

Slide 163 text

Conclusion

Slide 164

Slide 164 text

Its our job to build the best software possible using the best tools for it.

Slide 165

Slide 165 text

We go out there and learn from other communities to make ours better.

Slide 166

Slide 166 text

Python is not only for those that've been using it for the past 10 years...

Slide 167

Slide 167 text

...its also for those that will begin using it for the next 10.

Slide 168

Slide 168 text

There are smart people already working on how to improve this.

Slide 169

Slide 169 text

Awesome work already done.

Slide 170

Slide 170 text

Thank you to everyone helping make Python and it's ecosystem better.

Slide 171

Slide 171 text

From The Zen of Python

Slide 172

Slide 172 text

Beautiful is better than ugly.

Slide 173

Slide 173 text

Simple is better than complex.

Slide 174

Slide 174 text

There should be one and preferably only one obvious way to do it.

Slide 175

Slide 175 text

Now is better than never.

Slide 176

Slide 176 text

Thank you