PyCon 2013 Tutorial: Script to PyPI to GitHub & More
How do you start a new project? How do you deliver a script to co-workers? How do you develop it with best practices? How do you use virtualenv and pip? How do you package it? How do you automate testing, building, uploading to PyPI?
Python • Worked in HA, Search, Open Source, BI Worked in HA, Search, Open Source, BI and Storage and Storage • Author of multiple Python Books Author of multiple Python Books
basic Python knowledge • Hands on Hands on – (short) lecture (short) lecture – (short) code (short) code – repeat until time is gone repeat until time is gone
distribution mechanism: • Single file Single file • Zip file Zip file – Python entry point Python entry point – Splat/run Splat/run • System package System package • PyPi/Distribute/pip package PyPi/Distribute/pip package
root • At mercy of packager (maybe worse than At mercy of packager (maybe worse than PyPi) PyPi) • Reuse Reuse • Limited to single version Limited to single version • python -m modulename python -m modulename
approachable Plain text config is easily approachable • Careful with Python config on process Careful with Python config on process run by root run by root
Iterable Iterable. . • Can open/iterate many times Can open/iterate many times • Implementation depends on file Implementation depends on file • Need to manage closing file Need to manage closing file
def def parse parse(fin): (fin): ... ... for for line line in in upper(fin): upper(fin): ... ... sys sys. .stdout stdout. .write(line) write(line) >>> >>> def def upper upper(iterable): (iterable): ... ... for for item item in in iterable: iterable: ... ... yield yield str str(item) (item). .upper() upper()
Python • Runs as user Runs as user • Specify version Specify version • Sandboxed from system Sandboxed from system • Create multiple sandboxes Create multiple sandboxes
$ which python # not /usr/bin/python /home/matt/work/courses/script-pypi-github/env/b /home/matt/work/courses/script-pypi-github/env/b in/python in/python
django) • *~ *~ (emacs) (emacs) • Run Run git status git status and add outliers to and add outliers to .gitignore .gitignore • Make settings global: Make settings global: git config --global core.excludesfile git config --global core.excludesfile Python.gitignore Python.gitignore git config --global core.excludesfile git config --global core.excludesfile Python.gitignore Python.gitignore
Some include dunder meta in project docstring (requests requests __init__.py __init__.py): ): :copyright: (c) 2013 by Kenneth Reitz. :copyright: (c) 2013 by Kenneth Reitz. :license: Apache 2.0, see LICENSE for more :license: Apache 2.0, see LICENSE for more details. details. (note IANAL) (note IANAL)
that makes it easy to create Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, intelligent and beautiful documentation, written by Georg Brandl and licensed written by Georg Brandl and licensed under the BSD license. under the BSD license. http://sphinx-doc.org http://sphinx-doc.org
by default): by default): $ make file $ make file # will build dependentfile if # will build dependentfile if necessary necessary # then build file # then build file
file, need to use isn't a file, need to use .PHONY .PHONY to indicate that to to indicate that to make make. (If you had a file . (If you had a file named named clean clean it wouldn't try to build it). it wouldn't try to build it).
(Recursively Expanded) Variables (expanded when used): FILE = foo FILE = foo DATA = $(FILE) DATA = $(FILE) # If DATA expanded would be foo # If DATA expanded would be foo FILE = bar FILE = bar # If DATA expanded would be bar # If DATA expanded would be bar
make pwd pushd /etc pushd /etc make: pushd: Command not found make: pushd: Command not found make: *** [pwd] Error 127 make: *** [pwd] Error 127 ( (pushd pushd is a bash function) is a bash function)
its Each tab indented command runs in its own process. Use own process. Use ; ; and put in one line or and put in one line or use use \ \ for line continuation for line continuation
Testing with nose nose: : .PHONY: test .PHONY: test test: nose deps test: nose deps <TAB>$(NOSE) <TAB>$(NOSE) # nose depends on the nosetests binary # nose depends on the nosetests binary nose: $(NOSE) nose: $(NOSE) $(NOSE): env $(NOSE): env <TAB>$(PIP) install nose <TAB>$(PIP) install nose
Dynamic languages don't need anything like like make make, unless they have some , unless they have some compile-time interface dependencies compile-time interface dependencies between modules” between modules” http://stackoverflow.com/questions/758093 http://stackoverflow.com/questions/758093 9/why-are-there-no-makefiles-for-automati 9/why-are-there-no-makefiles-for-automati on-in-python-projects on-in-python-projects
distutils distutils doesn't doesn't download reqs only download reqs only documents documents them. Use them. Use requirements.txt requirements.txt in combo with in combo with pip pip. .
add files to MANIFEST.in MANIFEST.in (include in (include in package) package) • add files to add files to package_data package_data in in setup.py setup.py (include in install) Not recursive (include in install) Not recursive
pat1 pat2 ... • recursive-(include|exclude) dir pat1 pat2 ... recursive-(include|exclude) dir pat1 pat2 ... • global-(include|exclude) dir pat1 pat2 ... global-(include|exclude) dir pat1 pat2 ... • prune dir prune dir • graft dir graft dir http://docs.python.org/release/1.6/dist/sdist-cmd.html#sdist-c http://docs.python.org/release/1.6/dist/sdist-cmd.html#sdist-c md md
register upload $ python setup.py sdist register upload ... ... Creating tar archive Creating tar archive removing 'rst2odp-0.2.4' (and everything under it) removing 'rst2odp-0.2.4' (and everything under it) running register running register running check running check We need to know who you are, so please choose either: We need to know who you are, so please choose either: 1. use your existing login, 1. use your existing login, 2. register as a new user, 2. register as a new user, 3. have the server generate a new password for you (and email it to you), 3. have the server generate a new password for you (and email it to you), or or 4. quit 4. quit Your selection [default 1]: Your selection [default 1]: 1 1
Password: Password: Registering rst2odp to http://pypi.python.org/pypi Registering rst2odp to http://pypi.python.org/pypi Server response (200): OK Server response (200): OK I can store your PyPI login so future submissions will be faster. I can store your PyPI login so future submissions will be faster. (the login will be stored in /home/matt/.pypirc) (the login will be stored in /home/matt/.pypirc) Save your login (y/N)?y Save your login (y/N)?y running upload running upload Submitting dist/rst2odp-0.2.4.tar.gz to http://pypi.python.org/pypi Submitting dist/rst2odp-0.2.4.tar.gz to http://pypi.python.org/pypi Server response (200): OK Server response (200): OK
env/bin/nosetests NOSE := env/bin/nosetests # --------- Testing ---------- # --------- Testing ---------- .PHONY: test .PHONY: test test: nose deps test: nose deps <TAB>$(NOSE) <TAB>$(NOSE) # nose depends on the nosetests binary # nose depends on the nosetests binary nose: $(NOSE) nose: $(NOSE) $(NOSE): env $(NOSE): env <TAB>$(PIP) install nose <TAB>$(PIP) install nose
github • Sync repos on Profile page Sync repos on Profile page • Enable repo Enable repo • Create a Create a .travis.yml .travis.yml file on github file on github • Push a commit on github Push a commit on github