the 1990ties lots of Assembler, C, C++, distributed programming came to Python around 2001 co-founded PyPy project in 2003, core dev for many years author of pytest, tox, execnet libraries file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 3 of 41 03/01/2013 12:04 PM
it. Python: There should be one -- and preferably only one -- obvious way to do it. file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 5 of 41 03/01/2013 12:04 PM
installs: downloads/builds/installs and tests packages and deps cpan index has hundreds of public mirrors uses declarative metadata since 2003 can verify cryptographic signing of packages file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 8 of 41 03/01/2013 12:04 PM
perl packages: supports/can enforce versioning semantics development releases are mirrored but not indexed policy for taking over maintenance of abandoned packages provides a mirrored un-indexed "home" area for authors/uploaders for patches, little docs file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 9 of 41 03/01/2013 12:04 PM
core Python two ways to download/install packages (pip, easy_install): pip installs only source distributions easy_install also binary distributions slow: dependency discovery requires client-side crawling for packages in all download and homepage sites no reliable way to test before install no cryptographic signature verification file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 10 of 41 03/01/2013 12:04 PM
on your laptop no enforced versioning semantics no policy for taking over maintenance of abandoned packages brittle protocol: network and server failures of any download/homepage server lead to aborted installs or installation of wrong versions metadata through code (setup.py) has tons of problems, has been hard so far to move away from. and as to mirrors ... file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 11 of 41 03/01/2013 12:04 PM
changes first before your tool/idea can be used (was also my motto when starting PyPy) file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 21 of 41 03/01/2013 12:04 PM
it work for most use cases enable new facilities/standards from there file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 22 of 41 03/01/2013 12:04 PM
runner unittest/2: standard library TestCases zope.testing: layered-setup unittest-runner twisted.trial: running twisted/async test cases many other methods, including "make test" setuptools has "setup.py test" Would inventing a new test runner help to standardize testing? file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 24 of 41 03/01/2013 12:04 PM
Python configure multiple dependency configs install packages in virtualenv environments run tests with your tools of choice act as a frontend to CI servers See http://tox.testrun.org (http://tox.testrun.org) for details. file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 25 of 41 03/01/2013 12:04 PM
purposes: it's a new compatible index and upload server subcommands for managing release and QA workflows file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 29 of 41 03/01/2013 12:04 PM
good existing pypi server to build these features: multiple indexes and staging (dev->QA->PROD) merging of upstream (pypi.python.org) indexes server-side crawling of download/homepages recording of automated test results with each package seemlessly allow offline usage (e. g. on a laptop) file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 30 of 41 03/01/2013 12:04 PM
has an upstream index whose packages are seemlessly merged each index is "crawl-clean" for clients uploading a package to an index will not push it to upstream compatible for pip, easy_install and setup.py upload enough theory - on to a a little demo :) file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 32 of 41 03/01/2013 12:04 PM
private overlay index simple crawling at server-side makes it easy and fast for installers devpi maintains a nginx-servable static view but: typing in index server urls, maintaining .pypirc config is burdensome with multiple indexes ... therefore ... file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 33 of 41 03/01/2013 12:04 PM
install download and install one or more packages upload build and upload packages from a checkout test download and test a package push push a package to another index file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 34 of 41 03/01/2013 12:04 PM
download and unpack a package from current index call tox for provisioning and performing tests stream test results from a test run (currently pytest only, nose/unittest to follow) file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 35 of 41 03/01/2013 12:04 PM
uploaded packages to another index currently specified by "PKGNAME-VER" in the future this could be done to constraints like "all tests on windows need to pass" file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 36 of 41 03/01/2013 12:04 PM
server: $ devpi server 1. use a user-specific index: $ devpi use http://localhost:3141/~hpk42/dev/ (http://localhost:3141/~hpk42 /dev/) 2. upload a package from a checkout: $ devpi upload 3. perform tests on multiple machines from multiple users: $ devpi test PKGNAME 4. test results at http://localhost:3141/~hpk42/dev/tests (http://localhost:3141/~hpk42 /dev/tests) 5. when release is ready: $ devpi push PKG https://pypi.python.org 6. file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 37 of 41 03/01/2013 12:04 PM
early adopters and streamline processes according to real-life usage don't aim for standard lib or PEPs (for now) file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 38 of 41 03/01/2013 12:04 PM
plugin system for upload/serve/test/install implement mirroring of index as static files support declarative metadata / Wheel format /... support server-side PGP signing and verification a public, redundant,global web service CI: per-commit automatic integration, testing and packaging file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 40 of 41 03/01/2013 12:04 PM
public release, to involve early adopters and contributors for finalizing processes and good defaults. Approach me here at Pycon Russia 2013 and thanks for listening! No public channels yet, but am going to announce them via: http://twitter.com/hpk42 (http://twitter.com/hpk42) http://holgerkrekel.net (http://holgerkrekel.net) file:///home/hpk/p/pycon-russia-keynote/html/index.html#... 41 of 41 03/01/2013 12:04 PM
hpk42
furukawayasuto
ivargrimstad
tsoshiro
nkforwork
qnighy
izumin5210
hollycummins
10tera
fukubaka0825
kakao
watanave
otakakot
philnash
sugarenia
maggiecrowley
samlambert
bluesmoon
pauljervisheath
csswizardry
jponch
shpigford
denniskardys
brad_frost
bkeepers
