Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Python Packaging Simplified by Asheesh Laroia
Search
PyCon 2014
April 12, 2014
Technology
3
890
Python Packaging Simplified by Asheesh Laroia
Presented at PyCon 2014
PyCon 2014
April 12, 2014
Tweet
Share
More Decks by PyCon 2014
See All by PyCon 2014
Postgres Performance for Humans by Craig Kerstiens
pycon2014
29
3.6k
Technical Onboarding, Training, and Mentoring by Kate Heddleston and Nicole Zuckerman
pycon2014
1
2.3k
"My big gay adventure. Making, releasing and selling an indie game made in python." by Luke Miller
pycon2014
2
1.6k
Farewell and Welcome Home, Python in Two Genders by Naomi_Ceder
pycon2014
1
720
Deliver Your Software in an Envelope by Augie Fackler and Nathaniel Manista
pycon2014
1
530
Hitchhikers Guide to Free and Open Source Participation by Elena Williams
pycon2014
6
1.2k
Localization Revisted (aka. Translations Evolved) by Ruchi Varshney
pycon2014
0
690
Smart Dumpster by Bradley E. Angell
pycon2014
0
520
Software Engineering for Hackers: Bridging the Two Solitudes by Tavish Armstrong
pycon2014
0
720
Other Decks in Technology
See All in Technology
AWSを活用したIoTにおけるセキュリティ対策のご紹介
kwskyk
0
350
設計を積み重ねてシステムを刷新する
sansantech
PRO
0
160
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
150
スキルだけでは満たせない、 “組織全体に”なじむオンボーディング/Onboarding that fits “throughout the organization” and cannot be satisfied by skills alone
bitkey
0
180
EDRの検知の仕組みと検知回避について
chayakonanaika
12
4.9k
EMConf JP 2025 懇親会LT / EMConf JP 2025 social gathering
sugamasao
2
190
あなたが人生で成功するための5つの普遍的法則 #jawsug #jawsdays2025 / 20250301 HEROZ
yoshidashingo
2
290
OSS構成管理ツールCMDBuildを使ったAWSリソース管理の自動化
satorufunai
0
640
AIエージェント時代のエンジニアになろう #jawsug #jawsdays2025 / 20250301 Agentic AI Engineering
yoshidashingo
8
3.7k
急成長する企業で作った、エンジニアが輝ける制度/ 20250227 Rinto Ikenoue
shift_evolve
0
130
Pwned Labsのすゝめ
ken5scal
2
440
【Findy】「正しく」失敗できる チームの作り方 〜リアルな事例から紐解く失敗を恐れない組織とは〜 / A team that can fail correctly by findy
i35_267
5
890
Featured
See All Featured
Side Projects
sachag
452
42k
RailsConf 2023
tenderlove
29
1k
Reflections from 52 weeks, 52 projects
jeffersonlam
348
20k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
32
2.1k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
Music & Morning Musume
bryan
46
6.4k
Designing for Performance
lara
604
68k
For a Future-Friendly Web
brad_frost
176
9.6k
Designing Experiences People Love
moore
140
23k
A Philosophy of Restraint
colly
203
16k
Build your cross-platform service in a week with App Engine
jlugia
229
18k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Transcript
Python packaging simplified for end users, app developers, and open
source contributors Asheesh Laroia
Changes No Debian vs. ??? No compiled code talking, except
wheel No focus on Django apps
Goals 1. Be able to install 2. Be able to
distribute 3. Know how to learn more
Goals 1. Be able to install 2. Be able to
distribute 3. Know how to learn more
Goals 1. Be able to install 2. Be able to
distribute 3. Know how to learn more
Installation
$ python
$ python >>> import numbers
$ python >>> import numbers >>> numbers <module 'numbers' from
'/usr/lib/python2.7/numbers.pyc' >
>>> import sys >>> sys.path
>>> import sys >>> sys.path ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux- gnu', '/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/me/.local/lib/python2.7/site- packages', '/usr/local/lib/python2.7/dist- packages', '/usr/lib/python2.7/dist-packages',
>>> import sys >>> sys.path ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux- gnu', '/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/me/.local/lib/python2.7/site- packages', '/usr/local/lib/python2.7/dist- packages', '/usr/lib/python2.7/dist-packages',
$ python >>> import numbers >>> numbers <module 'numbers' from
'/usr/lib/python2.7/numbers.pyc' >
None
>>> import etsy >>> import etsy
>>> import etsy >>> import etsy cheeseshop cheeseshop pypi.python.org pypi.python.org
>>> import etsy >>> import etsy cheeseshop cheeseshop pypi.python.org pypi.python.org
[etsy wrapper] [etsy wrapper]
None
None
>>> import python-etsy File "<stdin>", line 1 import python-etsy ^
SyntaxError: invalid syntax
None
None
None
$ sudp pip install python-etsy $ sudp pip install python-etsy
$ sudp pip install python-etsy $ sudp pip install python-etsy
Genki Sudo Genki Sudo World Order World Order
>>> import sys >>> sys.path ['', # ... '/home/me/.local/lib/python2.7/ site-packages',
]
$ pip install --user python-etsy
$ pip install --user python-etsy Downloading/unpacking python-etsy Downloading python-etsy- 0.1.2.tar.gz
$ pip install --user python-etsy Downloading/unpacking python-etsy Downloading python-etsy- 0.1.2.tar.gz
Successfully installed python-etsy Cleaning up...
$ pip install --user python-etsy Downloading/unpacking python-etsy Downloading python-etsy- 0.1.2.tar.gz
Successfully installed python-etsy Cleaning up... $ python >>> import etsy
$ pip install --user python-etsy Downloading/unpacking python-etsy Downloading python-etsy- 0.1.2.tar.gz
Successfully installed python-etsy Cleaning up... $ python >>> import etsy >>> etsy <module 'etsy' from '/home/me/.local/lib/python2.7/ site-packages/etsy/__init__.pyc'>
None
# module # module >>> import etsy >>> import etsy
# module # module >>> import etsy >>> import etsy
# distribution # distribution $ pip install --user python-etsy $ pip install --user python-etsy
# module # module >>> import etsy >>> import etsy
# distribution # distribution $ pip install --user python-etsy $ pip install --user python-etsy # tools # tools $ pip install --user $ pip install --user
# module # module >>> import etsy >>> import etsy
# distribution # distribution $ pip install --user python-etsy $ pip install --user python-etsy # tools # tools $ pip install --user $ pip install --user # no sudo, no virtualenv # no sudo, no virtualenv
None
$ pip install --user etsy $ pip install --user etsy
$ pip install --user etsy $ pip install --user etsy
ValueError: Invalid IPv6 URL ValueError: Invalid IPv6 URL
PyPI: PyPI: Wrong or missing Wrong or missing download URLs
download URLs
PyPI: PyPI: Wrong or missing no Wrong or missing no
download URLs verification download URLs verification
PyPI: PyPI: Wrong or missing no Wrong or missing no
download URLs verification download URLs verification (fixed w/ pip 1.5) (remains) (fixed w/ pip 1.5) (remains)
pip: pip: under python 3.2: under python 3.2: $ pip
install --user jinja2 $ pip install --user jinja2 Downloading/unpacking jinja2 Downloading/unpacking jinja2
pip: pip: under python 3.2: under python 3.2: $ pip
install --user jinja2 $ pip install --user jinja2 Downloading/unpacking jinja2 Downloading/unpacking jinja2 # ... # ...
File "jinja2/runtime.py", File "jinja2/runtime.py", line 44 line 44 SyntaxError: invalid
syntax SyntaxError: invalid syntax
File "jinja2/runtime.py", File "jinja2/runtime.py", line 44 line 44 SyntaxError: invalid
syntax SyntaxError: invalid syntax Successfully installed jinja2 Successfully installed jinja2 Cleaning up... Cleaning up...
>>> import jinja2 >>> import jinja2
>>> import jinja2 >>> import jinja2 File File "…/jinja2/environment.py", line
"…/jinja2/environment.py", line 639 639 u'...'.encode('iso-8859-15') u'...'.encode('iso-8859-15') ^ ^ SyntaxError: invalid syntax SyntaxError: invalid syntax
Uninstallation
$ python >>> import etsy
$ python >>> import etsy $ pip uninstall etsy Cannot
uninstall requirement etsy, not installed
$ python >>> import etsy $ pip uninstall etsy Cannot
uninstall requirement etsy, not installed $ pip uninstall python-etsy
.../site-packages/ .../site-packages/ python_etsy*.egg-info/ python_etsy*.egg-info/ PKG-DATA PKG-DATA
.../site-packages/ .../site-packages/ python_etsy*.{dist,egg}-info/ python_etsy*.{dist,egg}-info/ {METADATA,PKG-INFO} {METADATA,PKG-INFO}
.../site-packages/ .../site-packages/ python_etsy*.{dist,egg}-info/ python_etsy*.{dist,egg}-info/ {METADATA,PKG-INFO} {METADATA,PKG-INFO} Name: python-etsy Name: python-etsy
Version: 0.1.2 Version: 0.1.2 Summary: Python wrapper for the Summary: Python wrapper for the Etsy api Etsy api
$ pip install --user python-etsy $ pip install --user python-etsy
# ... # ... Running setup.py egg_info Running setup.py egg_info for package python-etsy for package python-etsy
$ pip install --user python-etsy $ pip install --user python-etsy
# ... # ... Running Running setup.py egg_info setup.py egg_info for package python-etsy for package python-etsy
Installation of pip
$ pip pip: Command not found.
$ pip pip: Command not found. $ # download get-pip.py
$ pip pip: Command not found. $ # download get-pip.py
$ python get-pip.py --user
$ pip pip: Command not found. $ # download get-pip.py
$ python get-pip.py --user $ pip pip: Command not found.
$ pip pip: Command not found. $ # download get-pip.py
$ python get-pip.py --user $ pip pip: Command not found. $ PATH=~/.local/bin:$PATH
$ pip pip: Command not found. $ # download get-pip.py
$ python get-pip.py --user $ pip pip: Command not found. $ PATH=~/.local/bin:$PATH $ pip --version pip 1.5.4 from /home/me/.local/lib/python2.7/ site-packages (python 2.7)
Distributing your code
~ $ cd get-octopi
~ $ cd get-octopi ~/get-octopi $ ls octopi_getter.py
~ $ cd get-octopi ~/get-octopi $ ls octopi_getter.py # time
passes
~ $ cd get-octopi ~/get-octopi $ ls octopi_getter.py # time
passes ~/get-octopi $ ls octopi_getter.py setup.py LICENSE
$ python setup.py register running register running check Registering wikimarkup
to https://pypi.python.org/pypi
$ python setup.py register running register running check Registering wikimarkup
to https://pypi.python.org/pypi
$ python setup.py register running register running check Registering get-octopi
to https://pypi.python.org/pypi
$ python setup.py sdist upload python setup.py sdist upload running
sdist running check creating wikimarkup-1.0 creating dist Creating tar archive running upload
$ python setup.py sdist upload python setup.py sdist upload running
sdist running check creating wikimarkup-1.0 creating dist Creating tar archive running upload
$ python setup.py sdist upload python setup.py sdist upload running
sdist running check creating get-octopi-1.0 creating dist Creating tar archive running upload
$ python setup.py sdist upload python setup.py sdist upload running
sdist running check creating get-octopi-1.0 creating dist Creating tar archive running upload Submitting dist/get-octopi- 1.0.tar.gz to https://pypi.python.org/pypi
from distutils.core import setup setup( name='get-octopi', version='1.0', packages=['octopi_getter'], )
from distutils.core import setup setup( name='get-octopi', version='1.0', py_modules=['octopi_getter'], )
from distutils.core import setup if __name__ == '__main__': setup( name='get-octopi',
version='1.0', py_modules=['octopi_getter'], )
from distutils.core import setup setup( name='get-octopi', version='1.0', py_modules=['octopi_getter'], )
from distutils.core import setup setup( name='get-octopi', version='1.0', py_modules=['octopi_getter'], author='Asheesh Laroia',
email='
[email protected]
', )
>>> import octopi_getter >>> octopi_getter.main <function main at 0x7f2d64849500>
from distutils.core import setup # within setup() entry_points = {
'console_scripts': [ 'get-now = octopi_getter:main', ], },
from distutils.core import setup # within setup() entry_points = {
'console_scripts': [ 'get-now = octopi_getter:main', ], }, # finally $ pip install --user .
$ get-now $ get-now
$ get-now $ get-now get-now: Command not found. get-now: Command
not found.
$ get-now $ get-now get-now: Command not found. get-now: Command
not found. $ PATH=$PATH:~/.local/bin $ PATH=$PATH:~/.local/bin
$ get-now $ get-now
$ get-now $ get-now Logging into Etsy... Logging into Etsy...
$ get-now $ get-now Logging into Etsy... Logging into Etsy...
Identifying octopi... Identifying octopi...
$ get-now $ get-now Logging into Etsy... Logging into Etsy...
Identifying octopi... Identifying octopi... Purchasing... Purchasing...
$ get-now $ get-now Logging into Etsy... Logging into Etsy...
Identifying octopi... Identifying octopi... Purchasing... Purchasing... Purchased. You spent $325.21. Purchased. You spent $325.21.
$ get-now $ get-now
$ get-now $ get-now ImportError: No module named ImportError: No
module named etsy etsy
from distutils.core import setup setup( name='get-octopi', version='1.0', py_modules=['octopi_getter'], )
from distutils.core import setup setup( name='get-octopi', version='1.0', py_modules=['octopi_getter'], install_requires=[ 'python-etsy',
], )
zc.buildout vs requirements.txt vs setup( install_requires=[ 'python-etsy', ], )
Developing your code
$ pip install --user .
$ pip install --user . $ python >>> import octopi_getter
>>> octopi_getter <module ... from '.../site- packages/octopi_getter.pyc'>
$ pip install --user --editable . # aka setup.py develop
--user $ python >>> import octopi_getter >>> octopi_getter <module ... from '/home/me/ get-octopi/octopi_getter.pyc'>
$ pip install --user --editable . # aka setup.py develop
--user $ python >>> import octopi_getter >>> octopi_getter <module ... from '/home/me/ get-octopi/octopi_getter.pyc'>
from distutils.core import setup from setuptools import setup from distribute
import setup
from distutils.core import setup from setuptools import setup from distribute
import setup
from distutils.core import setup ## safe and featureless from setuptools
import setup ## pip provides this for free from distribute import setup
from distutils.core import setup setup( name='get-octopi', version='1.0', py_modules=['octopi_getter'], install_requires=[ 'python-etsy>0.1.1',
], )
$ pip install --user --editable . # ... CONFLICT
$ pip install --user --editable . # ... CONFLICT $
virtualenv . New python executable in ./bin/python Installing setuptools, pip...done. $ source bin/activate (get-octopi) $
$ pip install --user --editable . # ... CONFLICT $
virtualenv . New python executable in ./bin/python Installing setuptools, pip...done. $ source bin/activate (get-octopi) $ pip install --editable .
$ pip install --user --editable . # ... CONFLICT $
virtualenv . New python executable in ./bin/python Installing setuptools, pip...done. $ source bin/activate (get-octopi) $ pip install -e .
(get-octopi) $ pip install (get-octopi) $ pip install --user --editable
. --user --editable .
(get-octopi) $ pip install (get-octopi) $ pip install --user --editable
. --user --editable . Can not perform a '--user' Can not perform a '--user' install. User site-packages are install. User site-packages are not visible in this virtualenv. not visible in this virtualenv.
(get-octopi) $ (get-octopi) $
(get-octopi) $ (get-octopi) $ (new terminal window) (new terminal window)
$ $
(get-octopi) $ pip install lxml unable to execute 'x86_64-linux-gnu- gcc':
No such file or directory error: command 'x86_64-linux-gnu- gcc' failed with exit status 1
$ sudo apt-get install python-lxml $ python >>> import lxml
>>> $ pip install lxml Requirement already satisfied (use --upgrade to upgrade): lxml in /usr/lib/python2.7/dist-packages Cleaning up...
$ sudo apt-get install python-lxml $ python >>> import lxml
>>> $ source bin/activate (get-octopi) $ python >>> import lxml ImportError: No module called lxml.
$ sudo apt-get install python-lxml $ python >>> import lxml
>>> $ source bin/activate (get-octopi) $ python >>> import lxml ImportError: No module called lxml $ sudo apt-get build-dep python-lxml
None
Vagrant: 150MB VirtualBox: 102MB precise32.box: 300MB
Vagrant: 150MB VirtualBox: 102MB precise32.box: 300MB 1.5 hours @ 100KB/s
Vagrant: 150MB VirtualBox: 102MB precise32.box: 300MB 1.5 hours @ 100KB/s
$ vagrant provision
new engineers
new engineers new contributors
new engineers new contributors save your self
openhatch/ oh-mainline django in 5 min
- no virtualenv
- no virtualenv - no pip install
- no virtualenv - no pip install - no C
compiler
- no virtualenv - no pip install - no C
compiler - purely sqlite
# No compiler, no problem # Load Python Imaging Library
import Image
# No compiler, no problem # Load Python Imaging Library
try: import Image except:
# No compiler, no problem # Load Python Imaging Library
try: import Image except: import sys sys.modules['Image'] = ( sys.modules['sys'])
# No compiler, no problem # Load Python Imaging Library
try: import Image except: # beloved Django ImageField import sys sys.modules['Image'] = ( sys.modules['sys'])
# No compiler, no problem @skipIf(base.depends.Image is None) def test():
# ...
# No virtualenv vendor/
# No virtualenv vendor/ >>> sys.path.append(...)
# No virtualenv vendor/ >>> sys.path.append(...) search for: [kitsune vendor
library]
# settings.py DATABASES = { 'default': { 'NAME': 'site.db', 'ENGINE':
'sqlite3', }, }
# settings.py DATABASES = { 'default': { 'NAME': 'site.db', 'ENGINE':
'sqlite3', }, } if os.environ.get('USE_MYSQL', ''): DATABASES['default'] = ( OTHER_DATABASES['mysql'])
<openhatch_jenkins> Make sure the install instructions work (daily) build #230:
SUCCESS in 2 min 51 sec
<openhatch_jenkins> Make sure the install instructions work (daily) build #230:
SUCCESS in 2 min 51 sec + humans
$ time pip install twisted 41.277s total
$ time pip install twisted 41.277s total $ time pip
install twisted 5.653s total
$ time pip install twisted 41.277s total $ time pip
install twisted 5.653s total PIP_DOWNLOAD_CACHE PIP_FIND_LINKS PIP_USE_WHEEL PIP_WHEEL_DIR
$ time pip install twisted 41.277s total $ time pip
install twisted 5.653s total http://pip2014.com/
O WHEELY also conda pypi-install dh-virtualenv
Goals 1. Be able to install 2. Be able to
distribute 3. Know how to learn more
asheesh.org/scratch/Preso packaging.python.org pip2014.com Stick around for coderanger! asheesh.org @asheeshlaroia
(apologies & thanks) asheesh.org/scratch/Preso packaging.python.org pip2014.com Stick around for coderanger!
asheesh.org @asheeshlaroia
(apologies & thanks) asheesh.org/scratch/Preso packaging.python.org glyph.im/pip Stick around for coderanger!
asheesh.org @asheeshlaroia