We Buy Cheese in a Cheese Shop

Python packaging is ______. But PyPA is trying to solve the problem. Let's take a look at how we got into this mess in the first place, and how did/will PyPA do to fix it.

Transcript

1. We Buy Cheese in a Cheese Shop
2. Cheese Shop
3. Cheese Shop Sketch http://en.wikipedia.org/wiki/Cheese_Shop_sketch
4. https://pypi.python.org/pypi
6. Click this!
7. Why I Hate Django by Cal Henderson at DjangoCon 2008 DjangoCon 2008 59 “I buy my eggs at the cheese shop” - Aaron Straup Cope [MOON LANGUAGE]
8. Me • Call me TP • Follow @uranusjr • RTFM • http://uranusjr.logdown.com/pages/about
9. Terminology
10. PyPI
11. Python Package Index aka The Cheese Shop
12. PyPA
13. Python Packaging Authority Nobody Expects the Python Packaging Authority by Nick Coghlan at PyCon Australia 2013
14. The Spanish Inquisition http://en.wikipedia.org/wiki/The_Spanish_Inquisition_(Monty_Python)
15. Egg
16. Eggs are to Pythons as Jars are to Java http://peak.telecommunity.com/DevCenter/PythonEggs
18. Wheel
20. Wheels as in Cheese Shops
21. http://www.pythonwheels.com (retrieved on 2013-11-30)
22. http://www.python.org/dev/peps/pep-0427/
23. Wheel is… • The replacement of egg • Standardized • Alternative Python implementation • API tag • Packaged with better structure and metadata • Performance • Security
24. But Why? If It Ain’t Broke, Don’t Fix It
25. Because Eggs Are Broken No pun intended
26. Where We Are Now
27. Package Author Server You
28. Package Author Server You
29. distutils • I am no expert on neither itself nor its history • Built-in since Python 1.6 (2000) • Nothing really changed since about 10 years ago Full documentation: http://docs.python.org/library/distutils.html
30. Container What you want to deploy Full documentation: http://docs.python.org/library/distutils.html
31. The “Makefile” Full documentation: http://docs.python.org/library/distutils.html
32. Extra rules on non-module information Full documentation: http://docs.python.org/library/distutils.html
33. from distutils.core import setup! ! setup(! name='mypackage',! version='1.0',! packages=['mymodule'],! description='My package',! url='http://myawesomepackage.com',! author='Me the Author',! author_email='im@aweso.me'! ) setup.py Documentation: http://docs.python.org/distutils/setupscript.html
34. include README.rst! recursive-exclude * __pycache__! recursive-exclude * *.py[co]! include docs! prune docs/build MANIFEST.in Documentation: http://docs.python.org/distutils/sourcedist.html
35. $ python setup.py sdist! ! $ python setup.py bdist_wininst Documentation: http://docs.python.org/distutils/sourcedist.html
37. $ python setup.py bdist_egg Documentation: http://peak.telecommunity.com/DevCenter/PythonEggs from setuptools import setup! ! setup(! # ...! )
38. Source distribution
39. Binary egg
40. Windows installer
41. Does this make sense now? DjangoCon 2008 58 “I buy my eggs at the cheese shop” - Aaron Straup Cope
43. Package metadata (collected from setup)
44. The “zip safety” flag
45. Top level modules
46. Files in the package
47. Depended packages
48. Package information (as seen on PyPI)
49. Where to find packages (aside from PyPI)
50. Package Author Server You
51. $ python setup.py register! ! $ python setup.py sdist bdist_egg upload Documentation: http://docs.python.org/distutils/packageindex.html
52. Package Author Server You
53. Package Author PyPI You
54. Package Author PyPI mirror mirror You
55. Package Author PyPI Self-hosted Server You
56. [Y]ou don't have to worry about it. They've figured it all out already. The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky
57. Actually they did not; I just don’t want to talk about it.
58. Package Author Server You
59. 1. Go to PyPI and search for a package 2. Download 3. Install • Extract • python setup.py install The Manual Way
60. 1. Install Setuptools 2. Run easy_install <package>! 3. Profit! With Setuptools Documentation: https://pypi.python.org/pypi/setuptools#installation-instructions
61. Problem?
62. LIAR!!!1
64. What is zip-safe, anyway?
66. Why PIP? http://www.pip-installer.org/en/latest/other-tools.html#easy-install • All packages are downloaded before installation • Flat installation by default • Uninstallation • Clean code™ • PyPI with HTTPS by default
67. Why NOT PIP? http://www.pip-installer.org/en/latest/other-tools.html#easy-install • Incompatible with certain legacy packages (rare) • Cannot install from bdist packages (except eggs)
68. ઌผ㘸Ṝݸྃ! 㟬ᡒա distribute 䆩ʁ
69. Commit history of https://bitbucket.org/pypa/setuptools/
70. https://bitbucket.org/tarek/distribute/overview
71. distribute • Intended to replace Setuptools • Shares the same module names (setuptools and pkg_resources) • Active development https://bitbucket.org/tarek/distribute/overview
72. http://xkcd.com/927/
73. We Need a New Standard!
75. http://ziade.org/2010/03/03/the-fate-of-distutils-pycon-summit-packaging-sprint-detailed-report/
76. http://www.python.org/dev/peps/pep-0376/
77. Not Even in Python 3.4
78. This is what you get when developers fight with each other. Panel: Directions for Packaging, PyCON US 2013
82. Panel: Directions for Packaging, PyCON US 2013 ./setup.py install must die!
83. PyPA’s Schedule • Merge Distribute back to Setuptools (Done) • New binary distribution format (Done) • Built-in Setuptools and pip support in Python • Better security • …And more! The Future of Python Packaging, Python Packaging User Guide
84. https://github.com/pypa/pip/commit/b90fc1a
85. pip >= 1.4 setuptools >= 0.8 Upgrading notes: http://pythonhosted.org/setuptools/merge-faq.html
86. $ python setup.py bdist_wheel upload Documentation: docs.python.org/distutils/packageindex.html $ pip install --use-wheel <package>
87. MORE IS BETTER
88. Pillow-2.2.1-cp32-none-win32.whl Name Version Python tag ABI tag Platform I’m a wheel!
89. django_mosql-0.2-py27-none-any.whl Replaces dashes with underscores Implementation not relevant Does not need ABI information Platform independent
90. Description gets its own file!
91. Other metadata (similar to PKG-INFO)
92. Unchanged
93. ???
94. Metadata in JSON
95. Wheel format specification
96. SHA256 for each file in this directory
97. What Changed? • The One True Cheese Shop • Forward compatibility • Machine-friendliness • Safety
98. Wrapping Up • Python packaging is _____ • How did we get into this mess? • distutils • setuptools • PyPI • A way out
99. Wheel is the future. Adapt today!