We Buy Cheese in a Cheese Shop

We Buy Cheese in a Cheese Shop

Python packaging is ______. But PyPA is trying to solve the problem. Let's take a look at how we got into this mess in the first place, and how did/will PyPA do to fix it.

9dafad54b5b4f360b7aae5f482bc1c91?s=128

Tzu-ping Chung

December 26, 2013
Tweet

Transcript

  1. We Buy Cheese in a Cheese Shop

  2. Cheese Shop

  3. Cheese Shop Sketch http://en.wikipedia.org/wiki/Cheese_Shop_sketch

  4. https://pypi.python.org/pypi

  5. None
  6. Click this!

  7. Why I Hate Django by Cal Henderson at DjangoCon 2008

    DjangoCon 2008 59 “I buy my eggs at the cheese shop” - Aaron Straup Cope [MOON LANGUAGE]
  8. Me • Call me TP • Follow @uranusjr • RTFM

    • http://uranusjr.logdown.com/pages/about
  9. Terminology

  10. PyPI

  11. Python Package Index aka The Cheese Shop

  12. PyPA

  13. Python Packaging Authority Nobody Expects the Python Packaging Authority by

    Nick Coghlan at PyCon Australia 2013
  14. The Spanish Inquisition http://en.wikipedia.org/wiki/The_Spanish_Inquisition_(Monty_Python)

  15. Egg

  16. Eggs are to Pythons as Jars are to Java http://peak.telecommunity.com/DevCenter/PythonEggs

  17. None
  18. Wheel

  19. None
  20. Wheels as in Cheese Shops

  21. http://www.pythonwheels.com (retrieved on 2013-11-30)

  22. http://www.python.org/dev/peps/pep-0427/

  23. Wheel is… • The replacement of egg • Standardized •

    Alternative Python implementation • API tag • Packaged with better structure and metadata • Performance • Security
  24. But Why? If It Ain’t Broke, Don’t Fix It

  25. Because Eggs Are Broken No pun intended

  26. Where We Are Now

  27. Package Author Server You

  28. Package Author Server You

  29. distutils • I am no expert on neither itself nor

    its history • Built-in since Python 1.6 (2000) • Nothing really changed since about 10 years ago Full documentation: http://docs.python.org/library/distutils.html
  30. Container What you want to deploy Full documentation: http://docs.python.org/library/distutils.html

  31. The “Makefile” Full documentation: http://docs.python.org/library/distutils.html

  32. Extra rules on non-module information Full documentation: http://docs.python.org/library/distutils.html

  33. from distutils.core import setup! ! setup(! name='mypackage',! version='1.0',! packages=['mymodule'],! description='My

    package',! url='http://myawesomepackage.com',! author='Me the Author',! author_email='im@aweso.me'! ) setup.py Documentation: http://docs.python.org/distutils/setupscript.html
  34. include README.rst! recursive-exclude * __pycache__! recursive-exclude * *.py[co]! include docs!

    prune docs/build MANIFEST.in Documentation: http://docs.python.org/distutils/sourcedist.html
  35. $ python setup.py sdist! ! $ python setup.py bdist_wininst Documentation:

    http://docs.python.org/distutils/sourcedist.html
  36. None
  37. $ python setup.py bdist_egg Documentation: http://peak.telecommunity.com/DevCenter/PythonEggs from setuptools import setup!

    ! setup(! # ...! )
  38. Source distribution

  39. Binary egg

  40. Windows installer

  41. Does this make sense now? DjangoCon 2008 58 “I buy

    my eggs at the cheese shop” - Aaron Straup Cope
  42. None
  43. Package metadata (collected from setup)

  44. The “zip safety” flag

  45. Top level modules

  46. Files in the package

  47. Depended packages

  48. Package information (as seen on PyPI)

  49. Where to find packages (aside from PyPI)

  50. Package Author Server You

  51. $ python setup.py register! ! $ python setup.py sdist bdist_egg

    upload Documentation: http://docs.python.org/distutils/packageindex.html
  52. Package Author Server You

  53. Package Author PyPI You

  54. Package Author PyPI mirror mirror You

  55. Package Author PyPI Self-hosted Server You

  56. [Y]ou don't have to worry about it. They've figured it

    all out already. The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky
  57. Actually they did not; I just don’t want to talk

    about it.
  58. Package Author Server You

  59. 1. Go to PyPI and search for a package 2.

    Download 3. Install • Extract • python setup.py install The Manual Way
  60. 1. Install Setuptools 2. Run easy_install <package>! 3. Profit! With

    Setuptools Documentation: https://pypi.python.org/pypi/setuptools#installation-instructions
  61. Problem?

  62. LIAR!!!1

  63. None
  64. What is zip-safe, anyway?

  65. None
  66. Why PIP? http://www.pip-installer.org/en/latest/other-tools.html#easy-install • All packages are downloaded before installation

    • Flat installation by default • Uninstallation • Clean code™ • PyPI with HTTPS by default
  67. Why NOT PIP? http://www.pip-installer.org/en/latest/other-tools.html#easy-install • Incompatible with certain legacy packages

    (rare) • Cannot install from bdist packages (except eggs)
  68. ઌผ㘸Ṝݸྃ! 㟬ᡒա distribute 䆩ʁ

  69. Commit history of https://bitbucket.org/pypa/setuptools/

  70. https://bitbucket.org/tarek/distribute/overview

  71. distribute • Intended to replace Setuptools • Shares the same

    module names (setuptools and pkg_resources) • Active development https://bitbucket.org/tarek/distribute/overview
  72. http://xkcd.com/927/

  73. We Need a New Standard!

  74. None
  75. http://ziade.org/2010/03/03/the-fate-of-distutils-pycon-summit-packaging-sprint-detailed-report/

  76. http://www.python.org/dev/peps/pep-0376/

  77. Not Even in Python 3.4

  78. This is what you get when developers fight with each

    other. Panel: Directions for Packaging, PyCON US 2013
  79. None
  80. None
  81. None
  82. Panel: Directions for Packaging, PyCON US 2013 ./setup.py install must

    die!
  83. PyPA’s Schedule • Merge Distribute back to Setuptools (Done) •

    New binary distribution format (Done) • Built-in Setuptools and pip support in Python • Better security • …And more! The Future of Python Packaging, Python Packaging User Guide
  84. https://github.com/pypa/pip/commit/b90fc1a

  85. pip >= 1.4 setuptools >= 0.8 Upgrading notes: http://pythonhosted.org/setuptools/merge-faq.html

  86. $ python setup.py bdist_wheel upload Documentation: docs.python.org/distutils/packageindex.html $ pip install

    --use-wheel <package>
  87. MORE IS BETTER

  88. Pillow-2.2.1-cp32-none-win32.whl Name Version Python tag ABI tag Platform I’m a

    wheel!
  89. django_mosql-0.2-py27-none-any.whl Replaces dashes with underscores Implementation not relevant Does not

    need ABI information Platform independent
  90. Description gets its own file!

  91. Other metadata (similar to PKG-INFO)

  92. Unchanged

  93. ???

  94. Metadata in JSON

  95. Wheel format specification

  96. SHA256 for each file in this directory

  97. What Changed? • The One True Cheese Shop • Forward

    compatibility • Machine-friendliness • Safety
  98. Wrapping Up • Python packaging is _____ • How did

    we get into this mess? • distutils • setuptools • PyPI • A way out
  99. Wheel is the future. Adapt today!

  100. None