Inside The Cheeseshop: How Python Packaging Works (PyOhio 2018)

Questions and confusion about the Python packaging ecosystem abound. What is this `setup.py` file? What's the difference between wheels and eggs? Do I use setuptools or distutils? Why should I use twine? Do I put my projects dependencies in a `requirements.txt` or in `setup.py`? How do I just get my module up on PyPI? Should I use `pip` or `conda`? Wait, what is Warehouse?

This talk identifies the key tools one might encounter when trying to distribute Python software, what they are used for, why they exist, and their history (including where their weird names come from). In addition, we'll see how they all work together, what it takes to make them work, and what the future has in store for Python packaging.

Transcript

1. PyOhio 2018 @di_codes
2. Hi, I'm Dus+n
3. Who am I? • Python Packaging Working Group (Packaging-WG) member • Python Packaging Authority (PyPA) member • Python Package Index (PyPI) maintainer/contributor/admin • Developer Advocate @ Google @di_codes
4. {{ talk.title }} {{ talk.subtitle }} @di_codes
5. Python Packaging Ge#ng the Code You Wrote To the People That Want It Using the Same Language You Wrote It In @di_codes
7. Python Packaging In Just Five Easy Steps! @di_codes
9. Python Packaging In Just FOUR Easy Steps! @di_codes
11. Python Packaging It's Rela*vely Painless Now Go Ahead and Use It @di_codes
13. Python Packaging We're S'll Trying To Make It Be5er @di_codes
15. Python Packaging Let's Just Throw It All Away And Start Over From Scratch @di_codes
17. Python Packaging Let Me Just Get You Up To Speed On Everything That's Changed Since Last Time @di_codes
19. Python Packaging There's A Lot Of Stuff Here You Might Not Need All Of It @di_codes
21. Python Packaging In the Simplest Terms Possible For Anyone That Cares @di_codes
23. Python Packaging So Easy A Caveman Could Do It @di_codes
25. Hello I Am a PyPI Maintainer At the Very Least I Should Be Able To Tell You How To Use PyPI @di_codes
27. Python Packaging @di_codes
28. Python Packaging But Indiana Jones-themed @di_codes
29. Python Packaging But Indiana Jones-themed (Also In French) @di_codes
31. Links • h#ps:/ /www.youtube.com/watch?v=qOH-h-EKKac • h#ps:/ /www.youtube.com/watch?v=5BqAeN-F9Qs • h#ps:/ /www.youtube.com/watch?v=gc9dkktg1gU • h#ps:/ /www.youtube.com/watch?v=W8A2bOKPtJU • h#ps:/ /www.youtube.com/watch?v=eLPiPHr6TVI @di_codes
32. The End @di_codes
33. Inside The Cheeseshop How Python Packaging Works @di_codes
36. h"ps:/ /xkcd.com/1987/
39. totally_awesome_library.py @di_codes
40. Problem: How do I get this to users? @di_codes
41. Solu%on: Email? @di_codes
42. Solu%on: Personal Website? @di_codes
44. Problem: How do I find Python code? @di_codes
45. Solu%on: The Vaults of Parnassus @di_codes
47. Problem: How do I build this? @di_codes
48. Solu%on: distutils @di_codes
49. $ python setup.py <...> @di_codes
50. $ python setup.py build @di_codes
51. Solu%on: Source Distribu-ons @di_codes
52. AKA: sdist @di_codes
53. AKA: ssssssssssssssdist @di_codes
54. $ python setup.py sdist @di_codes
55. Problem: Building takes too long @di_codes
56. Solu%on: Built Distribu+ons @di_codes
57. AKA: bdist @di_codes
58. $ python setup.py bdist @di_codes
59. Problem: How can I "do" packaging? @di_codes
60. Solu%on: Use pla(orm packaging? @di_codes
61. Sub-Problem: My pla'orm doesn't have packaging @di_codes
62. Sub-Problem: My pla'orm does have packaging, but I want things now! @di_codes
63. Solu%on: The Python Package Index @di_codes
65. AKA: "PyPI" @di_codes
66. !"# @di_codes
67. AKA: "The Cheeseshop" @di_codes
69. Problem: How can I specify dependencies? @di_codes
70. Solu%on: setuptools @di_codes
71. Problem: Installing is too hard! @di_codes
72. Solu%on: easy_install @di_codes
73. The Egg Distribu.on @di_codes
75. Problem: easy_install problems @di_codes
76. Solu%on: pyinstall @di_codes
77. Problem: "pyinstall" is too long @di_codes
78. Solu%on: pip @di_codes
79. eggs @di_codes
80. Problem: Applica'on dependencies? @di_codes
81. Solu%on: requirements.txt @di_codes
82. pip install -r requirements.txt @di_codes
83. Problem: Installing from PyPI is slow @di_codes
84. Problem: Trus%ng 3rd Party Domains @di_codes
85. Solu%on: PyPI begins hos-ng releases @di_codes
87. Problem: We need built distribu-ons (again) @di_codes
88. Solu%on: The Wheel Distribu.on @di_codes
91. Problem: python setup.py upload doesn't use HTTPS @di_codes
92. Solu%on: twine @di_codes
93. @di_codes
94. Problem: PyPI is showing it's age @di_codes
99. Solu%on: Rewrite PyPI from scratch @di_codes
101. AKA: Warehouse @di_codes
103. The End @di_codes
104. Current Problems @di_codes
105. Problem: Packaging is s*ll kinda hard @di_codes
106. Solu%on: The Python Packaging Guide h"ps:/ /packaging.python.org @di_codes
107. Solu%on: sampleproject h"ps:/ /github.com/pypa/sampleproject @di_codes
108. Solu%on: General care & maintenance @di_codes
109. Problem: Packaging is a li+le too easy @di_codes
110. Python Packaging So Easy a Spammer Could Do It @di_codes
111. Problem: Some%mes I need more than Python (some%mes I don't even have it) @di_codes
112. Solu%on: conda @di_codes
113. Problem: Reproducible Environments @di_codes
114. Solu%on: pipfile/pipfile.lock h"ps:/ /github.com/pypa/pipfile @di_codes
115. Problem: Arbitrary code in setup.py @di_codes
117. Problem: The distu*ls/setuptools dance @di_codes
118. Solu%on: PEP 517/518 @di_codes
121. pyproject.toml @di_codes
122. How to help @di_codes
124. Packaging BoF! @di_codes
125. How to get help @di_codes
126. How to get help • h#ps:/ /packaging.python.org • Issue tracker for a given tool • On IRC: #pypa Freenode channel • h#ps:/ /github.com/pypa/packaging-problems @di_codes
127. Summary @di_codes
128. Thanks! • Github: @di • Twi,er: @di_codes • Email: di@python.org @di_codes