Inside The Cheeseshop: How Python Packaging Works (PyGotham 2018)

Questions and confusion about the Python packaging ecosystem abound. What is this setup.py file? What’s the difference between wheels and eggs? Do I use setuptools or distutils? Why should I use twine? Do I put my projects dependencies in a requirements.txt or in setup.py? How do I just get my module up on PyPI? Wait, what is Warehouse?

This talk will identify the key tools one might encounter when trying to distribute Python software, what they are used for, why they exist, and their history (including where their weird names come from). In addition, we’ll see how they all work together, what it takes to make them work, and what the future has in store for Python packaging.

Transcript

1. PyGotham 2018 @di_codes
2. Hi, I'm Dus+n
3. Who am I? • Python Packaging Working Group (Packaging-WG) member • Python Packaging Authority (PyPA) member • Python Package Index (PyPI) maintainer/contributor/admin • Developer Advocate @ Google @di_codes
4. {{ talk.title }} {{ talk.subtitle }} @di_codes
5. Python Packaging Ge#ng the Code You Wrote To the People That Want It Using the Same Language You Wrote It In @di_codes
7. Python Packaging In Just Five Easy Steps! @di_codes
9. Python Packaging In Just FOUR Easy Steps! @di_codes
11. Python Packaging It's Rela*vely Painless Now Go Ahead and Use It @di_codes
13. Python Packaging We're S'll Trying To Make It Be5er @di_codes
15. Python Packaging Let's Just Throw It All Away And Start Over From Scratch @di_codes
17. Python Packaging Let Me Just Get You Up To Speed On Everything That's Changed Since Last Time @di_codes
19. Python Packaging There's A Lot Of Stuff Here You Might Not Need All Of It @di_codes
21. Python Packaging In the Simplest Terms Possible For Anyone That Cares @di_codes
23. Python Packaging So Easy A Caveman Could Do It @di_codes
25. Hello I Am a PyPI Maintainer At the Very Least I Should Be Able To Tell You How To Use PyPI @di_codes
27. Python Packaging @di_codes
28. Python Packaging But Indiana Jones-themed @di_codes
29. Python Packaging But Indiana Jones-themed (Also In French) @di_codes
31. Links • h#ps:/ /www.youtube.com/watch?v=qOH-h-EKKac • h#ps:/ /www.youtube.com/watch?v=5BqAeN-F9Qs • h#ps:/ /www.youtube.com/watch?v=gc9dkktg1gU • h#ps:/ /www.youtube.com/watch?v=W8A2bOKPtJU • h#ps:/ /www.youtube.com/watch?v=eLPiPHr6TVI • h#ps:/ /www.youtube.com/watch?v=iLVNWfPWAC8 @di_codes
32. The End @di_codes
33. Inside The Cheeseshop How Python Packaging Works @di_codes
36. h"ps:/ /xkcd.com/1987/
39. totally_awesome_library.py @di_codes
40. Problem: How do I get this to users? @di_codes
41. Solu%on: Email? @di_codes
42. Solu%on: Personal Website? @di_codes
44. Problem: How do I find Python code? @di_codes
45. Solu%on: The Vaults of Parnassus @di_codes
47. Problem: How do I build this? @di_codes
48. Solu%on: distutils @di_codes
49. $ python setup.py <...> @di_codes
50. $ python setup.py build @di_codes
51. Solu%on: Source Distribu-ons @di_codes
52. AKA: sdist @di_codes
53. AKA: ssssssssssssssdist @di_codes
54. $ python setup.py sdist @di_codes
55. Problem: Building takes too long @di_codes
56. Solu%on: Built Distribu+ons @di_codes
57. AKA: bdist @di_codes
58. $ python setup.py bdist @di_codes
59. Problem: How can I "do" packaging? @di_codes
60. Solu%on: Use pla(orm packaging? @di_codes
61. @di_codes
62. Sub-Problem: My pla'orm doesn't have packaging @di_codes
63. Sub-Problem: My pla'orm does have packaging, but I want things now! @di_codes
64. Solu%on: The Python Package Index @di_codes
66. AKA: "PyPI" @di_codes
70. AKA: "The Cheeseshop" @di_codes
72. Problem: How can I specify dependencies? @di_codes
73. Solu%on: setuptools @di_codes
74. Problem: Installing is too hard! @di_codes
75. Solu%on: easy_install @di_codes
76. The Egg Distribu.on @di_codes
78. Problem: easy_install problems @di_codes
79. Solu%on: pyinstall @di_codes
80. Problem: "pyinstall" is too long @di_codes
81. Solu%on: pip @di_codes
82. eggs @di_codes
83. Problem: Applica'on dependencies? @di_codes
84. Solu%on: requirements.txt @di_codes
85. pip install -r requirements.txt @di_codes
86. Problem: Installing from PyPI is slow @di_codes
87. Problem: Trus%ng 3rd Party Domains @di_codes
88. Solu%on: PyPI begins hos-ng releases @di_codes
90. Problem: We need built distribu-ons (again) @di_codes
91. Solu%on: The Wheel Distribu.on @di_codes
94. Problem: python setup.py upload doesn't use HTTPS @di_codes
95. Solu%on: twine @di_codes
96. @di_codes
97. Problem: PyPI is showing it's age @di_codes
102. Solu%on: Rewrite PyPI from scratch @di_codes
104. AKA: Warehouse @di_codes
106. The End @di_codes
107. Current Problems @di_codes
108. Problem: Packaging is s*ll kinda hard @di_codes
109. Solu%on: The Python Packaging Guide h"ps:/ /packaging.python.org @di_codes
110. Solu%on: sampleproject h"ps:/ /github.com/pypa/sampleproject @di_codes
111. Solu%on: General care & maintenance @di_codes
112. Problem: Packaging is a li+le too easy @di_codes
113. Python Packaging So Easy a Spammer Could Do It @di_codes
114. Problem: Some%mes I need more than Python (some%mes I don't even have it) @di_codes
115. Solu%on: conda @di_codes
116. Problem: Reproducible Environments @di_codes
117. Solu%on: pipfile/pipfile.lock h"ps:/ /github.com/pypa/pipfile @di_codes
118. Problem: Arbitrary code in setup.py @di_codes
119. Problem: The distu*ls/setuptools dance @di_codes
120. Solu%on: PEP 517/518 @di_codes
123. pyproject.toml @di_codes
124. How to help @di_codes
126. Bloomberg Packaging Sprints October 27 & 28 in NYC & LON @di_codes
128. How to get help @di_codes
129. How to get help • @PyPI, @ThePyPA • h,ps:/ /packaging.python.org • Issue tracker for a given tool • On IRC: #pypa Freenode channel • h,ps:/ /github.com/pypa/packaging-problems @di_codes
130. Summary @di_codes
131. Thanks! • Github: @di • Twi,er: @di_codes • Email: di@python.org @di_codes