Inside The Cheeseshop: How Python Packaging Works (SciPy 2018)

Questions and confusion about the Python packaging ecosystem abound. What is this `setup.py` file? What's the difference between wheels and eggs? Do I use setuptools or distutils? Why should I use twine? Do I put my projects dependencies in a `requirements.txt` or in `setup.py`? How do I just get my module up on PyPI? Should I use `pip` or `conda`? Wait, what is Warehouse?

This talk identifies the key tools one might encounter when trying to distribute Python software, what they are used for, why they exist, and their history (including where their weird names come from). In addition, we'll see how they all work together, what it takes to make them work, and what the future has in store for Python packaging.

Transcript

1. SciPy 2018 SciPy 2018 — Dus/n Ingram — @di_codes
2. Hi, I'm Dus+n
3. Who am I? • Director @ PromptWorks in Aus4n, TX • PyTexas Organizer • Python Packaging Working Group (Packaging-WG) member • Python Packaging Authority (PyPA) member • Python Package Index (PyPI) maintainer SciPy 2018 — Dus/n Ingram — @di_codes
4. {{ talk.title }} {{ talk.subtitle }} SciPy 2018 — Dus/n Ingram — @di_codes
5. Python Packaging Ge#ng the Code You Wrote To People That Want It Using the Same Language You Wrote It In SciPy 2018 — Dus/n Ingram — @di_codes
7. Python Packaging In Just Five Easy Steps! SciPy 2018 — Dus/n Ingram — @di_codes
9. Python Packaging In Just FOUR Easy Steps! SciPy 2018 — Dus/n Ingram — @di_codes
11. Python Packaging It's Rela*vely Painless Now Go Ahead and Use It SciPy 2018 — Dus/n Ingram — @di_codes
13. Python Packaging We're S'll Trying To Make It Be5er SciPy 2018 — Dus/n Ingram — @di_codes
15. Python Packaging Let's Just Throw It All Away And Start Over From Scratch SciPy 2018 — Dus/n Ingram — @di_codes
17. Python Packaging Let Me Just Get You Up To Speed On Everything That's Changed Since Last Time SciPy 2018 — Dus/n Ingram — @di_codes
19. Python Packaging There's A Lot Of Stuff Here You Might Not Need All Of It SciPy 2018 — Dus/n Ingram — @di_codes
21. Python Packaging In the Simplest Terms Possible For Anyone That Cares SciPy 2018 — Dus/n Ingram — @di_codes
23. Python Packaging So Easy A Caveman Could Do It SciPy 2018 — Dus/n Ingram — @di_codes
25. Hello I Am a PyPI Maintainer At the Very Least I Should Be Able To Tell You How To Use PyPI SciPy 2018 — Dus/n Ingram — @di_codes
27. Python Packaging SciPy 2018 — Dus/n Ingram — @di_codes
28. Python Packaging But Indiana Jones-themed SciPy 2018 — Dus/n Ingram — @di_codes
29. Python Packaging But Indiana Jones-themed (Also In French) SciPy 2018 — Dus/n Ingram — @di_codes
31. Links • h#ps:/ /www.youtube.com/watch?v=qOH-h-EKKac • h#ps:/ /www.youtube.com/watch?v=5BqAeN-F9Qs • h#ps:/ /www.youtube.com/watch?v=gc9dkktg1gU • h#ps:/ /www.youtube.com/watch? v=W8A2bOKPtJU • h#ps:/ /www.youtube.com/watch?v=eLPiPHr6TVI • h#ps:/ /www.youtube.com/watch? SciPy 2018 — Dus/n Ingram — @di_codes
32. The End SciPy 2018 — Dus/n Ingram — @di_codes
33. Inside The Cheeseshop How Python Packaging Works SciPy 2018 — Dus/n Ingram — @di_codes
39. totally_awesome_library.py SciPy 2018 — Dus/n Ingram — @di_codes
40. Problem: How do I get this to users? SciPy 2018 — Dus/n Ingram — @di_codes
41. Solu%on: Email? SciPy 2018 — Dus/n Ingram — @di_codes
42. Solu%on: Personal Website? SciPy 2018 — Dus/n Ingram — @di_codes
44. Problem: How do I find Python code? SciPy 2018 — Dus/n Ingram — @di_codes
45. Solu%on: The Vaults of Parnassus SciPy 2018 — Dus/n Ingram — @di_codes
47. Problem: How do I build this? SciPy 2018 — Dus/n Ingram — @di_codes
48. Solu%on: distutils SciPy 2018 — Dus/n Ingram — @di_codes
49. $ python setup.py <...> SciPy 2018 — Dus/n Ingram — @di_codes
50. $ python setup.py build SciPy 2018 — Dus/n Ingram — @di_codes
51. Solu%on: Source Distribu-ons SciPy 2018 — Dus/n Ingram — @di_codes
52. AKA: sdist SciPy 2018 — Dus/n Ingram — @di_codes
53. $ python setup.py sdist SciPy 2018 — Dus/n Ingram — @di_codes
54. Problem: Building takes too long SciPy 2018 — Dus/n Ingram — @di_codes
55. Solu%on: Built Distribu+ons SciPy 2018 — Dus/n Ingram — @di_codes
56. AKA: bdist SciPy 2018 — Dus/n Ingram — @di_codes
57. $ python setup.py bdist SciPy 2018 — Dus/n Ingram — @di_codes
58. Problem: How can I "do" packaging? SciPy 2018 — Dus/n Ingram — @di_codes
59. Solu%on: Use pla(orm packaging? SciPy 2018 — Dus/n Ingram — @di_codes
60. Sub-Problem: My pla'orm doesn't have packaging SciPy 2018 — Dus/n Ingram — @di_codes
61. Sub-Problem: My pla'orm does have packaging, but I want things now! SciPy 2018 — Dus/n Ingram — @di_codes
62. Solu%on: The Python Package Index SciPy 2018 — Dus/n Ingram — @di_codes
64. AKA: "PyPI" SciPy 2018 — Dus/n Ingram — @di_codes
65. !"# SciPy 2018 — Dus/n Ingram — @di_codes
66. AKA: "The Cheeseshop" SciPy 2018 — Dus/n Ingram — @di_codes
68. Problem: How can I specify dependencies? SciPy 2018 — Dus/n Ingram — @di_codes
69. Solu%on: setuptools SciPy 2018 — Dus/n Ingram — @di_codes
70. Problem: Installing is too hard! SciPy 2018 — Dus/n Ingram — @di_codes
71. Solu%on: easy_install SciPy 2018 — Dus/n Ingram — @di_codes
72. The Egg Distribu.on SciPy 2018 — Dus/n Ingram — @di_codes
74. Problem: easy_install problems SciPy 2018 — Dus/n Ingram — @di_codes
75. Solu%on: pyinstall SciPy 2018 — Dus/n Ingram — @di_codes
76. Problem: "pyinstall" is too long SciPy 2018 — Dus/n Ingram — @di_codes
77. Solu%on: pip SciPy 2018 — Dus/n Ingram — @di_codes
78. eggs SciPy 2018 — Dus/n Ingram — @di_codes
79. Problem: Applica'on dependencies? SciPy 2018 — Dus/n Ingram — @di_codes
80. Solu%on: requirements.txt SciPy 2018 — Dus/n Ingram — @di_codes
81. pip install -r requirements.txt SciPy 2018 — Dus/n Ingram — @di_codes
82. Problem: Installing from PyPI is slow SciPy 2018 — Dus/n Ingram — @di_codes
83. Problem: Trus%ng 3rd Party Domains SciPy 2018 — Dus/n Ingram — @di_codes
84. Solu%on: PyPI begins hos-ng releases SciPy 2018 — Dus/n Ingram — @di_codes
86. Problem: Scien&fic Python is a thing SciPy 2018 — Dus/n Ingram — @di_codes
87. Sub-Problem: distu&ls is not up to the task SciPy 2018 — Dus/n Ingram — @di_codes
88. Sub-Problem: Build environments? SciPy 2018 — Dus/n Ingram — @di_codes
89. Sub-Problem: I need prebuilt binaries SciPy 2018 — Dus/n Ingram — @di_codes
90. Sub-Problem: I need more than Python SciPy 2018 — Dus/n Ingram — @di_codes
91. Sub-Problem: I don't even have Python SciPy 2018 — Dus/n Ingram — @di_codes
92. Solu%on: Anaconda & Conda SciPy 2018 — Dus/n Ingram — @di_codes
93. Problem: We need built distribu-ons (again) SciPy 2018 — Dus/n Ingram — @di_codes
94. Solu%on: The Wheel Distribu.on SciPy 2018 — Dus/n Ingram — @di_codes
97. Problem: PyPI is showing it's age SciPy 2018 — Dus/n Ingram — @di_codes
102. Solu%on: Rewrite PyPI from scratch SciPy 2018 — Dus/n Ingram — @di_codes
104. AKA: Warehouse SciPy 2018 — Dus/n Ingram — @di_codes
106. The End SciPy 2018 — Dus/n Ingram — @di_codes
107. Current Problems SciPy 2018 — Dus/n Ingram — @di_codes
108. Problem: Packaging is s*ll kinda hard SciPy 2018 — Dus/n Ingram — @di_codes
109. Solu%on: The Python Packaging Guide h"ps:/ /packaging.python.org SciPy 2018 — Dus/n Ingram — @di_codes
110. Solu%on: sampleproject h"ps:/ /github.com/pypa/sampleproject SciPy 2018 — Dus/n Ingram — @di_codes
111. Solu%on: General care & maintenance SciPy 2018 — Dus/n Ingram — @di_codes
112. Problem: Packaging is a li+le too easy SciPy 2018 — Dus/n Ingram — @di_codes
113. Python Packaging So Easy a Spammer Could Do It SciPy 2018 — Dus/n Ingram — @di_codes
114. Problem: Reproducible Environments SciPy 2018 — Dus/n Ingram — @di_codes
115. Solu%on: pipfile/pipfile.lock h"ps:/ /github.com/pypa/pipfile SciPy 2018 — Dus/n Ingram — @di_codes
116. Problem: Arbitrary code in setup.py SciPy 2018 — Dus/n Ingram — @di_codes
118. Problem: A distu(ls/setuptools dance SciPy 2018 — Dus/n Ingram — @di_codes
119. Solu%on: PEP 517/518 SciPy 2018 — Dus/n Ingram — @di_codes
122. pyproject.toml SciPy 2018 — Dus/n Ingram — @di_codes
123. How to help SciPy 2018 — Dus/n Ingram — @di_codes
125. How to get help SciPy 2018 — Dus/n Ingram — @di_codes
126. How to get help • h#ps:/ /packaging.python.org • Issue tracker for a given tool • On IRC: #pypa Freenode channel • h#ps:/ /github.com/pypa/packaging-problems SciPy 2018 — Dus/n Ingram — @di_codes
127. Talk to me • Github: @di • Twi,er: @di_codes • Email: di@python.org SciPy 2018 — Dus/n Ingram — @di_codes
128. Summary SciPy 2018 — Dus/n Ingram — @di_codes
129. Thanks! SciPy 2018 — Dus/n Ingram — @di_codes
130. Talk to me • Github: @di • Twi,er: @di_codes • Email: di@python.org • Or, ask me ques,ons now! SciPy 2018 — Dus/n Ingram — @di_codes