Save 37% off PRO during our Black Friday Sale! »

Inside The Cheeseshop: How Python Packaging Works (SciPy 2018)

Inside The Cheeseshop: How Python Packaging Works (SciPy 2018)

Questions and confusion about the Python packaging ecosystem abound. What is this `setup.py` file? What's the difference between wheels and eggs? Do I use setuptools or distutils? Why should I use twine? Do I put my projects dependencies in a `requirements.txt` or in `setup.py`? How do I just get my module up on PyPI? Should I use `pip` or `conda`? Wait, what is Warehouse?

This talk identifies the key tools one might encounter when trying to distribute Python software, what they are used for, why they exist, and their history (including where their weird names come from). In addition, we'll see how they all work together, what it takes to make them work, and what the future has in store for Python packaging.

8db101b3a812b0464c53e5409c12ea94?s=128

Dustin Ingram

July 11, 2018
Tweet

Transcript

  1. SciPy 2018 SciPy 2018 — Dus/n Ingram — @di_codes

  2. Hi, I'm Dus+n

  3. Who am I? • Director @ PromptWorks in Aus4n, TX

    • PyTexas Organizer • Python Packaging Working Group (Packaging-WG) member • Python Packaging Authority (PyPA) member • Python Package Index (PyPI) maintainer SciPy 2018 — Dus/n Ingram — @di_codes
  4. {{ talk.title }} {{ talk.subtitle }} SciPy 2018 — Dus/n

    Ingram — @di_codes
  5. Python Packaging Ge#ng the Code You Wrote To People That

    Want It Using the Same Language You Wrote It In SciPy 2018 — Dus/n Ingram — @di_codes
  6. None
  7. Python Packaging In Just Five Easy Steps! SciPy 2018 —

    Dus/n Ingram — @di_codes
  8. None
  9. Python Packaging In Just FOUR Easy Steps! SciPy 2018 —

    Dus/n Ingram — @di_codes
  10. None
  11. Python Packaging It's Rela*vely Painless Now Go Ahead and Use

    It SciPy 2018 — Dus/n Ingram — @di_codes
  12. None
  13. Python Packaging We're S'll Trying To Make It Be5er SciPy

    2018 — Dus/n Ingram — @di_codes
  14. None
  15. Python Packaging Let's Just Throw It All Away And Start

    Over From Scratch SciPy 2018 — Dus/n Ingram — @di_codes
  16. None
  17. Python Packaging Let Me Just Get You Up To Speed

    On Everything That's Changed Since Last Time SciPy 2018 — Dus/n Ingram — @di_codes
  18. None
  19. Python Packaging There's A Lot Of Stuff Here You Might

    Not Need All Of It SciPy 2018 — Dus/n Ingram — @di_codes
  20. None
  21. Python Packaging In the Simplest Terms Possible For Anyone That

    Cares SciPy 2018 — Dus/n Ingram — @di_codes
  22. None
  23. Python Packaging So Easy A Caveman Could Do It SciPy

    2018 — Dus/n Ingram — @di_codes
  24. None
  25. Hello I Am a PyPI Maintainer At the Very Least

    I Should Be Able To Tell You How To Use PyPI SciPy 2018 — Dus/n Ingram — @di_codes
  26. None
  27. Python Packaging SciPy 2018 — Dus/n Ingram — @di_codes

  28. Python Packaging But Indiana Jones-themed SciPy 2018 — Dus/n Ingram

    — @di_codes
  29. Python Packaging But Indiana Jones-themed (Also In French) SciPy 2018

    — Dus/n Ingram — @di_codes
  30. None
  31. Links • h#ps:/ /www.youtube.com/watch?v=qOH-h-EKKac • h#ps:/ /www.youtube.com/watch?v=5BqAeN-F9Qs • h#ps:/ /www.youtube.com/watch?v=gc9dkktg1gU

    • h#ps:/ /www.youtube.com/watch? v=W8A2bOKPtJU • h#ps:/ /www.youtube.com/watch?v=eLPiPHr6TVI • h#ps:/ /www.youtube.com/watch? SciPy 2018 — Dus/n Ingram — @di_codes
  32. The End SciPy 2018 — Dus/n Ingram — @di_codes

  33. Inside The Cheeseshop How Python Packaging Works SciPy 2018 —

    Dus/n Ingram — @di_codes
  34. None
  35. None
  36. None
  37. None
  38. None
  39. totally_awesome_library.py SciPy 2018 — Dus/n Ingram — @di_codes

  40. Problem: How do I get this to users? SciPy 2018

    — Dus/n Ingram — @di_codes
  41. Solu%on: Email? SciPy 2018 — Dus/n Ingram — @di_codes

  42. Solu%on: Personal Website? SciPy 2018 — Dus/n Ingram — @di_codes

  43. None
  44. Problem: How do I find Python code? SciPy 2018 —

    Dus/n Ingram — @di_codes
  45. Solu%on: The Vaults of Parnassus SciPy 2018 — Dus/n Ingram

    — @di_codes
  46. None
  47. Problem: How do I build this? SciPy 2018 — Dus/n

    Ingram — @di_codes
  48. Solu%on: distutils SciPy 2018 — Dus/n Ingram — @di_codes

  49. $ python setup.py <...> SciPy 2018 — Dus/n Ingram —

    @di_codes
  50. $ python setup.py build SciPy 2018 — Dus/n Ingram —

    @di_codes
  51. Solu%on: Source Distribu-ons SciPy 2018 — Dus/n Ingram — @di_codes

  52. AKA: sdist SciPy 2018 — Dus/n Ingram — @di_codes

  53. $ python setup.py sdist SciPy 2018 — Dus/n Ingram —

    @di_codes
  54. Problem: Building takes too long SciPy 2018 — Dus/n Ingram

    — @di_codes
  55. Solu%on: Built Distribu+ons SciPy 2018 — Dus/n Ingram — @di_codes

  56. AKA: bdist SciPy 2018 — Dus/n Ingram — @di_codes

  57. $ python setup.py bdist SciPy 2018 — Dus/n Ingram —

    @di_codes
  58. Problem: How can I "do" packaging? SciPy 2018 — Dus/n

    Ingram — @di_codes
  59. Solu%on: Use pla(orm packaging? SciPy 2018 — Dus/n Ingram —

    @di_codes
  60. Sub-Problem: My pla'orm doesn't have packaging SciPy 2018 — Dus/n

    Ingram — @di_codes
  61. Sub-Problem: My pla'orm does have packaging, but I want things

    now! SciPy 2018 — Dus/n Ingram — @di_codes
  62. Solu%on: The Python Package Index SciPy 2018 — Dus/n Ingram

    — @di_codes
  63. None
  64. AKA: "PyPI" SciPy 2018 — Dus/n Ingram — @di_codes

  65. !"# SciPy 2018 — Dus/n Ingram — @di_codes

  66. AKA: "The Cheeseshop" SciPy 2018 — Dus/n Ingram — @di_codes

  67. None
  68. Problem: How can I specify dependencies? SciPy 2018 — Dus/n

    Ingram — @di_codes
  69. Solu%on: setuptools SciPy 2018 — Dus/n Ingram — @di_codes

  70. Problem: Installing is too hard! SciPy 2018 — Dus/n Ingram

    — @di_codes
  71. Solu%on: easy_install SciPy 2018 — Dus/n Ingram — @di_codes

  72. The Egg Distribu.on SciPy 2018 — Dus/n Ingram — @di_codes

  73. None
  74. Problem: easy_install problems SciPy 2018 — Dus/n Ingram — @di_codes

  75. Solu%on: pyinstall SciPy 2018 — Dus/n Ingram — @di_codes

  76. Problem: "pyinstall" is too long SciPy 2018 — Dus/n Ingram

    — @di_codes
  77. Solu%on: pip SciPy 2018 — Dus/n Ingram — @di_codes

  78. eggs SciPy 2018 — Dus/n Ingram — @di_codes

  79. Problem: Applica'on dependencies? SciPy 2018 — Dus/n Ingram — @di_codes

  80. Solu%on: requirements.txt SciPy 2018 — Dus/n Ingram — @di_codes

  81. pip install -r requirements.txt SciPy 2018 — Dus/n Ingram —

    @di_codes
  82. Problem: Installing from PyPI is slow SciPy 2018 — Dus/n

    Ingram — @di_codes
  83. Problem: Trus%ng 3rd Party Domains SciPy 2018 — Dus/n Ingram

    — @di_codes
  84. Solu%on: PyPI begins hos-ng releases SciPy 2018 — Dus/n Ingram

    — @di_codes
  85. None
  86. Problem: Scien&fic Python is a thing SciPy 2018 — Dus/n

    Ingram — @di_codes
  87. Sub-Problem: distu&ls is not up to the task SciPy 2018

    — Dus/n Ingram — @di_codes
  88. Sub-Problem: Build environments? SciPy 2018 — Dus/n Ingram — @di_codes

  89. Sub-Problem: I need prebuilt binaries SciPy 2018 — Dus/n Ingram

    — @di_codes
  90. Sub-Problem: I need more than Python SciPy 2018 — Dus/n

    Ingram — @di_codes
  91. Sub-Problem: I don't even have Python SciPy 2018 — Dus/n

    Ingram — @di_codes
  92. Solu%on: Anaconda & Conda SciPy 2018 — Dus/n Ingram —

    @di_codes
  93. Problem: We need built distribu-ons (again) SciPy 2018 — Dus/n

    Ingram — @di_codes
  94. Solu%on: The Wheel Distribu.on SciPy 2018 — Dus/n Ingram —

    @di_codes
  95. None
  96. None
  97. Problem: PyPI is showing it's age SciPy 2018 — Dus/n

    Ingram — @di_codes
  98. None
  99. None
  100. None
  101. None
  102. Solu%on: Rewrite PyPI from scratch SciPy 2018 — Dus/n Ingram

    — @di_codes
  103. None
  104. AKA: Warehouse SciPy 2018 — Dus/n Ingram — @di_codes

  105. None
  106. The End SciPy 2018 — Dus/n Ingram — @di_codes

  107. Current Problems SciPy 2018 — Dus/n Ingram — @di_codes

  108. Problem: Packaging is s*ll kinda hard SciPy 2018 — Dus/n

    Ingram — @di_codes
  109. Solu%on: The Python Packaging Guide h"ps:/ /packaging.python.org SciPy 2018 —

    Dus/n Ingram — @di_codes
  110. Solu%on: sampleproject h"ps:/ /github.com/pypa/sampleproject SciPy 2018 — Dus/n Ingram —

    @di_codes
  111. Solu%on: General care & maintenance SciPy 2018 — Dus/n Ingram

    — @di_codes
  112. Problem: Packaging is a li+le too easy SciPy 2018 —

    Dus/n Ingram — @di_codes
  113. Python Packaging So Easy a Spammer Could Do It SciPy

    2018 — Dus/n Ingram — @di_codes
  114. Problem: Reproducible Environments SciPy 2018 — Dus/n Ingram — @di_codes

  115. Solu%on: pipfile/pipfile.lock h"ps:/ /github.com/pypa/pipfile SciPy 2018 — Dus/n Ingram —

    @di_codes
  116. Problem: Arbitrary code in setup.py SciPy 2018 — Dus/n Ingram

    — @di_codes
  117. None
  118. Problem: A distu(ls/setuptools dance SciPy 2018 — Dus/n Ingram —

    @di_codes
  119. Solu%on: PEP 517/518 SciPy 2018 — Dus/n Ingram — @di_codes

  120. None
  121. None
  122. pyproject.toml SciPy 2018 — Dus/n Ingram — @di_codes

  123. How to help SciPy 2018 — Dus/n Ingram — @di_codes

  124. None
  125. How to get help SciPy 2018 — Dus/n Ingram —

    @di_codes
  126. How to get help • h#ps:/ /packaging.python.org • Issue tracker

    for a given tool • On IRC: #pypa Freenode channel • h#ps:/ /github.com/pypa/packaging-problems SciPy 2018 — Dus/n Ingram — @di_codes
  127. Talk to me • Github: @di • Twi,er: @di_codes •

    Email: di@python.org SciPy 2018 — Dus/n Ingram — @di_codes
  128. Summary SciPy 2018 — Dus/n Ingram — @di_codes

  129. Thanks! SciPy 2018 — Dus/n Ingram — @di_codes

  130. Talk to me • Github: @di • Twi,er: @di_codes •

    Email: di@python.org • Or, ask me ques,ons now! SciPy 2018 — Dus/n Ingram — @di_codes