Inside The Cheeseshop: How Python Packaging Works (PyOhio 2018)

Inside The Cheeseshop: How Python Packaging Works (PyOhio 2018)

Questions and confusion about the Python packaging ecosystem abound. What is this `setup.py` file? What's the difference between wheels and eggs? Do I use setuptools or distutils? Why should I use twine? Do I put my projects dependencies in a `requirements.txt` or in `setup.py`? How do I just get my module up on PyPI? Should I use `pip` or `conda`? Wait, what is Warehouse?

This talk identifies the key tools one might encounter when trying to distribute Python software, what they are used for, why they exist, and their history (including where their weird names come from). In addition, we'll see how they all work together, what it takes to make them work, and what the future has in store for Python packaging.

8db101b3a812b0464c53e5409c12ea94?s=128

Dustin Ingram

July 28, 2018
Tweet

Transcript

  1. PyOhio 2018 @di_codes

  2. Hi, I'm Dus+n

  3. Who am I? • Python Packaging Working Group (Packaging-WG) member

    • Python Packaging Authority (PyPA) member • Python Package Index (PyPI) maintainer/contributor/admin • Developer Advocate @ Google @di_codes
  4. {{ talk.title }} {{ talk.subtitle }} @di_codes

  5. Python Packaging Ge#ng the Code You Wrote To the People

    That Want It Using the Same Language You Wrote It In @di_codes
  6. None
  7. Python Packaging In Just Five Easy Steps! @di_codes

  8. None
  9. Python Packaging In Just FOUR Easy Steps! @di_codes

  10. None
  11. Python Packaging It's Rela*vely Painless Now Go Ahead and Use

    It @di_codes
  12. None
  13. Python Packaging We're S'll Trying To Make It Be5er @di_codes

  14. None
  15. Python Packaging Let's Just Throw It All Away And Start

    Over From Scratch @di_codes
  16. None
  17. Python Packaging Let Me Just Get You Up To Speed

    On Everything That's Changed Since Last Time @di_codes
  18. None
  19. Python Packaging There's A Lot Of Stuff Here You Might

    Not Need All Of It @di_codes
  20. None
  21. Python Packaging In the Simplest Terms Possible For Anyone That

    Cares @di_codes
  22. None
  23. Python Packaging So Easy A Caveman Could Do It @di_codes

  24. None
  25. Hello I Am a PyPI Maintainer At the Very Least

    I Should Be Able To Tell You How To Use PyPI @di_codes
  26. None
  27. Python Packaging @di_codes

  28. Python Packaging But Indiana Jones-themed @di_codes

  29. Python Packaging But Indiana Jones-themed (Also In French) @di_codes

  30. None
  31. Links • h#ps:/ /www.youtube.com/watch?v=qOH-h-EKKac • h#ps:/ /www.youtube.com/watch?v=5BqAeN-F9Qs • h#ps:/ /www.youtube.com/watch?v=gc9dkktg1gU

    • h#ps:/ /www.youtube.com/watch?v=W8A2bOKPtJU • h#ps:/ /www.youtube.com/watch?v=eLPiPHr6TVI @di_codes
  32. The End @di_codes

  33. Inside The Cheeseshop How Python Packaging Works @di_codes

  34. None
  35. None
  36. h"ps:/ /xkcd.com/1987/

  37. None
  38. None
  39. totally_awesome_library.py @di_codes

  40. Problem: How do I get this to users? @di_codes

  41. Solu%on: Email? @di_codes

  42. Solu%on: Personal Website? @di_codes

  43. None
  44. Problem: How do I find Python code? @di_codes

  45. Solu%on: The Vaults of Parnassus @di_codes

  46. None
  47. Problem: How do I build this? @di_codes

  48. Solu%on: distutils @di_codes

  49. $ python setup.py <...> @di_codes

  50. $ python setup.py build @di_codes

  51. Solu%on: Source Distribu-ons @di_codes

  52. AKA: sdist @di_codes

  53. AKA: ssssssssssssssdist @di_codes

  54. $ python setup.py sdist @di_codes

  55. Problem: Building takes too long @di_codes

  56. Solu%on: Built Distribu+ons @di_codes

  57. AKA: bdist @di_codes

  58. $ python setup.py bdist @di_codes

  59. Problem: How can I "do" packaging? @di_codes

  60. Solu%on: Use pla(orm packaging? @di_codes

  61. Sub-Problem: My pla'orm doesn't have packaging @di_codes

  62. Sub-Problem: My pla'orm does have packaging, but I want things

    now! @di_codes
  63. Solu%on: The Python Package Index @di_codes

  64. None
  65. AKA: "PyPI" @di_codes

  66. !"# @di_codes

  67. AKA: "The Cheeseshop" @di_codes

  68. None
  69. Problem: How can I specify dependencies? @di_codes

  70. Solu%on: setuptools @di_codes

  71. Problem: Installing is too hard! @di_codes

  72. Solu%on: easy_install @di_codes

  73. The Egg Distribu.on @di_codes

  74. None
  75. Problem: easy_install problems @di_codes

  76. Solu%on: pyinstall @di_codes

  77. Problem: "pyinstall" is too long @di_codes

  78. Solu%on: pip @di_codes

  79. eggs @di_codes

  80. Problem: Applica'on dependencies? @di_codes

  81. Solu%on: requirements.txt @di_codes

  82. pip install -r requirements.txt @di_codes

  83. Problem: Installing from PyPI is slow @di_codes

  84. Problem: Trus%ng 3rd Party Domains @di_codes

  85. Solu%on: PyPI begins hos-ng releases @di_codes

  86. None
  87. Problem: We need built distribu-ons (again) @di_codes

  88. Solu%on: The Wheel Distribu.on @di_codes

  89. None
  90. None
  91. Problem: python setup.py upload doesn't use HTTPS @di_codes

  92. Solu%on: twine @di_codes

  93. @di_codes

  94. Problem: PyPI is showing it's age @di_codes

  95. None
  96. None
  97. None
  98. None
  99. Solu%on: Rewrite PyPI from scratch @di_codes

  100. None
  101. AKA: Warehouse @di_codes

  102. None
  103. The End @di_codes

  104. Current Problems @di_codes

  105. Problem: Packaging is s*ll kinda hard @di_codes

  106. Solu%on: The Python Packaging Guide h"ps:/ /packaging.python.org @di_codes

  107. Solu%on: sampleproject h"ps:/ /github.com/pypa/sampleproject @di_codes

  108. Solu%on: General care & maintenance @di_codes

  109. Problem: Packaging is a li+le too easy @di_codes

  110. Python Packaging So Easy a Spammer Could Do It @di_codes

  111. Problem: Some%mes I need more than Python (some%mes I don't

    even have it) @di_codes
  112. Solu%on: conda @di_codes

  113. Problem: Reproducible Environments @di_codes

  114. Solu%on: pipfile/pipfile.lock h"ps:/ /github.com/pypa/pipfile @di_codes

  115. Problem: Arbitrary code in setup.py @di_codes

  116. None
  117. Problem: The distu*ls/setuptools dance @di_codes

  118. Solu%on: PEP 517/518 @di_codes

  119. None
  120. None
  121. pyproject.toml @di_codes

  122. How to help @di_codes

  123. None
  124. Packaging BoF! @di_codes

  125. How to get help @di_codes

  126. How to get help • h#ps:/ /packaging.python.org • Issue tracker

    for a given tool • On IRC: #pypa Freenode channel • h#ps:/ /github.com/pypa/packaging-problems @di_codes
  127. Summary @di_codes

  128. Thanks! • Github: @di • Twi,er: @di_codes • Email: di@python.org

    @di_codes