Inside The Cheeseshop: How Python Packaging Works (PyGotham 2018)

Inside The Cheeseshop: How Python Packaging Works (PyGotham 2018)

Questions and confusion about the Python packaging ecosystem abound. What is this setup.py file? What’s the difference between wheels and eggs? Do I use setuptools or distutils? Why should I use twine? Do I put my projects dependencies in a requirements.txt or in setup.py? How do I just get my module up on PyPI? Wait, what is Warehouse?

This talk will identify the key tools one might encounter when trying to distribute Python software, what they are used for, why they exist, and their history (including where their weird names come from). In addition, we’ll see how they all work together, what it takes to make them work, and what the future has in store for Python packaging.

8db101b3a812b0464c53e5409c12ea94?s=128

Dustin Ingram

October 06, 2018
Tweet

Transcript

  1. PyGotham 2018 @di_codes

  2. Hi, I'm Dus+n

  3. Who am I? • Python Packaging Working Group (Packaging-WG) member

    • Python Packaging Authority (PyPA) member • Python Package Index (PyPI) maintainer/contributor/admin • Developer Advocate @ Google @di_codes
  4. {{ talk.title }} {{ talk.subtitle }} @di_codes

  5. Python Packaging Ge#ng the Code You Wrote To the People

    That Want It Using the Same Language You Wrote It In @di_codes
  6. None
  7. Python Packaging In Just Five Easy Steps! @di_codes

  8. None
  9. Python Packaging In Just FOUR Easy Steps! @di_codes

  10. None
  11. Python Packaging It's Rela*vely Painless Now Go Ahead and Use

    It @di_codes
  12. None
  13. Python Packaging We're S'll Trying To Make It Be5er @di_codes

  14. None
  15. Python Packaging Let's Just Throw It All Away And Start

    Over From Scratch @di_codes
  16. None
  17. Python Packaging Let Me Just Get You Up To Speed

    On Everything That's Changed Since Last Time @di_codes
  18. None
  19. Python Packaging There's A Lot Of Stuff Here You Might

    Not Need All Of It @di_codes
  20. None
  21. Python Packaging In the Simplest Terms Possible For Anyone That

    Cares @di_codes
  22. None
  23. Python Packaging So Easy A Caveman Could Do It @di_codes

  24. None
  25. Hello I Am a PyPI Maintainer At the Very Least

    I Should Be Able To Tell You How To Use PyPI @di_codes
  26. None
  27. Python Packaging @di_codes

  28. Python Packaging But Indiana Jones-themed @di_codes

  29. Python Packaging But Indiana Jones-themed (Also In French) @di_codes

  30. None
  31. Links • h#ps:/ /www.youtube.com/watch?v=qOH-h-EKKac • h#ps:/ /www.youtube.com/watch?v=5BqAeN-F9Qs • h#ps:/ /www.youtube.com/watch?v=gc9dkktg1gU

    • h#ps:/ /www.youtube.com/watch?v=W8A2bOKPtJU • h#ps:/ /www.youtube.com/watch?v=eLPiPHr6TVI • h#ps:/ /www.youtube.com/watch?v=iLVNWfPWAC8 @di_codes
  32. The End @di_codes

  33. Inside The Cheeseshop How Python Packaging Works @di_codes

  34. None
  35. None
  36. h"ps:/ /xkcd.com/1987/

  37. None
  38. None
  39. totally_awesome_library.py @di_codes

  40. Problem: How do I get this to users? @di_codes

  41. Solu%on: Email? @di_codes

  42. Solu%on: Personal Website? @di_codes

  43. None
  44. Problem: How do I find Python code? @di_codes

  45. Solu%on: The Vaults of Parnassus @di_codes

  46. None
  47. Problem: How do I build this? @di_codes

  48. Solu%on: distutils @di_codes

  49. $ python setup.py <...> @di_codes

  50. $ python setup.py build @di_codes

  51. Solu%on: Source Distribu-ons @di_codes

  52. AKA: sdist @di_codes

  53. AKA: ssssssssssssssdist @di_codes

  54. $ python setup.py sdist @di_codes

  55. Problem: Building takes too long @di_codes

  56. Solu%on: Built Distribu+ons @di_codes

  57. AKA: bdist @di_codes

  58. $ python setup.py bdist @di_codes

  59. Problem: How can I "do" packaging? @di_codes

  60. Solu%on: Use pla(orm packaging? @di_codes

  61. @di_codes

  62. Sub-Problem: My pla'orm doesn't have packaging @di_codes

  63. Sub-Problem: My pla'orm does have packaging, but I want things

    now! @di_codes
  64. Solu%on: The Python Package Index @di_codes

  65. None
  66. AKA: "PyPI" @di_codes

  67. None
  68. None
  69. None
  70. AKA: "The Cheeseshop" @di_codes

  71. None
  72. Problem: How can I specify dependencies? @di_codes

  73. Solu%on: setuptools @di_codes

  74. Problem: Installing is too hard! @di_codes

  75. Solu%on: easy_install @di_codes

  76. The Egg Distribu.on @di_codes

  77. None
  78. Problem: easy_install problems @di_codes

  79. Solu%on: pyinstall @di_codes

  80. Problem: "pyinstall" is too long @di_codes

  81. Solu%on: pip @di_codes

  82. eggs @di_codes

  83. Problem: Applica'on dependencies? @di_codes

  84. Solu%on: requirements.txt @di_codes

  85. pip install -r requirements.txt @di_codes

  86. Problem: Installing from PyPI is slow @di_codes

  87. Problem: Trus%ng 3rd Party Domains @di_codes

  88. Solu%on: PyPI begins hos-ng releases @di_codes

  89. None
  90. Problem: We need built distribu-ons (again) @di_codes

  91. Solu%on: The Wheel Distribu.on @di_codes

  92. None
  93. None
  94. Problem: python setup.py upload doesn't use HTTPS @di_codes

  95. Solu%on: twine @di_codes

  96. @di_codes

  97. Problem: PyPI is showing it's age @di_codes

  98. None
  99. None
  100. None
  101. None
  102. Solu%on: Rewrite PyPI from scratch @di_codes

  103. None
  104. AKA: Warehouse @di_codes

  105. None
  106. The End @di_codes

  107. Current Problems @di_codes

  108. Problem: Packaging is s*ll kinda hard @di_codes

  109. Solu%on: The Python Packaging Guide h"ps:/ /packaging.python.org @di_codes

  110. Solu%on: sampleproject h"ps:/ /github.com/pypa/sampleproject @di_codes

  111. Solu%on: General care & maintenance @di_codes

  112. Problem: Packaging is a li+le too easy @di_codes

  113. Python Packaging So Easy a Spammer Could Do It @di_codes

  114. Problem: Some%mes I need more than Python (some%mes I don't

    even have it) @di_codes
  115. Solu%on: conda @di_codes

  116. Problem: Reproducible Environments @di_codes

  117. Solu%on: pipfile/pipfile.lock h"ps:/ /github.com/pypa/pipfile @di_codes

  118. Problem: Arbitrary code in setup.py @di_codes

  119. Problem: The distu*ls/setuptools dance @di_codes

  120. Solu%on: PEP 517/518 @di_codes

  121. None
  122. None
  123. pyproject.toml @di_codes

  124. How to help @di_codes

  125. None
  126. Bloomberg Packaging Sprints October 27 & 28 in NYC &

    LON @di_codes
  127. None
  128. How to get help @di_codes

  129. How to get help • @PyPI, @ThePyPA • h,ps:/ /packaging.python.org

    • Issue tracker for a given tool • On IRC: #pypa Freenode channel • h,ps:/ /github.com/pypa/packaging-problems @di_codes
  130. Summary @di_codes

  131. Thanks! • Github: @di • Twi,er: @di_codes • Email: di@python.org

    @di_codes