Simple Python Packaging Tips

Transcript

1. Building Open-Source Tools via GitHub VERA Talk November 13, 2019 Adina Feinstein !1
2. Outline 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics !2
3. 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !3
4. 1. Setting up a GitHub repository !4
5. 1. Setting up a GitHub repository !4
6. 1. Setting up a GitHub repository !5
7. 1. Setting up a GitHub repository !5
8. 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !6
9. 2. Accessing your repository locally There are two ways to get your repository on your machine: 1. You can use git clone $ git clone username@host:/path/to/repository !7
10. 2a. Using the Git commands !8
11. 2a. Using the Git commands !8
12. 2a. Using the Git commands !8
13. The shortcuts you need to know 1. Proposing changes to given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9
14. The shortcuts you need to know 1. Proposing changes to given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9
15. The shortcuts you need to know 1. Proposing changes to given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9
16. The shortcuts you need to know 1. Proposing changes to given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9
17. The shortcuts you need to know 1. Proposing changes to given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9
18. If you’re updating software that other people use, it is recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10
19. If you’re updating software that other people use, it is recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10
20. If you’re updating software that other people use, it is recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10
21. If you’re updating software that other people use, it is recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10
22. There are two ways to get your repository on your machine: 2. You can download the GitHub desktop app (for macOS & Windows) https://desktop.github.com/ 2. Accessing your repository locally !11
23. 2b. Cloning from the GitHub app !12
24. 2b. Cloning from the GitHub app !12
25. 2b. Cloning from the GitHub app !12
26. 2b. Cloning from the GitHub app !12
27. 2b. Cloning from the GitHub app !12
28. 2b. Using the Git app: Basics !13
29. 2b. Using the Git app: Basics “git add” !13
30. 2b. Using the Git app: Basics “git add” “git commit” !13
31. 2b. Using the Git app: Basics “git add” “git commit” !13
32. 2b. Using the Git app: Basics “git add” “git commit” “git push” !13
33. 2b. Using the Git app: Branches !14
34. 2b. Using the Git app: Branches !14
35. 2b. Using the Git app: Branches !14
36. 2b. Using the Git app: Branches !14
37. 2b. Using the Git app: Branches !14
38. 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !15
39. !16 3. Creating basic documentation
40. !16 3. Creating basic documentation
41. !16 3. Creating basic documentation README.md is written in Markdown (kind of similar to HTML)
42. !17 3. Creating basic documentation
43. !17 3. Creating basic documentation You can create fancier docs with Read the Docs!
44. 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !18
45. !19 4. Using Read the Docs Read the Docs & Sphinx are documentation generating softwares that make it easy to keep your documentation up-to-date. In order to use the software, you must create a Read the Docs account and link it to your GitHub page: https://readthedocs.org/dashboard/ After importing your repository you will get a https://xx.readthedocs.io website.
46. !20 4. Using Read the Docs In order to use Sphinx to then build your documentation, you need a directory in your GitHub repository called docs and within docs you simply run the command $ sphinx-quickstart To see what your documentation looks like, you can run $ make html
47. !21 4a. Proper documentation format Sphinx reads in a very specific format for your documentation.
48. !21 4a. Proper documentation format Sphinx reads in a very specific format for your documentation. class Source(object): ”””A single source observed by TESS. Parameters ---------- tic : int, optional The TIC ID of the source. gaia : int, optional The GAIA DR2 source_id. coords : tuple, astropy.coordinates.SkyCoord The (RA, Dec) coords of the obj. Attributes ---------- tess_mag : float The TESS magnitude from the TIC. “”” def __init__(self, tic=None, gaia=None, coords=None): self.tic = tic self.gaia = gaia self.coords = coords
49. !21 4a. Proper documentation format Sphinx reads in a very specific format for your documentation. class Source(object): ”””A single source observed by TESS. Parameters ---------- tic : int, optional The TIC ID of the source. gaia : int, optional The GAIA DR2 source_id. coords : tuple, astropy.coordinates.SkyCoord The (RA, Dec) coords of the obj. Attributes ---------- tess_mag : float The TESS magnitude from the TIC. “”” def __init__(self, tic=None, gaia=None, coords=None): self.tic = tic self.gaia = gaia self.coords = coords )
50. !22 4b. Setting up tabs: API Your api.rst file allows your comments to be built to to the website. These should include all of the classes in your package.
51. !23 4b. Setting up tabs: Homepage Your index.rst file will serve as the homepage to your website.
52. !23 4b. Setting up tabs: Homepage Your index.rst file will serve as the homepage to your website.
53. !23 4b. Setting up tabs: Homepage Your index.rst file will serve as the homepage to your website.
54. !23 4b. Setting up tabs: Homepage Your index.rst file will serve as the homepage to your website.
55. !23 4b. Setting up tabs: Homepage Your index.rst file will serve as the homepage to your website.
56. !24 4b. Setting up tabs: getting_started package/docs/getting_started will host additional documentation and tutorial Jupyter notebooks. Each tab is a different .rst or .ipynb file.
57. !25 4b. Setting up tabs: Useful formatting guide 1. Creating headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces
58. !25 4b. Setting up tabs: Useful formatting guide 1. Creating headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces
59. !25 4b. Setting up tabs: Useful formatting guide 1. Creating headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces
60. !25 4b. Setting up tabs: Useful formatting guide 1. Creating headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces
61. !26 4b. Setting up tabs: Index Your index.rst file will also be where you build your index.
62. !27 4c. Travis CI
63. !27 4c. Travis CI You can also set up automatic tests which run with every GitHub push!
64. !28 4c. Travis CI Travis CI depends on test files that are located in /<package-name>/<package-name>/tests You should have tests for every class in your package. With every GitHub push, Travis will build package and run all of your tests. The progress can be tracked on https://travis-ci.org You have to make an account with Travis CI and link it to your GitHub.
65. !29 4d. Linking docs to a personal website Your docs should be copied to a new branch called something like gh-pages
66. !29 4d. Linking docs to a personal website Your docs should be copied to a new branch called something like gh-pages
67. !29 4d. Linking docs to a personal website
68. !29 4d. Linking docs to a personal website
69. !29 4d. Linking docs to a personal website
70. 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !30
71. !31 5. PyPI: Making packages easy to install You know when you want to download a new package and you do pip install <package-name>? This is how to make your code do that! PyPI = Python Package Index https://pypi.org You have to register for an account to get a PyPI package name.
72. !31 5. PyPI: Making packages easy to install You know when you want to download a new package and you do pip install <package-name>? This is how to make your code do that! PyPI = Python Package Index https://pypi.org You have to register for an account to get a PyPI package name.
73. !31 5. PyPI: Making packages easy to install You know when you want to download a new package and you do pip install <package-name>? This is how to make your code do that! PyPI = Python Package Index https://pypi.org You have to register for an account to get a PyPI package name.
74. !32 5a. Creating your setup.py file
75. !32 5a. Creating your setup.py file The name of the directory where your packaged files are
76. !32 5a. Creating your setup.py file The name of the directory where your packaged files are A description for the PyPI website
77. !32 5a. Creating your setup.py file
78. !33 5a. Creating your setup.py file
79. !33 5a. Creating your setup.py file The packages you’re creating with your setup file
80. !33 5a. Creating your setup.py file The packages you’re creating with your setup file Packages yours is dependent on and any specific versions needed
81. !34 5b. Versions Your package directory should always have a version.py file. This is all that’s in that file: __version__ = "0.0.1" It’s good practice to do a pre-release before every version update. Pre-releases can be pip installed by specifying the version number. Then, once you test everything you want, you can upload the new version for real. Pre-releases are specified by __version__ = “0.0.1rc1" Version numbers have to be unique and always increasing!
82. !35 5c. Uploading to PyPI You need to install twine to upload first python -m pip install —user —upgrade twine To setup, you’ll need to login to your PyPI account python twine —repository-url https://test.pypi.org/legacy/ dist/* From there, every update is simple: python setup.py sdist python setup.py sdist —upload
83. !35 5c. Uploading to PyPI You need to install twine to upload first python -m pip install —user —upgrade twine To setup, you’ll need to login to your PyPI account python twine —repository-url https://test.pypi.org/legacy/ dist/* From there, every update is simple: python setup.py sdist python setup.py sdist —upload
84. !35 5c. Uploading to PyPI You need to install twine to upload first python -m pip install —user —upgrade twine To setup, you’ll need to login to your PyPI account python twine —repository-url https://test.pypi.org/legacy/ dist/* From there, every update is simple: python setup.py sdist python setup.py sdist —upload
85. 1. Setting up a GitHub respository 2. Accessing your repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !36
86. !37 6a. Files every package should have version.py This will keep track of the version you’re on. __version__ = “0.0.1” This will need to be manually updated before each PyPI update.
87. !38 6a. Files every package should have __init__.py This will load in all of your classes to be easily called import os PACKAGEDIR = os.path.abspath(os.path.dirname(__file__)) from .version import __version__ from .targetdata import * from .source import *
88. !38 6a. Files every package should have __init__.py This will load in all of your classes to be easily called import os PACKAGEDIR = os.path.abspath(os.path.dirname(__file__)) from .version import __version__ from .targetdata import * from .source import *
89. !39 6a. Files every package should have from .source import *
90. !39 6a. Files every package should have from .source import * Imports package path
91. !39 6a. Files every package should have from .source import * Imports package path Imports classes/functions from other files in <package>
92. !39 6a. Files every package should have from .source import * Imports package path Imports classes/functions from other files in <package> Identifies classes/ functions to import in *