Simple Python Packaging Tips

Transcript

1. Building Open-Source Tools via GitHub VERA Talk November 13, 2019

   Adina Feinstein !1

2. Outline 1. Setting up a GitHub respository 2. Accessing your

   repository locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics !2

3. 1. Setting up a GitHub respository 2. Accessing your repository

   locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !3

4. 1. Setting up a GitHub repository !4

5. 1. Setting up a GitHub repository !4

6. 1. Setting up a GitHub repository !5

7. 1. Setting up a GitHub repository !5

8. 1. Setting up a GitHub respository 2. Accessing your repository

   locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !6

9. 2. Accessing your repository locally There are two ways to

   get your repository on your machine: 1. You can use git clone $ git clone username@host:/path/to/repository !7

10. 2a. Using the Git commands !8

11. 2a. Using the Git commands !8

12. 2a. Using the Git commands !8

13. The shortcuts you need to know 1. Proposing changes to

    given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9

14. The shortcuts you need to know 1. Proposing changes to

    given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9

15. The shortcuts you need to know 1. Proposing changes to

    given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9

16. The shortcuts you need to know 1. Proposing changes to

    given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9

17. The shortcuts you need to know 1. Proposing changes to

    given file(s) $ git add <filename> 2. Committing your changes to the repository $ git commit -m “Commit message” 3. Sending your changes to your remote repository $ git push origin master 4. Checking if there are any updates to the code (should always be done before making changes or else you might have to deal with merge conflicts!) $ git pull 2a. Using the Git commands: Basics !9

18. If you’re updating software that other people use, it is

    recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10

19. If you’re updating software that other people use, it is

    recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10

20. If you’re updating software that other people use, it is

    recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10

21. If you’re updating software that other people use, it is

    recommended to make changes in a new branch and then merge branches. 1. Creating a new branch $ git checkout -b <branch-name> 2. Pushing the branch to GitHub $ git push origin 3. Switching to a branch that was created on GitHub $ git checkout <branch-name> 2a. Using the Git commands: Branches !10

22. There are two ways to get your repository on your

    machine: 2. You can download the GitHub desktop app (for macOS & Windows) https://desktop.github.com/ 2. Accessing your repository locally !11

23. 2b. Cloning from the GitHub app !12

24. 2b. Cloning from the GitHub app !12

25. 2b. Cloning from the GitHub app !12

26. 2b. Cloning from the GitHub app !12

27. 2b. Cloning from the GitHub app !12

28. 2b. Using the Git app: Basics !13

29. 2b. Using the Git app: Basics “git add” !13

30. 2b. Using the Git app: Basics “git add” “git commit”

    !13

31. 2b. Using the Git app: Basics “git add” “git commit”

    !13

32. 2b. Using the Git app: Basics “git add” “git commit”

    “git push” !13

33. 2b. Using the Git app: Branches !14

34. 2b. Using the Git app: Branches !14

35. 2b. Using the Git app: Branches !14

36. 2b. Using the Git app: Branches !14

37. 2b. Using the Git app: Branches !14

38. 1. Setting up a GitHub respository 2. Accessing your repository

    locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !15

39. !16 3. Creating basic documentation

40. !16 3. Creating basic documentation

41. !16 3. Creating basic documentation README.md is written in Markdown

    (kind of similar to HTML)

42. !17 3. Creating basic documentation

43. !17 3. Creating basic documentation You can create fancier docs

    with Read the Docs!

44. 1. Setting up a GitHub respository 2. Accessing your repository

    locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !18

45. !19 4. Using Read the Docs Read the Docs &

    Sphinx are documentation generating softwares that make it easy to keep your documentation up-to-date. In order to use the software, you must create a Read the Docs account and link it to your GitHub page: https://readthedocs.org/dashboard/ After importing your repository you will get a https://xx.readthedocs.io website.

46. !20 4. Using Read the Docs In order to use

    Sphinx to then build your documentation, you need a directory in your GitHub repository called docs and within docs you simply run the command $ sphinx-quickstart To see what your documentation looks like, you can run $ make html

47. !21 4a. Proper documentation format Sphinx reads in a very

    specific format for your documentation.

48. !21 4a. Proper documentation format Sphinx reads in a very

    specific format for your documentation. class Source(object): ”””A single source observed by TESS. Parameters ---------- tic : int, optional The TIC ID of the source. gaia : int, optional The GAIA DR2 source_id. coords : tuple, astropy.coordinates.SkyCoord The (RA, Dec) coords of the obj. Attributes ---------- tess_mag : float The TESS magnitude from the TIC. “”” def __init__(self, tic=None, gaia=None, coords=None): self.tic = tic self.gaia = gaia self.coords = coords

49. !21 4a. Proper documentation format Sphinx reads in a very

    specific format for your documentation. class Source(object): ”””A single source observed by TESS. Parameters ---------- tic : int, optional The TIC ID of the source. gaia : int, optional The GAIA DR2 source_id. coords : tuple, astropy.coordinates.SkyCoord The (RA, Dec) coords of the obj. Attributes ---------- tess_mag : float The TESS magnitude from the TIC. “”” def __init__(self, tic=None, gaia=None, coords=None): self.tic = tic self.gaia = gaia self.coords = coords )

50. !22 4b. Setting up tabs: API Your api.rst file allows

    your comments to be built to to the website. These should include all of the classes in your package.

51. !23 4b. Setting up tabs: Homepage Your index.rst file will

    serve as the homepage to your website.

52. !23 4b. Setting up tabs: Homepage Your index.rst file will

    serve as the homepage to your website.

53. !23 4b. Setting up tabs: Homepage Your index.rst file will

    serve as the homepage to your website.

54. !23 4b. Setting up tabs: Homepage Your index.rst file will

    serve as the homepage to your website.

55. !23 4b. Setting up tabs: Homepage Your index.rst file will

    serve as the homepage to your website.

56. !24 4b. Setting up tabs: getting_started package/docs/getting_started will host additional

    documentation and tutorial Jupyter notebooks. Each tab is a different .rst or .ipynb file.

57. !25 4b. Setting up tabs: Useful formatting guide 1. Creating

    headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces

58. !25 4b. Setting up tabs: Useful formatting guide 1. Creating

    headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces

59. !25 4b. Setting up tabs: Useful formatting guide 1. Creating

    headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces

60. !25 4b. Setting up tabs: Useful formatting guide 1. Creating

    headers ============= 2. Creating subheaders ------------------------ 3. Bullet points that are highlighted blue * <bullet-header> 4. Tabbing for inline code tab + 5 spaces

61. !26 4b. Setting up tabs: Index Your index.rst file will

    also be where you build your index.

62. !27 4c. Travis CI

63. !27 4c. Travis CI You can also set up automatic

    tests which run with every GitHub push!

64. !28 4c. Travis CI Travis CI depends on test files

    that are located in /<package-name>/<package-name>/tests You should have tests for every class in your package. With every GitHub push, Travis will build package and run all of your tests. The progress can be tracked on https://travis-ci.org You have to make an account with Travis CI and link it to your GitHub.

65. !29 4d. Linking docs to a personal website Your docs

    should be copied to a new branch called something like gh-pages

66. !29 4d. Linking docs to a personal website Your docs

    should be copied to a new branch called something like gh-pages

67. !29 4d. Linking docs to a personal website

68. !29 4d. Linking docs to a personal website

69. !29 4d. Linking docs to a personal website

70. 1. Setting up a GitHub respository 2. Accessing your repository

    locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !30

71. !31 5. PyPI: Making packages easy to install You know

    when you want to download a new package and you do pip install <package-name>? This is how to make your code do that! PyPI = Python Package Index https://pypi.org You have to register for an account to get a PyPI package name.

72. !31 5. PyPI: Making packages easy to install You know

    when you want to download a new package and you do pip install <package-name>? This is how to make your code do that! PyPI = Python Package Index https://pypi.org You have to register for an account to get a PyPI package name.

73. !31 5. PyPI: Making packages easy to install You know

    when you want to download a new package and you do pip install <package-name>? This is how to make your code do that! PyPI = Python Package Index https://pypi.org You have to register for an account to get a PyPI package name.

74. !32 5a. Creating your setup.py file

75. !32 5a. Creating your setup.py file The name of the

    directory where your packaged files are

76. !32 5a. Creating your setup.py file The name of the

    directory where your packaged files are A description for the PyPI website

77. !32 5a. Creating your setup.py file

78. !33 5a. Creating your setup.py file

79. !33 5a. Creating your setup.py file The packages you’re creating

    with your setup file

80. !33 5a. Creating your setup.py file The packages you’re creating

    with your setup file Packages yours is dependent on and any specific versions needed

81. !34 5b. Versions Your package directory should always have a

    version.py file. This is all that’s in that file: __version__ = "0.0.1" It’s good practice to do a pre-release before every version update. Pre-releases can be pip installed by specifying the version number. Then, once you test everything you want, you can upload the new version for real. Pre-releases are specified by __version__ = “0.0.1rc1" Version numbers have to be unique and always increasing!

82. !35 5c. Uploading to PyPI You need to install twine

    to upload first python -m pip install —user —upgrade twine To setup, you’ll need to login to your PyPI account python twine —repository-url https://test.pypi.org/legacy/ dist/* From there, every update is simple: python setup.py sdist python setup.py sdist —upload

83. !35 5c. Uploading to PyPI You need to install twine

    to upload first python -m pip install —user —upgrade twine To setup, you’ll need to login to your PyPI account python twine —repository-url https://test.pypi.org/legacy/ dist/* From there, every update is simple: python setup.py sdist python setup.py sdist —upload

84. !35 5c. Uploading to PyPI You need to install twine

    to upload first python -m pip install —user —upgrade twine To setup, you’ll need to login to your PyPI account python twine —repository-url https://test.pypi.org/legacy/ dist/* From there, every update is simple: python setup.py sdist python setup.py sdist —upload

85. 1. Setting up a GitHub respository 2. Accessing your repository

    locally 3. Creating basic documentation 4. Creating Build the Docs 5. Using PyPI 6. Package basics Outline !36

86. !37 6a. Files every package should have version.py This will

    keep track of the version you’re on. __version__ = “0.0.1” This will need to be manually updated before each PyPI update.

87. !38 6a. Files every package should have __init__.py This will

    load in all of your classes to be easily called import os PACKAGEDIR = os.path.abspath(os.path.dirname(__file__)) from .version import __version__ from .targetdata import * from .source import *

88. !38 6a. Files every package should have __init__.py This will

    load in all of your classes to be easily called import os PACKAGEDIR = os.path.abspath(os.path.dirname(__file__)) from .version import __version__ from .targetdata import * from .source import *

89. !39 6a. Files every package should have from .source import

    *

90. !39 6a. Files every package should have from .source import

    * Imports package path

91. !39 6a. Files every package should have from .source import

    * Imports package path Imports classes/functions from other files in <package>

92. !39 6a. Files every package should have from .source import

    * Imports package path Imports classes/functions from other files in <package> Identifies classes/ functions to import in *