Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Simple Python Packaging Tips

Adina
November 13, 2019

Simple Python Packaging Tips

Adina

November 13, 2019
Tweet

More Decks by Adina

Other Decks in Programming

Transcript

  1. Building Open-Source
    Tools via GitHub
    VERA Talk

    November 13, 2019

    Adina Feinstein
    !1

    View full-size slide

  2. Outline
    1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    !2

    View full-size slide

  3. 1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    Outline
    !3

    View full-size slide

  4. 1. Setting up a GitHub repository
    !4

    View full-size slide

  5. 1. Setting up a GitHub repository
    !4

    View full-size slide

  6. 1. Setting up a GitHub repository
    !5

    View full-size slide

  7. 1. Setting up a GitHub repository
    !5

    View full-size slide

  8. 1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    Outline
    !6

    View full-size slide

  9. 2. Accessing your repository locally
    There are two ways to get your repository on your machine:

    1. You can use git clone
    $ git clone username@host:/path/to/repository
    !7

    View full-size slide

  10. 2a. Using the Git commands
    !8

    View full-size slide

  11. 2a. Using the Git commands
    !8

    View full-size slide

  12. 2a. Using the Git commands
    !8

    View full-size slide

  13. The shortcuts you need to know

    1. Proposing changes to given file(s)

    $ git add
    2. Committing your changes to the repository
    $ git commit -m “Commit message”
    3. Sending your changes to your remote repository
    $ git push origin master
    4. Checking if there are any updates to the code (should always be done before making
    changes or else you might have to deal with merge conflicts!)
    $ git pull
    2a. Using the Git commands: Basics
    !9

    View full-size slide

  14. The shortcuts you need to know

    1. Proposing changes to given file(s)

    $ git add
    2. Committing your changes to the repository
    $ git commit -m “Commit message”
    3. Sending your changes to your remote repository
    $ git push origin master
    4. Checking if there are any updates to the code (should always be done before making
    changes or else you might have to deal with merge conflicts!)
    $ git pull
    2a. Using the Git commands: Basics
    !9

    View full-size slide

  15. The shortcuts you need to know

    1. Proposing changes to given file(s)

    $ git add
    2. Committing your changes to the repository
    $ git commit -m “Commit message”
    3. Sending your changes to your remote repository
    $ git push origin master
    4. Checking if there are any updates to the code (should always be done before making
    changes or else you might have to deal with merge conflicts!)
    $ git pull
    2a. Using the Git commands: Basics
    !9

    View full-size slide

  16. The shortcuts you need to know

    1. Proposing changes to given file(s)

    $ git add
    2. Committing your changes to the repository
    $ git commit -m “Commit message”
    3. Sending your changes to your remote repository
    $ git push origin master
    4. Checking if there are any updates to the code (should always be done before making
    changes or else you might have to deal with merge conflicts!)
    $ git pull
    2a. Using the Git commands: Basics
    !9

    View full-size slide

  17. The shortcuts you need to know

    1. Proposing changes to given file(s)

    $ git add
    2. Committing your changes to the repository
    $ git commit -m “Commit message”
    3. Sending your changes to your remote repository
    $ git push origin master
    4. Checking if there are any updates to the code (should always be done before making
    changes or else you might have to deal with merge conflicts!)
    $ git pull
    2a. Using the Git commands: Basics
    !9

    View full-size slide

  18. If you’re updating software that other people use, it is recommended
    to make changes in a new branch and then merge branches.

    1. Creating a new branch

    $ git checkout -b
    2. Pushing the branch to GitHub
    $ git push origin
    3. Switching to a branch that was created on GitHub
    $ git checkout
    2a. Using the Git commands: Branches
    !10

    View full-size slide

  19. If you’re updating software that other people use, it is recommended
    to make changes in a new branch and then merge branches.

    1. Creating a new branch

    $ git checkout -b
    2. Pushing the branch to GitHub
    $ git push origin
    3. Switching to a branch that was created on GitHub
    $ git checkout
    2a. Using the Git commands: Branches
    !10

    View full-size slide

  20. If you’re updating software that other people use, it is recommended
    to make changes in a new branch and then merge branches.

    1. Creating a new branch

    $ git checkout -b
    2. Pushing the branch to GitHub
    $ git push origin
    3. Switching to a branch that was created on GitHub
    $ git checkout
    2a. Using the Git commands: Branches
    !10

    View full-size slide

  21. If you’re updating software that other people use, it is recommended
    to make changes in a new branch and then merge branches.

    1. Creating a new branch

    $ git checkout -b
    2. Pushing the branch to GitHub
    $ git push origin
    3. Switching to a branch that was created on GitHub
    $ git checkout
    2a. Using the Git commands: Branches
    !10

    View full-size slide

  22. There are two ways to get your repository on your machine:

    2. You can download the GitHub desktop app (for macOS &
    Windows)

    https://desktop.github.com/
    2. Accessing your repository locally
    !11

    View full-size slide

  23. 2b. Cloning from the GitHub app
    !12

    View full-size slide

  24. 2b. Cloning from the GitHub app
    !12

    View full-size slide

  25. 2b. Cloning from the GitHub app
    !12

    View full-size slide

  26. 2b. Cloning from the GitHub app
    !12

    View full-size slide

  27. 2b. Cloning from the GitHub app
    !12

    View full-size slide

  28. 2b. Using the Git app: Basics
    !13

    View full-size slide

  29. 2b. Using the Git app: Basics
    “git add”
    !13

    View full-size slide

  30. 2b. Using the Git app: Basics
    “git add”
    “git commit”
    !13

    View full-size slide

  31. 2b. Using the Git app: Basics
    “git add”
    “git commit”
    !13

    View full-size slide

  32. 2b. Using the Git app: Basics
    “git add”
    “git commit”
    “git push”
    !13

    View full-size slide

  33. 2b. Using the Git app: Branches
    !14

    View full-size slide

  34. 2b. Using the Git app: Branches
    !14

    View full-size slide

  35. 2b. Using the Git app: Branches
    !14

    View full-size slide

  36. 2b. Using the Git app: Branches
    !14

    View full-size slide

  37. 2b. Using the Git app: Branches
    !14

    View full-size slide

  38. 1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    Outline
    !15

    View full-size slide

  39. !16
    3. Creating basic documentation

    View full-size slide

  40. !16
    3. Creating basic documentation

    View full-size slide

  41. !16
    3. Creating basic documentation
    README.md is written in Markdown (kind of similar to HTML)

    View full-size slide

  42. !17
    3. Creating basic documentation

    View full-size slide

  43. !17
    3. Creating basic documentation
    You can create fancier docs

    with Read the Docs!

    View full-size slide

  44. 1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    Outline
    !18

    View full-size slide

  45. !19
    4. Using Read the Docs
    Read the Docs & Sphinx are documentation generating softwares
    that make it easy to keep your documentation up-to-date.

    In order to use the software, you must create a Read the Docs
    account and link it to your GitHub page:

    https://readthedocs.org/dashboard/

    After importing your repository you will get a

    https://xx.readthedocs.io

    website.

    View full-size slide

  46. !20
    4. Using Read the Docs
    In order to use Sphinx to then build your documentation, you need a
    directory in your GitHub repository called docs and within docs you
    simply run the command



    $ sphinx-quickstart
    To see what your documentation looks like, you can run

    $ make html

    View full-size slide

  47. !21
    4a. Proper documentation format
    Sphinx reads in a very specific format for your documentation.

    View full-size slide

  48. !21
    4a. Proper documentation format
    Sphinx reads in a very specific format for your documentation.
    class Source(object):
    ”””A single source observed by TESS.
    Parameters
    ----------
    tic : int, optional
    The TIC ID of the source.
    gaia : int, optional
    The GAIA DR2 source_id.
    coords : tuple, astropy.coordinates.SkyCoord
    The (RA, Dec) coords of the obj.
    Attributes
    ----------
    tess_mag : float
    The TESS magnitude from the TIC.
    “””
    def __init__(self, tic=None, gaia=None, coords=None):
    self.tic = tic
    self.gaia = gaia
    self.coords = coords

    View full-size slide

  49. !21
    4a. Proper documentation format
    Sphinx reads in a very specific format for your documentation.
    class Source(object):
    ”””A single source observed by TESS.
    Parameters
    ----------
    tic : int, optional
    The TIC ID of the source.
    gaia : int, optional
    The GAIA DR2 source_id.
    coords : tuple, astropy.coordinates.SkyCoord
    The (RA, Dec) coords of the obj.
    Attributes
    ----------
    tess_mag : float
    The TESS magnitude from the TIC.
    “””
    def __init__(self, tic=None, gaia=None, coords=None):
    self.tic = tic
    self.gaia = gaia
    self.coords = coords
    )

    View full-size slide

  50. !22
    4b. Setting up tabs: API
    Your api.rst file allows your comments to be built to to the website.
    These should include all of
    the classes in your
    package.

    View full-size slide

  51. !23
    4b. Setting up tabs: Homepage
    Your index.rst file will serve as the homepage to your website.

    View full-size slide

  52. !23
    4b. Setting up tabs: Homepage
    Your index.rst file will serve as the homepage to your website.

    View full-size slide

  53. !23
    4b. Setting up tabs: Homepage
    Your index.rst file will serve as the homepage to your website.

    View full-size slide

  54. !23
    4b. Setting up tabs: Homepage
    Your index.rst file will serve as the homepage to your website.

    View full-size slide

  55. !23
    4b. Setting up tabs: Homepage
    Your index.rst file will serve as the homepage to your website.

    View full-size slide

  56. !24
    4b. Setting up tabs: getting_started
    package/docs/getting_started will host additional documentation and
    tutorial Jupyter notebooks. Each tab is a different .rst or .ipynb file.

    View full-size slide

  57. !25
    4b. Setting up tabs: Useful formatting guide
    1. Creating headers
    =============
    2. Creating subheaders

    ------------------------
    3. Bullet points that are highlighted blue

    *
    4. Tabbing for inline code

    tab + 5 spaces

    View full-size slide

  58. !25
    4b. Setting up tabs: Useful formatting guide
    1. Creating headers
    =============
    2. Creating subheaders

    ------------------------
    3. Bullet points that are highlighted blue

    *
    4. Tabbing for inline code

    tab + 5 spaces

    View full-size slide

  59. !25
    4b. Setting up tabs: Useful formatting guide
    1. Creating headers
    =============
    2. Creating subheaders

    ------------------------
    3. Bullet points that are highlighted blue

    *
    4. Tabbing for inline code

    tab + 5 spaces

    View full-size slide

  60. !25
    4b. Setting up tabs: Useful formatting guide
    1. Creating headers
    =============
    2. Creating subheaders

    ------------------------
    3. Bullet points that are highlighted blue

    *
    4. Tabbing for inline code

    tab + 5 spaces

    View full-size slide

  61. !26
    4b. Setting up tabs: Index
    Your index.rst file will also be where you build your index.

    View full-size slide

  62. !27
    4c. Travis CI

    View full-size slide

  63. !27
    4c. Travis CI
    You can also set up
    automatic tests which run
    with every GitHub push!

    View full-size slide

  64. !28
    4c. Travis CI
    Travis CI depends on test files that are located in

    ///tests
    You should have tests for every class in your package.

    With every GitHub push, Travis will build package and run all of your
    tests. The progress can be tracked on



    https://travis-ci.org

    You have to make an account with Travis CI and link it to your GitHub.

    View full-size slide

  65. !29
    4d. Linking docs to a personal website
    Your docs should be copied to a new branch called something like

    gh-pages

    View full-size slide

  66. !29
    4d. Linking docs to a personal website
    Your docs should be copied to a new branch called something like

    gh-pages

    View full-size slide

  67. !29
    4d. Linking docs to a personal website

    View full-size slide

  68. !29
    4d. Linking docs to a personal website

    View full-size slide

  69. !29
    4d. Linking docs to a personal website

    View full-size slide

  70. 1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    Outline
    !30

    View full-size slide

  71. !31
    5. PyPI: Making packages easy to install
    You know when you want to download a new package and you do

    pip install ?
    This is how to make your code do that!

    PyPI = Python Package Index



    https://pypi.org

    You have to register for an account to get a PyPI package name.

    View full-size slide

  72. !31
    5. PyPI: Making packages easy to install
    You know when you want to download a new package and you do

    pip install ?
    This is how to make your code do that!

    PyPI = Python Package Index



    https://pypi.org

    You have to register for an account to get a PyPI package name.

    View full-size slide

  73. !31
    5. PyPI: Making packages easy to install
    You know when you want to download a new package and you do

    pip install ?
    This is how to make your code do that!

    PyPI = Python Package Index



    https://pypi.org

    You have to register for an account to get a PyPI package name.

    View full-size slide

  74. !32
    5a. Creating your setup.py file

    View full-size slide

  75. !32
    5a. Creating your setup.py file
    The name of the directory
    where your packaged
    files are

    View full-size slide

  76. !32
    5a. Creating your setup.py file
    The name of the directory
    where your packaged
    files are
    A description for the PyPI
    website

    View full-size slide

  77. !32
    5a. Creating your setup.py file

    View full-size slide

  78. !33
    5a. Creating your setup.py file

    View full-size slide

  79. !33
    5a. Creating your setup.py file
    The packages you’re
    creating with your setup
    file

    View full-size slide

  80. !33
    5a. Creating your setup.py file
    The packages you’re
    creating with your setup
    file
    Packages yours is
    dependent on and any
    specific versions needed

    View full-size slide

  81. !34
    5b. Versions
    Your package directory should always have a version.py file. This
    is all that’s in that file:



    __version__ = "0.0.1"
    It’s good practice to do a pre-release before every version update.
    Pre-releases can be pip installed by specifying the version
    number. Then, once you test everything you want, you can upload
    the new version for real. Pre-releases are specified by

    __version__ = “0.0.1rc1"
    Version numbers have to be unique and always increasing!

    View full-size slide

  82. !35
    5c. Uploading to PyPI
    You need to install twine to upload first

    python -m pip install —user —upgrade twine
    To setup, you’ll need to login to your PyPI account

    python twine —repository-url https://test.pypi.org/legacy/ dist/*
    From there, every update is simple:

    python setup.py sdist
    python setup.py sdist —upload

    View full-size slide

  83. !35
    5c. Uploading to PyPI
    You need to install twine to upload first

    python -m pip install —user —upgrade twine
    To setup, you’ll need to login to your PyPI account

    python twine —repository-url https://test.pypi.org/legacy/ dist/*
    From there, every update is simple:

    python setup.py sdist
    python setup.py sdist —upload

    View full-size slide

  84. !35
    5c. Uploading to PyPI
    You need to install twine to upload first

    python -m pip install —user —upgrade twine
    To setup, you’ll need to login to your PyPI account

    python twine —repository-url https://test.pypi.org/legacy/ dist/*
    From there, every update is simple:

    python setup.py sdist
    python setup.py sdist —upload

    View full-size slide

  85. 1. Setting up a GitHub respository

    2. Accessing your repository locally

    3. Creating basic documentation

    4. Creating Build the Docs

    5. Using PyPI

    6. Package basics
    Outline
    !36

    View full-size slide

  86. !37
    6a. Files every package should have
    version.py
    This will keep track of the version you’re on.

    __version__ = “0.0.1”
    This will need to be manually updated before each PyPI update.

    View full-size slide

  87. !38
    6a. Files every package should have
    __init__.py
    This will load in all of your classes to be easily called

    import os
    PACKAGEDIR = os.path.abspath(os.path.dirname(__file__))
    from .version import __version__
    from .targetdata import *
    from .source import *

    View full-size slide

  88. !38
    6a. Files every package should have
    __init__.py
    This will load in all of your classes to be easily called

    import os
    PACKAGEDIR = os.path.abspath(os.path.dirname(__file__))
    from .version import __version__
    from .targetdata import *
    from .source import *

    View full-size slide

  89. !39
    6a. Files every package should have
    from .source import *

    View full-size slide

  90. !39
    6a. Files every package should have
    from .source import *
    Imports package

    path

    View full-size slide

  91. !39
    6a. Files every package should have
    from .source import *
    Imports package

    path
    Imports classes/functions from
    other files in

    View full-size slide

  92. !39
    6a. Files every package should have
    from .source import *
    Imports package

    path
    Imports classes/functions from
    other files in
    Identifies classes/
    functions to import

    in *

    View full-size slide