Linters and Travis and Badges (oh my)

Slides from a GRPUG talk about Continuous Integration on Open Source Python projects.

A template implementing the practices in this presentation:

https://github.com/jacebrowning/template-python

A live demonstration of the above template:

https://github.com/jacebrowning/template-python-demo

A couple examples that use this template and demonstrate badges:

https://github.com/jacebrowning/curmit
https://github.com/jacebrowning/doorstop

Transcript

1. Linters and Travis and Badges (oh my) Continuous Integration on Open Source Python @JaceBrowning
2. Outline • Definitions • “Integration” • “Continuous” • Build Status • Integrating • Compiling • Analyzing • Testing • Publishing • Putting it all Together • Questions • Demonstation
3. Continuous / Integration
4. Integration • What are we integrating? • “branches” • “merges”
 • Version Control Systems (VCS)
5. Early 2nd Generation VCS • Examples: CVS, PVCS, Dimensions • Branches: implicit • Merging: harder • Integration (in practice): • single person manually merging files • locking to prevent concurrent merge efforts
6. Late 2nd Generation VCS • Examples: Subversion • Branches: explicit and implicit • Merging: sometimes harder • Integration (in practice): • development off the same trunk • merging before each commit, or locking
7. 3rd Generation VCS • Examples: Git, Mercurial • Branches: explicit • Merging: easier • Integration (in practice): • all “topics” in explicit branches • servers periodically merge branches in the background • when a topic is complete, branches are explicitly merged
8. Continuous • 2nd Generation VCS • XP = “at least daily” • 3rd Generation VCS • as fast as the server will allow
9. Build Status • Continuous Integration = “merging branches as soon as they are ready to be merged” • When are they ready? • Code review • Passed all build phases: integrating, compiling, analyzing, testing, publishing
10. Integrating • VCS Hosing
 
 
 
 
 
 
 
 
 
 Hosting Cost OSS GitHub Cloud Free Public Repos Paid Private Repos No Bitbucket Cloud Free Public Repos
 Free/Paid Private Repos No GitLab Cloud
 Local Free Yes Stash Local Paid No
11. Integrating • CI Service
 
 
 
 
 
 
 
 
 
 Hosting Config Cost OSS Travis
 Magnum
 Circle Cloud File Free for Public No Jenkins Local Web Free Yes TeamCity Bamboo Local Web Paid Yes Buildbot Local File Free Yes
12. Compiling • Depends on language and project • For Python, this is simply: • package “compilation” • dependency installation • package installation
13. Analyzing • Static build verification • Some Python options:
 
 
 
 
 
 
 
 Style Documentation Warnings Conventions pep8 x pep257 x pyflakes x flake8 x x pylint x x x x
14. Testing • Dynamic build verification
 • Some Python options: • Frameworks: unittest, py.test • Runners: nose, py.test, coverage
15. Publishing • Release • Documentation / Websites • GitHub pages • ReadTheDocs • Personal website • Badges • Build Status: (most CI services) • Coverage: Coveralls • Code Quality: Scrutinizer, Code Climate
16. Putting it all Together • Automate all the build, analysis, and test steps
 • Make them runnable on all developers machines
 • Run the same commands on the CI server
17. Putting it all Together • My current Python CI choices: • Integration: GitHub and Travis • Compilation: install into a virtual environment • Analysis: pep8, pep257, pylint • Testing: py.test and coverage • Publication: GitHub pages, Coveralls, Scrutinizer
18. Questions? (and then a demo)
19. jacebrowning/template-python jacebrowning/template-python-demo jacebrowning/curmit jacebrowning/doorstop