Continuous Integration

Overview of Continuous Integration principles and why to incorporate CI on your team. Presented at DevOps Day Nairobi 2014

Transcript

1. Continuous Integration Adam Nelson adam@kili.io @varud
2. What is it? • Code, library, and package modifications happen continuously • Modifications from different developers and upstream sources integrated quickly (i.e. no long-lived branches)
3. Continuous Deployment • Continuous Deployment is the evolution of continuous integration into actually deploying modifications continuously • We won’t discuss that today
4. Version Control • Git - Created by Linus Torvalds to allow for distributed development where each developer has the entire changeset on her machine (~175MB for complete Linux history) • Mercurial (Hg)- Similar to git and written in Python • Please use one of these ^^^ (just use git) • Outdated version control systems: cvs, subversion, Visual Source Safe
5. Time spent in 'merge hell' Time between developers’ branches
6. Integrate Early and Integrate Often
7. But Johnny Developer’s commit just broke my environment!!!
8. Unit Tests • Fast (seconds, not minutes) • No external IO (i.e. database interaction should be 'mocked') • Atomic - unit tests on one class should not overflow into another (that would be an integration test)
9. Code Linting • Fast (seconds not minutes) • Static analysis of code requires no interpretation • Style guide can be enforced on a per-commit basis (pep8)
10. Pre-commit Hook • Developer attempts commit • Unit tests run • Linter runs • Failure blocks commit • Johnny Developer’s code much less likely to break your working environment
11. How big should a commit be? • Big enough to get the task done • Not bigger … if the task can be broken down into discrete units, it should be • And each unit should have a unit test
12. Bugs • Bugs first - Don’t start new features and chores until all bugs are fixed • Regressions - When you fix a bug, test it so if it is re-introduced your tests will find it • If you can’t justify fixing a bug quickly, move on because it’s not important. Don’t maintain a long list of bugs.
13. Integration Tests • If you want to test how data flows from your cache system to your database system, unit tests won’t cut it • Integration tests can actually test how data moves through a test system
14. Build Server • On successful commit, a webhook triggers all tests (including integration) to run on a build server • If the integration tests fail, an email is generated so developers know that the build is broken
15. Build Server • Jenkins CI (http://jenkins-ci.org/) - Handles builds for any type of project • Travis CI (http://travis-ci.com/) - Handles builds for Github projects
16. Continuous Deployment Ok I lied … just a little taste
17. Feature Flipping • Get your code live and stressed a bit before turning on the new feature • Separate code deployment from feature release • Warm up a feature by deploying to 10% (or even release internally first like Facebook does). • Django Waffle works well for this
18. Monitoring • If you find out about bugs from your users or customer service, you’re probably not monitoring enough • Investigate all exceptions • Log all exceptions • Design robust warning, info and debug logging so that you can increase and decrease verbosity as necessary • Aggregate logs so they can be found and correlated (http://papertrailapp.com)
19. Performance • Did that change you just deployed increase load time 50% on a certain page? • http://newrelic.com/
20. Learn More • Come to the Nairobi Python Meetup (http:// www.meetup.com/Python-Nairobi) • Etsy Devops blog: (http://codeascraft.com) • Devops Cafe: (http://devopscafe.org)
21. Adam Nelson adam@kili.io @varud