Continuous Integration Via Hudson - PICC11

Talk given at http://www.picconf.org/ in 2011.

Continuous Integration via Hudson
Nick Silkey

The introduction of continuous integration (CI) yields a net benefit to developer workflows but also greatly benefits operations processes. This talk centers around Yale’s adoption of tenets of CI via the stand-up of tools which compliment this streamined workflow from an operations perspective. In addition to notable benefits on the J2EE side of the infrastructure, these CI tools and practices have been extended to other stacks such as the central Drupal service namely via Hudson. This is Yale’s contribution to the many praises of Hudson as a developer and operations framework for getting things done quickly and accurately.

Transcript

1. CONTINUOUS INTEGRATION VIA HUDSON Nick Silkey Yale University @filler
2. Old-school build => release Devs freeze + build + ‘punt’ release Ops shuffles release + bops services No yelling? ‘Success’ Yelling? Revert + ‘punt’ error data Wash, rinse, repeat
3. Continuous Integration per Wikipedia In software engineering, continuous integration (CI) implements continuous processes of applying quality control - small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.
4. Hudson selling points Continuous Integration tools can help Improve developer workflows Improve operations workflows (!) Hudson*did for us *: s/Hudson/Jenkins/
5. Hudson selling points Hudson builds on schedule or on demand Hudson builds consistently or with (programmed) variance
6. Hudson selling points Hudson can install your code during its build +1 for ops Hudson leaves build audit trails
7. Hudson selling points Hudson can enable non-technical people to ‘build’ via a click (theres an API also) Hudson provides granular authorization per ‘project’ (devs get n privs in dev env ; n-1 privs in prd env) Hudson integrates easily into existing authentication realm
8. CI yields change beyond code Automation => Repeatability => Less Risk Helps overcome culture of fear deploying to prod Shrinking barriers to getting dev’s code upstream => devs can produce “more features, faster” Fast devs need fast ops Hudson can enable automation for ops to become faster
9. Doing Drupal via Hudson Initially automated ops build tasks with Apache Ant Yay. These deploys were documented, but manual/interactive Exposed build accounts, privs, etc. To humans. Yuck.
10. Doing Drupal via Hudson Hudson cleaned deploy process up for all Hudson enabled on-demand builds (watch SCM -> deploy to infrastructure) Hudson enabled the non-technical to consume (wrapper for documented-yet-cryptic interactive Ant)
11. Doing Drupal via Hudson Hudson automated tasks, such as code moves (bzr push branches into svn repos @ schedule, on demand) Also code management (user/team branches in svn stood up the same every time) Also data management (db migrate prod -> dev; very federate-able)
12. Apache Ant Ant can automate dev or ops work ‘make’, in Java, via imperative XML Enumerate targets; chain targets into dependencies Build predictively, programmatically, repeatably Used in J2EE a lot, we used for Drupal deploy framework This allowed us to automate ops workflows
13. Apache Maven Other J2EE projects at Yale wrap Maven around Ant Maven allows those close to code to express dependencies Gets ops out of business of ‘upgrade pkg x’ or ‘install pkg y’ Example: build may need Ant but ops doesnt have to bother, Maven fetches Ant via dependency chain and makes ready Maven also makes SCM resource calls, builds, tests and pluggable to boot
14. jFrog Artifactory Yale joined Artifactory to Maven Allows for integration with non-Yale deps programmatically (fetch+cache external resources) Allows Yale build artifacts from Maven to persist Hudson build => Maven built pkg pushed to Artifactory Hudson install => Maven install pkg from above Artifactory
15. Yale uses Hudson more and more now Hudson => Maven => Artifactory => Hudson => Apps Hudson => bzr/svn => Hudson => Apps Bedework, institutional event calendaring Central Authentication Service (CAS), institutional SSO Drupal, institutional CMS for pub web Improv, institutional account/resource mgmt stack Shibboleth, institutional SP-initiated SAML 2.0 stack
16. Typical Yale Java dev use Bang out something release worthy Hudson uses Maven against VCS to build EAR, pushes to Artifactory Deploy version from Artifactory to dev env via Hudson
17. Typical Yale Java ops use Use version of artifact listed by devs to run build as param Installs artifact (built by automation + tested) into test Test/QA/etc Repeat on release schedule to push artifact to production
18. Typical Yale Drupal dev use Work with local LAMP/MAMP/XAMP + bzr branch/co Something worth sharing? Merge/ci to dev branch Use pre-conf Hudson web UI to fire off a build of code to dev Freeze dev branch changes when complete, hail Ops NB: Some non-ITS devs have branches in svn Hudson watches these branches every 5m Change detected? Deploy to their dev infrastructure
19. Typical Yale Drupal ops use Use Hudson to merge bzr dev branch upstream Use Hudson to deploy merged code in upstream branch to env Done on release schedule based on customer needs/reqs
20. Takeaways Its a pervasive technology. Get it going with one dev group who wants it Tthers will want in. Even if devs dont want in, ops can use for automation/standards Hudson install + config + maintenance is so easy, which is refreshing given its power No matter what you ‘do’ (Java, PHP, Ruby, C/C++), someone out there has used Hudson to do something similar Use contrib plugins! Mgmt can pay Cloudbees for Nectar/support contract ($ fuels open-source progress)