Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Continuous Integration before and after Chef

4ca6230b05046f1a809835985b1f8bc2?s=47 Ignasi Barrera
September 02, 2014

Continuous Integration before and after Chef

How we built our Abiquo continuous integration and continuous deployment pipelines to scale using jclouds and Chef.

4ca6230b05046f1a809835985b1f8bc2?s=128

Ignasi Barrera

September 02, 2014
Tweet

Transcript

  1. CI before and after Chef Chef Users London Meetup

  2. Disclaimer

  3. Xavier Fernandez CTO and proud founder at Abiquo Ignasi Barrera

    Senior Engineer at Abiquo Apache jclouds committer & PMC @IgnasiBarrera @xfernandez
  4. • About abiquo • Why CI is important in abiquo

    • Infrastructure software needs • Our CI/CD pipelines • jclouds-chef • Lessons learned Agenda
  5. abiquo - anycloud.abiquo.com • Founded 2006 • Venture Capital Funded

    – $24.5m • 25 employees, with 16 in Engineering ◦ 7 in UK ◦ 18 in Spain • Locations ◦ Headquartered in Heathrow, UK ◦ R&D in Barcelona, Catalonia, Spain • + 3000 customers around the world
  6. Why CI is important in abiquo • We introduced Agile

    Programming (something like scrum) from the beginning. • TDD and other test techniques are good to ensure the quality of a new feature but… CI helps preserving quality over time
  7. Nexenta NetApp Generic NFS Generic iSCSI Infrastructure software LVM Monolithic

    Distributed Multi-datacenter ESX Hyper-V KVM XenServer OracleVM Amazon Rackspace HP Google DigitalOcean ElasticHosts CloudSigma DEPLOYMENTS HYPERVISORS PUBLIC CLOUDS STORAGE
  8. Nexenta NetApp Generic NFS Generic iSCSI Infrastructure software LVM Monolithic

    Distributed Multi-datacenter ESX Hyper-V KVM XenServer OracleVM Amazon Rackspace HP Google DigitalOcean ElasticHosts CloudSigma DEPLOYMENTS HYPERVISORS PUBLIC CLOUDS STORAGE HOW MANY COMBINATIONS?
  9. • Too many infrastructure combinations to have static staging /

    testing environments • Too many resources required • Continuous evolution of third party products to be aligned with the evolution of your product Infrastructure software
  10. CI/CD pipelines needs • Customizable • Scalable • Extensible •

    User friendly • End-to-end Jenkins provides everything we need
  11. Continuous integration Provision Jenkins Slaves Commit Integration tests Build

  12. Continuous deployment CI Create OS packages & build repo Provision

    staging environment Functional + UI tests ISO
  13. Configurable pipelines

  14. jclouds-chef • Chef is not only for ruby fans! •

    Complete Chef API for Java • Open Source & Enterprise Chef • Chef Solo • knife bootstrap
  15. Bootstrap example ChefContext ctx = ContextBuilder.newBuilder("chef") .endpoint(“http://abiquo-chef.bcn.abiquo.com”) .credentials("provisioner", readFile (“provisioner.pem”))

    .buildView(ChefContext.class); ChefService chef = ctx.getChefService(); BootstrapConfig bootstrapConfig = BootstrapConfig.builder() .runList(new RunListBuilder().addRole("abiquo-nightly").build()); .attributes(new JsonBall(readFile (“attributes.json”)))); .environment(“Staging”); .build(); chef.updateBootstrapConfigForGroup(“staging”, bootstrapConfig); Statement bootstrap = chef.createBootstrapScriptForGroup(“staging”); compute.runScriptOnNodesMatching(inGroup (“staging”), bootstrap); attributes.json { "set_fqdn": "{{hostname}}", "abiquo": { "license": "{{license}}", "yum": { "nightly-repo": "http://staging.bcn.abiquo.com/{{version}}/rpm" } } } Dynamic run lists Dynamic environments Dynamic attributes Reusable & persistent configuration
  16. Lessons learned • Infrastructure as code • Test everything •

    Use the right tools • Think big
  17. Thank you! Q&A