Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud CI

maniotis
November 22, 2013

Cloud CI

maniotis

November 22, 2013
Tweet

Other Decks in Technology

Transcript

  1. Gates? (1) QA department gates for customers (2) CI system

    gates code for developers (3) … What to gate on is important
  2. Gerrit and Code Review • Human • Robot • Combined:

    - Developers can see the tests that automated feedback is based on, and can request more or provide feedback about coverage requirements.
  3. A feel for this system • Entry point: Jenkins -

    What's the actual error/env/context? • All code change committed: Git - The integration system itself has a chef repository • All repositories & code: Gerrit • Bugs in the bug tracker (hookup Jenkins and Gerrit) • Communication: IRC/Flowdock/Zulip/email/...
  4. CI Programming at a High Level • Use Jenkins Job

    Builder: http://ci.openstack.org/jenkins-job-builder - Yaml templates - Source control - Autogeneration - Too many plug-ins to mention (over 50 installed)
  5. Defining Continuous Integration Continuous how? Continuous throughout events: (a) source

    code modification (committing, reviewing, merging, release branching) (b) development cycle (c) testing (d) building (e) … deploying/delivering? (f) even bug tracking …
  6. Development Environments • What are you trying to do? •

    What part of the stack? • How many HW resources do you have? • How much time do you have?
  7. CI's affect on Dev Envs • Deduplication • Single source

    of truth • … Develop development environments as an organization
  8. Special Cases for CI? • Is Cloud CI special? •

    Bootstrapping infrastructure • Building packages to upload, so another job continuously deploy them... and so on. Nope : • More complexity • More pieces to integrate
  9. Testing in layers • From unprovisioned bare metal → to

    provisioned bare metal • The OpenStack layer → Tempest • “Above” the OpenStack layer
  10. Test Coverage Example (1) If test #1 passes, we can

    get from bare metal knowing only MAC address and go from IMPI → usable resource (2) If test #2 passes, we can get from usable resources to configured, managed resources. (3) If test #3 passes, we can get from cloud configured to run OpenStack → cloud running OpenStack services deemed as working.
  11. Hardware Resources for CI • 16 nodes (HW boxes) -

    virtualized deployments, virtualized or containerized (Docker) tests, and • 4 bare metal environments - targets for continuous deployment • (WIP) HA all services: gerrit, jenkins, etc • Environment upkeep
  12. "Wouldn't it be great if we had an Auto-Hans?” Taken

    from a demonstration I did at Cloudscaling
  13. CI Application Participation Are you a developer who doesn't have

    or use CI system because it doesn't apply to you or your organization?
  14. • “I'll write some tests before we release this product”

    • “I'll tell the developer of this code to review my patch set” • “Other developers test their code” • “You do development, I do testing, he does releasing, and she does management...” • “We all sit in the same room; of course we can keep track of what a few developers are doing and how they are doing it!”
  15. These all become more explicit... The CI system can be

    viewed as an explicit implementation of an implicit process. And because they're connected - “integrated” - a proper CI system will battle the siloing of your organization!
  16. Tips: Thinking in the CI Paradigm • “If you want

    to increase your success rate, double your failure rate.” Thomas John Watson (IBM CEO from 1914 – 1956) a.k.a “fail faster” • Define iteration cycle before speeding it up • “People will visit a Web site less often if it is slower than a close competitor by more than 250 milliseconds” - NYT interview Microsoft and Google researchers Are developers not users too? Provide some (useful) feedback ASAP! • Make sure every developer has a useful (to them) CI component (hint: ask them, collaborate)... Can increase everyones visibility! • All on the same page: Jenkins as the dashboard for your development team; expose pressing issues on the first page • Expose a component in the CI system before it is done • Optimize after – don't wait for a test to become speedy before gating on it
  17. Enforcement Suggestions • Dogfooding • Developer demos • Feed CI

    stats → managers, and managers enforce what the system/organization needs • If CI is designed correctly, then developers wil automatically want to use it! (^^ This is a test ^^ CI will tell you how much it is used :)
  18. Emergent Systems? A few, albeit abstract, concepts from biology: Wikipedia

    says: “... emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions” Is a CI system – when part of software development - made of these simple interactions? • Automation and Integration affect design patters in a complex stack. • Coherent design arises from complexity with help of interactions between the parts. • Let principles of emergent systems help you “intelligently design” things.