Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Developing a CI/CD Pipeline with GitHub

Developing a CI/CD Pipeline with GitHub

GitHub integrates with a wide spectrum of third party tools, giving you considerable flexibility when setting up a deployment pipeline. So where do you begin?

In this tutorial, we will give a demonstration how one uses these integrations in practice. You will learn how to implement CI and CD incrementally, so you can start building, or iterate on, you organization's pipeline. We will also cover more advanced techniques, to give you a broader sense of what is possible. Throughout, we will touch on the foundation for all these integrations: the GitHub API. You will gain an appreciation for how existing third party tools use the API, and learn how to use it to implement new tools of your own. In addition, we will provide an overview of GitHub's own software deivery processes, and how we have evolved them over the years.

With that tooling in place, we will then review how best to use it. That will include identifying common mistakes, and how to avoid them. We will also outline the challenges that arise when creating depolyment pipelines for large, complex projects.

Patrick McKenna

May 27, 2015
Tweet

More Decks by Patrick McKenna

Other Decks in Technology

Transcript

  1. the best way to build and ship software ! CD

    Definition 2 automated software delivery process covers build, deploy, test, release " • know that you can get (machine-level) feedback w/o asking anyone for it
  2. the best way to build and ship software ! CD

    - Why 3 •reduces cycle time •lower stress •admits granular releases •improves feedback at all stages •increases time for creative work •measures failure or success consistently
  3. the best way to build and ship software CD -

    What 4 # $ ! FAST RELIABLE (incl. repeatable) • caveat: process varies greatly by application
  4. the best way to build and ship software ! 5

    % & " BUILD UNIT TESTS ACCEPTANCE TESTS MANUAL TESTS RELEASE Deployment Pipeline each commit triggers new instance of pipeline (which typically won't reach final stage) 1. system works at technical level 2. system works at functional level, meets specs 3. exploratory, integration envs, UAT 4. deliver system to users
  5. the best way to build and ship software ! 6

    ' ( DELIVERY DEPLOYMENT manual automated Continuous [ ]
  6. the best way to build and ship software ! Production-Like

    7 ) * + LIVE CUSTOMER TRAFFIC or possibility of it SAME OS, SOFTWARE no dev tools MANAGED LIKE PROD provisioning, &c.
  7. the best way to build and ship software ! Role

    of GitHub Platform 8 record of how they went source for deploys , - • Integrations, API enable this • GitHub = Central, highly visible
  8. the best way to build and ship software ! CI

    Goals 9 . / 0 CATCH BUGS EARLY REDUCE MERGE CONFLICTS REGULARLY PROVE APP WORKING • primary goal: provide fast, useful feedback on most common failures • bugs easier to fix when fresh in mind • merge conflicts detested, and take unpredictable amount of time to resolve • nb: proving app works for given config, env • devs integrate ops’ perspective
  9. the best way to build and ship software ! Continuous

    Integration 10 •build/test every push •display results prominently process, not tool • tests means to an end, not end in and of themselves • highest benefit/cost ratio of any step in deploy pipeline
  10. the best way to build and ship software ! Human

    Response 11 feedback useful iff fast 1
  11. the best way to build and ship software ! Implementing

    CI 12 cultural requirements •agreed failure threshold •test locally first •never base work on broken build •revert if needed •share code ownership *always exceptions, e.g. tests taking too long locally • local tests = quickly id any problems (esp. common when `master` updated since proj started); keeps `master` working while you fix what you'd have to fix anyways • prioritize fixing breaking changes • revert especially if fix taking longer than some threshold amount of time • anyone can refactor “others’" code
  12. the best way to build and ship software ! 13

    2 3 4 EVOLVE MODULAR LOGICAL STRUCTURE NOT ENV- SPECIFIC Designing Tests • as dev progresses, supplement w/ a few more involved checks for common failures • prioritize commit stage scripts as you would source code
  13. the best way to build and ship software ! Designing

    Tests 14 • fail fast • not always at first error •broader failure criteria • architecture violation • slow test • increased warnings • style breach
  14. the best way to build and ship software Continuous Deployment

    15 ! delays in execution of (and thus feedback from) testing/ops after dev "done" is often a major bottleneck
  15. the best way to build and ship software ! Deploy

    Steps 16 deploy desired version of app into this env create, config infrastructure on which app will run 5 6 1. includes hardware, networking, middleware, external services 2. app should be built already
  16. the best way to build and ship software ! Deploy

    Steps 17 7 8 9 configure app run deploy tests 1. includes req’d data, state 2. make sure app is up and running, external services working; give basic diagnostics if app won't run
  17. the best way to build and ship software ! Infrastructure:

    Principles, Goals 18 •same process for managing all environments •declarative, •idempotent •automated •autonomic CAN YOU DEPLOY FROM SCRATCH ON VANILLA SEVERS? • changes to configs can be more dangerous than changes to source code — latter typically better guarded against (compiler, tests)
  18. the best way to build and ship software ! Automation

    > Documentation 19 • reduces risk • automated = auditable • enforces consistency • simpler error diagnosis • simpler to recreate
  19. the best way to build and ship software ! Build/Deploy

    Automation 20 •build once • rebuild inefficient • introduce new errors •same deploy process for all envs •in effect testing “deploy to prod” process •eliminate “works on my machine” •encourages all to test more
  20. the best way to build and ship software ! 22

    9 USE WEBHOOKS SET UP PROXY FOR API CALLS (if you must) Say No to Agressive Polling every single dep’t in GitHub calls this public enemy #1