PuppetConf 2017: Beyond rspec - Innovative Strategies for Confident CI

Continuous Integration (CI) testing provides instant feedback as you develop Puppet code. While it’s great that your rspec-puppet and beaker tests pass using fixtures and stubs, you can get a false sense of security from unanticipated edge cases, unrealistic assumptions, or the lack of complete coverage. You may be tempted to address these limitations by writing even more tests, but that would only exacerbate your frustration. Instead, you could use some alternate techniques to make your CI pipeline predict real-world behavior more accurately, and thereby gain confidence in your test results. In this session, we will explore some less traditional yet more powerful techniques for automated testing of Puppet code. Real-time catalog compilation testing obtains more realistic results by simulating your production environment with the latest data within CI. Learn how GitHub scaled this strategy to test hundreds of unique combinations while keeping build times reasonable. Catalog difference testing with GitHub’s open source catalog difference tool (octocatalog-diff) increases development velocity and reduces deployment risk. The speaker, who is the original author of the tool, will explain why and how octocatalog-diff was developed and demonstrate how it can be used to streamline code reviews and validate Puppet upgrades. Real host testing of catalog compilation on actual nodes can exercise external dependencies and custom facts to provide the most accurate results. Overcome the limitations of human-defined fact fixtures and assumptions, avoiding situations that defy fixtures without you realizing it. Throughout the session, pick up other useful ideas, including how and why to run puppet-lint as a CI job, how to test project setup, and how Hubot keeps Dockerfiles up to date at GitHub. You will leave this session with fresh ideas to improve your CI pipeline, which will lead to confident and reliable deployments.

Transcript

1. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017 1
   Beyond rspec:
   Innovative Strategies for Confident CI
   Kevin Paulisse
   SRE @ GitHub
   

   View Slide

2. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Introduction: About GitHub
   2
   !
   

   View Slide

3. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Introduction: About GitHub
   3
   !
   

   View Slide

4. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Introduction: About Kevin Paulisse
   4
   "
   !
   Text
   @kpaulisse
   kpaulisse
   kpaulisse@gmail.com
   x
   #
   ↑ Work: GitHub ↓ Live: Madison, Wisc.
   

   View Slide

5. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Agenda
   5
   !
   1. Introduction
   

   2. Traditional Puppet Testing
   - Unit Tests
   - Integration Tests
   3. Less Traditional Puppet Testing
   - Tools
   - Techniques
   - Practical Applications
   4. Time for Questions
   

   View Slide

6. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Definition: What is CI?
   6
   !
   CI = Continuous Integration
   Verifying each code commit with

   one or more automated tests.
   

   View Slide

7. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Definition: What is CI?
   7
   !
   CI = Continuous Integration
   Verifying each code commit with

   one or more automated tests.
   CD = Continuous Delivery
   Continuous Deployment
   Producing software in short cycles

   so that software can be released

   at any time.
   

   View Slide

8. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Introduction: Puppet at GitHub
   8
   !
   • First Puppet Commit: September 8, 2008 (GitHub launched on April 10, 2008)
   • Puppet Versions: 0.24.x - 4.8.x
   • Lines of Code: 200,000+
   • Puppet Resources: 2,000+ per node
   

   View Slide

9. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
   Introduction: Puppet Culture
   9
   !
   https://www.youtube.com/watch?v=H7cQcoXVacU
   PuppetConf 2016: Scaling Puppet (and Puppet Culture)
   

   View Slide

10. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Introduction: GitHub Flow
    10
    !
    $
    Branch
    %
    Pull
    Request
    &
    Review
    '
    Test
    (CI)
    (
    Branch
    Deploy
    )
    Ship
    *
    Clone
    

    View Slide

11. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Introduction: Kubernetes at GitHub
    11
    !
    https://githubengineering.com/kubernetes-at-github/
    

    View Slide

12. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    CI is for Humans
    12
    !
    

    View Slide

13. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    CI is for Humans
    13
    !
    Good CI stops humans from shipping bad code.
    Great CI helps humans to ship good code.
    

    View Slide

14. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Puppet CI at GitHub
    14
    !
    • Run 15 CI jobs with each push
    

    View Slide

15. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    rspec-puppet
    15
    !
    https://github.com/rodjek/rspec-puppet
    

    View Slide

16. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    rspec-puppet
    16
    !
    puppet-distributed:
    rspec-puppet for classes and defines
    Distributed in parallel across 6 containers
    puppet-functions:
    rspec-puppet for custom functions
    puppet-ops-mon:
    Split out tests for our monitoring system hosts
    because those catalogs take 2+ minutes to compile
    puppet-windows:
    rspec-puppet tests for the one Windows system
    that we manage (being retired)
    

    View Slide

17. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Example: rspec-puppet
    17
    !
    

    View Slide

18. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Problems
    18
    !
    

    View Slide

19. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Problems
    19
    !
    

    View Slide

20. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Problems
    20
    !
    

    View Slide

21. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    The Fix ... Until Next Time
    21
    !
    

    View Slide

22. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    The Hard Truth
    22
    !
    

    View Slide

23. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octofacts
    23
    !
    Octofacts
    Automated, real fact fixtures for rspec-puppet
    Authors: @kpaulisse, @antonio
    License: MIT
    URL: https://github.com/github/octofacts
    

    View Slide

24. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octofacts in Action
    24
    !
    

    View Slide

25. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octofacts Fixture
    25
    !
    

    View Slide

26. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octofacts Updates
    26
    !
    

    View Slide

27. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octofacts Updates
    27
    !
    

    View Slide

28. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octofacts Conclusion
    28
    !
    Octofacts
    Automated, real fact fixtures for rspec-puppet
    Authors: @kpaulisse, @antonio
    License: MIT
    URL: https://github.com/github/octofacts
    

    View Slide

29. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Puppet CI at GitHub
    29
    !
    • Run 15 CI jobs with each push
    • "Traditional" CI jobs
    • rspec-puppet
    • integration
    

    View Slide

30. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Puppet CI at GitHub
    30
    !
    • Run 15 CI jobs with each push
    • "Traditional" CI jobs
    • rspec-puppet
    • integration
    

    View Slide

31. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Integration Tests Powered by Docker-Compose
    31
    !
    

    View Slide

32. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Integration Tests Powered by Docker-Compose
    32
    !
    

    View Slide

33. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Provisioning: Old Style
    33
    !
    Base Operating System
    One-Shot Provisioning
    Final System
    Run Puppet
    45+ Minutes
    

    View Slide

34. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Image Based Provisioning Workflow
    34
    !
    tar.gz
    Integration
    Test
    Docker
    Container
    Export
    Hiera
    Data
    Puppet
    Code
    Stub
    Data
    Provisioning
    Provisioning
    Physical
    Nodes
    EC2
    Nodes
    S3
    AMI
    * Extract
    * Snapshot
    * Build AMI
    

    View Slide

35. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Provisioning Methods Compared
    35
    !
    Base Operating System
    One-Shot Provisioning Image Based Provisioning
    Final System
    Run Puppet
    45+ Minutes
    Base Operating System
    Tested Image (AMI or S3 tar.gz)
    Integration Tests
    Final System
    Run Puppet
    5 Minutes
    CI
    

    View Slide

36. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Puppet CI at GitHub
    36
    !
    • Run 15 CI jobs with each push
    • "Traditional" CI jobs
    • rspec-puppet
    • integration
    • "Non-traditional" CI jobs
    • puppet-lint
    • puppet-catalogs
    • puppet-real-host-compile
    • puppet-utility
    

    View Slide

37. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-lint" CI Job
    37
    !
    Puppet Lint
    Check that your Puppet manifests conform
    to the style guide
    Author: Tim Sharpe
    License: MIT
    URL: https://github.com/rodjek/puppet-lint
    

    View Slide

38. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Example Output from "puppet-lint"
    38
    !
    

    View Slide

39. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-catalogs" CI Job
    39
    !
    PuppetDB
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Inventory
    Service
    Host
    Filter
    All Hosts Host
    Recent
    Facts
    

    View Slide

40. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-catalogs" CI Job
    40
    !
    PuppetDB
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Inventory
    Service
    Host
    Filter
    All Hosts Host
    Recent
    Facts
    

    View Slide

41. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Output from "puppet-catalogs"
    41
    !
    

    View Slide

42. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Limitations of "puppet-catalogs"
    42
    !
    1. Pulling facts from PuppetDB
    Adding, removing, or changing facts in your
    new code will not be reflected.
    

    View Slide

43. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Limitations of "puppet-catalogs"
    43
    !
    1. Pulling facts from PuppetDB
    Adding, removing, or changing facts in your
    new code will not be reflected.
    2. Successful compile != correct resources
    Just because the catalog compiles, that
    doesn't mean it's what you expect.
    

    View Slide

44. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Limitations of "puppet-catalogs"
    44
    !
    1. Pulling facts from PuppetDB
    Adding, removing, or changing facts in your
    new code will not be reflected.
    3. Successful compile != successful apply
    Compiling the catalog does not mean the
    catalog will apply correctly on the agents.
    2. Successful compile != correct resources
    Just because the catalog compiles, that
    doesn't mean it's what you expect.
    

    View Slide

45. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Summary of "puppet-catalogs"
    45
    !
    Strengths Weaknesses
    • Detects problems across multiple roles
    • Does not touch production nodes
    • Pass/fail output
    • Does not exercise actual facts
    • Does not verify catalog is as desired
    • Does not verify catalog will apply
    Sales Pitch
    Confirm via a pass-fail test that your changes didn't break critical roles
    

    View Slide

46. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-real-host-compile" CI Job
    46
    !
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Facter
    

    View Slide

47. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Limitations of "puppet-real-host-compile"
    47
    !
    1. Only covers one role
    We don't want our CI environment to be able
    to touch real servers.
    

    View Slide

48. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Limitations of "puppet-real-host-compile"
    48
    !
    1. Only covers one role
    We don't want our CI environment to be able
    to touch real servers.
    3. Successful compile != successful apply
    Compiling the catalog does not mean the
    catalog will apply correctly on the agents.
    2. Successful compile != correct resources
    Just because the catalog compiles, that
    doesn't mean it's what you expect.
    

    View Slide

49. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Summary of "puppet-real-host-compile"
    49
    !
    Strengths Weaknesses
    • Exercises custom facts
    • Does not touch production systems
    • Pass/fail output
    • Only covers one role
    • Does not verify catalog is as desired
    • Does not verify catalog will apply
    Sales Pitch
    Compile an actual Puppet catalog on an actual node with no shortcuts
    

    View Slide

50. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-utility" CI Job
    50
    !
    

    View Slide

51. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-utility": Hiera YAML Validation
    51
    !
    

    View Slide

52. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-utility": Hiera YAML Validation
    52
    !
    Test strategy:
    1. Parse YAML to hash
    2. Regenerate YAML
    3. Parse regenerated YAML
    4. Compare key counts
    

    View Slide

53. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    "puppet-utility": Project Setup
    53
    !
    

    View Slide

54. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    What Changed?
    54
    !
    

    View Slide

55. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff
    55
    !
    Octocatalog-Diff
    Compile and Compare Puppet Catalogs
    Author: @kpaulisse
    License: MIT
    URL: https://github.com/github/octocatalog-diff
    

    View Slide

56. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: History
    56
    !
    

    View Slide

57. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: History
    57
    !
    for host in $hosts ; do
    git checkout master
    puppet master --compile $host > /tmp/old.json
    git checkout my-changed-branch
    puppet master --compile $host > /tmp/new.json
    diff /tmp/old.json /tmp/new.json > /tmp/$host.diff
    done
    Caution: Over-simplified pseudo-code!
    

    View Slide

58. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff
    58
    !
    Recent
    Facts
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Master Branch
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Feature Branch
    PuppetDB ENC
    ENC Data
    

    View Slide

59. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff Command Line Usage
    59
    !
    Catalog compilation:
    • Check out "from" and "to" branches
    • Munge hiera config, ENC, etc.
    • Facts from PuppetDB
    • Build catalogs using Puppet
    Comparison analysis:
    • Resources added, removed, changed
    • Display human-readable output
    

    View Slide

60. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff Across the Fleet
    60
    !
    

    View Slide

61. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff and Code Reviews
    61
    !
    

    View Slide

62. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff and Code Reviews
    62
    !
    

    View Slide

63. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Optimizing Octocatalog-Diff
    63
    !
    

    View Slide

64. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Optimizing Octocatalog-Diff
    64
    !
    

    View Slide

65. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Load on PuppetDB
    65
    !
    CI Node CI Node CI Node CI Node CI Node CI Node CI Node CI Node
    PuppetDB
    PuppetDB
    PuppetDB
    catalog-diff #1
    catalog-diff #2
    catalog-diff #3
    ELB
    Old Catalog
    New Catalog
    Old Catalog
    New Catalog
    Queries:
    -Facts
    -Exported Resources
    postgresql
    

    View Slide

66. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Optimizing (or, Working Around) PuppetDB
    66
    !
    

    View Slide

67. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Optimizing (or, Working Around) PuppetDB
    67
    !
    Not recommended for production use
    

    View Slide

68. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff Limitation: Underlying Providers
    68
    !
    

    View Slide

69. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff Limitation: Underlying Providers
    69
    !
    

    View Slide

70. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff Limitation: Agents Apply Catalogs
    70
    !
    

    View Slide

71. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff Limitation: Fact Gathering
    71
    !
    Recent
    Facts
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Master Branch
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Feature Branch
    PuppetDB ENC
    ENC Data
    From Last Puppet Run
    (not from your branch)
    From Your Branch
    

    View Slide

72. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: What's an ENC?
    72
    !
    ENC:
    External Node Classifier
    A script that runs on the
    Puppet server and queries an
    external service. It gathers
    classification information
    and parameters for a host.
    

    View Slide

73. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating ENC Changes
    73
    !
    

    View Slide

74. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating ENC Changes
    74
    !
    

    View Slide

75. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating Fact Changes
    75
    !
    

    View Slide

76. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating Fact Changes
    76
    !
    

    View Slide

77. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating Fact Changes
    77
    !
    --to-fact-override 'gh_host_app=(nil)' --to-fact-override 'gh_host_role=(nil)'
    

    View Slide

78. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating Puppet Server Upgrade
    78
    !
    https://puppet.com/blog/upgrading-to-puppet-4-at-github
    

    View Slide

79. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating Puppet Agent Upgrade
    79
    !
    4.x Fact Files
    in S3
    Recent Facts
    Puppet 3.x
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Master Branch
    Puppet
    Agent
    Catalog
    (JSON)
    Hiera
    Data
    Puppet
    Code
    Master or Feature Branch
    PuppetDB ENC
    

    View Slide

80. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Octocatalog-Diff: Validating Puppet Agent Upgrade
    80
    !
    

    View Slide

81. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    When CI is not Enough...
    81
    !
    

    View Slide

82. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Branch Deploys
    82
    !
    

    View Slide

83. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Real Deploys
    83
    !
    

    View Slide

84. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Other Tips...
    84
    !
    

    View Slide

85. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Protected Branches and Code Review
    85
    !
    • Protected branches:
    • Require certain CI jobs to
    pass before merging
    • Require an approved code
    review before merging
    • Require branches to be

    up-to-date before merging
    

    View Slide

86. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Time to Completion
    86
    !
    

    View Slide

87. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Time to Completion
    87
    !
    

    View Slide

88. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Re-Evaluate Necessity
    88
    !
    

    View Slide

89. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Conclusion
    89
    !
    

    View Slide

90. ! @kpaulisse - Beyond rspec: Innovative Strategies for Confident CI PUPPETCONF 2017
    Beyond rspec: Innovative Strategies for Confident CI
    90
    !
    @kpaulisse
    kpaulisse
    x
    kpaulisse@gmail.com
    #
    kpaulisse (puppetcommunity.slack.com)
    Kevin Paulisse - Contact Info & Links
    https://github.com/github/octofacts
    https://linkedin.com/in/kpaulisse
    https://github.com/github/octocatalog-diff
    x
    

    View Slide