Infrastructure as Code and Chef

Every business is becoming a software business. The transformations underway require dynamic infrastructure, automating the stack, and changing the way value is delivered to your customers. This talk will look at the ways Chef helps you manage your infrastructure as code. We'll explore the workflow changes that Chef enables and end with a demonstration of the entire Chef workflow. Along the way, you'll learn about Chef, DevOps, Continuous Delivery, and Compliance.

This talk was given at the .NET Prague Meetup on 3 Feb 2016. http://www.meetup.com/Prague-NET-Meetup/events/228106089/

Transcript

1. Nathen Harvey | VP, Community Development | nharvey@chef.io
   

   View Slide

2. EVERY business is a software business
   We’re going to be a software
   company with airplanes.
   – CIO, Alaska Airlines
   

   View Slide

3. Quality and innovation, historically a tradeoff
   QUALITY/COMPLIANCE
   RATE OF INNOVATION
   

   View Slide

4. Challenges
   Manual processes
   Weeks to setup new systems or software
   Legacy systems and tools
   Inflexible, hard-to-change hardware and software
   Organizational silos
   Unwieldy divisions of responsibility
   Infrequent, large releases
   Fear of deployment due to risk to SLAs
   Regulatory burdens
   Compliance bottleneck at the end of a project
   

   View Slide

5. Idea Value
   æ
   S
   ƴ
   Ä
   Nj
   Manual processes
   Weeks to setup new systems or software
   Legacy systems and tools
   Inflexible, hard-to-change hardware and software
   Organizational silos
   Unwieldy divisions of responsibility
   Regulatory burdens
   Compliance bottleneck at the end of a project
   Infrequent, large releases
   Fear of deployment due to risk to SLAs
   Automation
   New systems and software updates deployed in minutes
   Solutions to the challenges
   

   View Slide

6. Idea Value
   æ
   S
   ƴ
   Ä
   Nj
   Manual processes
   Weeks to setup new systems or software
   Legacy systems and tools
   Inflexible, hard-to-change hardware and software
   Organizational silos
   Unwieldy divisions of responsibility
   Regulatory burdens
   Compliance bottleneck at the end of a project
   Infrequent, large releases
   Fear of deployment due to risk to SLAs
   Dynamic infrastructure
   Easy migration to on-demand, cloud-based infrastructure and
   management of heterogeneous networks
   Automation
   New systems and software updates deployed in minutes
   Solutions to the challenges
   

   View Slide

7. Idea Value
   æ
   S
   ƴ
   Ä
   Nj
   Manual processes
   Weeks to setup new systems or software
   Legacy systems and tools
   Inflexible, hard-to-change hardware and software
   Organizational silos
   Unwieldy divisions of responsibility
   Regulatory burdens
   Compliance bottleneck at the end of a project
   Infrequent, large releases
   Fear of deployment due to risk to SLAs
   Increased cooperation and trust
   Teams are aligned towards common goals
   Dynamic infrastructure
   Easy migration to on-demand, cloud-based infrastructure and
   management of heterogeneous networks
   Automation
   New systems and software updates deployed in minutes
   Solutions to the challenges
   

   View Slide

8. Idea Value
   æ
   S
   ƴ
   Ä
   Nj
   Manual processes
   Weeks to setup new systems or software
   Legacy systems and tools
   Inflexible, hard-to-change hardware and software
   Organizational silos
   Unwieldy divisions of responsibility
   Regulatory burdens
   Compliance bottleneck at the end of a project
   Infrequent, large releases
   Fear of deployment due to risk to SLAs
   Increased cooperation and trust
   Teams are aligned towards common goals
   Dynamic infrastructure
   Easy migration to on-demand, cloud-based infrastructure and
   management of heterogeneous networks
   Automation
   New systems and software updates deployed in minutes
   Solutions to the challenges
   Continuous delivery of infrastructure and
   applications
   Safe, rapid delivery of incremental value
   

   View Slide

9. Idea Value
   æ
   S
   ƴ
   Ä
   Nj
   Manual processes
   Weeks to setup new systems or software
   Legacy systems and tools
   Inflexible, hard-to-change hardware and software
   Organizational silos
   Unwieldy divisions of responsibility
   Regulatory burdens
   Compliance bottleneck at the end of a project
   Infrequent, large releases
   Fear of deployment due to risk to SLAs
   Compliance at velocity
   Integration of compliance into the workflow using automated
   tests
   Increased cooperation and trust
   Teams are aligned towards common goals
   Dynamic infrastructure
   Easy migration to on-demand, cloud-based infrastructure and
   management of heterogeneous networks
   Automation
   New systems and software updates deployed in minutes
   Continuous delivery of infrastructure and
   applications
   Safe, rapid delivery of incremental value
   Solutions to the challenges
   

   View Slide

10. The game changer: rapid time to value
    Innovation
    Quality/
    Compliance
    Dynamic
    Infrastructure
    

    View Slide

11. Dynamic infrastructure
    Migrate applications to the cloud and support hybrid and multi-cloud
    environments. Automate the management of heterogeneous
    networks, including legacy systems.
    •  Provisioning and setting up environments
    •  Dynamic scaling of compute resources
    •  Migrating legacy workloads to the cloud
    •  Multi cloud and hybrid cloud deployment
    •  Support for heterogeneous environments
    

    View Slide

12. The game changer: rapid time to value
    Innovation
    Quality/
    Compliance
    Dynamic
    Infrastructure
    Infrastructure as Code
    Automate the Stack
    +
    

    View Slide

13. Infrastructure as Code
    •  Programmatically provision and configure
    components
    

    View Slide

14. Infrastructure as Code
    •  Programmatically provision and configure
    components
    •  Treat like any other code base
    

    View Slide

15. Infrastructure as Code
    •  Programmatically provision and configure
    components
    •  Treat like any other code base
    •  Reconstruct business from code repository, data
    backup, and compute resources
    

    View Slide

16. Automation
    Turn infrastructure into code—infrastructure as code is versionable,
    testable and repeatable. Manual processes become a thing of the
    past.
    •  Automated, full-stack application policies
    •  Package and service installation
    •  Versionable, testable, repeatable workflow
    •  Scalable application policies
    •  Management of interdependencies across nodes
    

    View Slide

17. Chef Server
    Policy
    State
    State queries
    Servers, VMs, cloud instances,
    etc.
    running the Chef client
    •  The Chef server stores policy and configuration data
    •  The Chef client periodically runs on each node in the network
    •  Chef clients poll the server for the latest policies
    •  Chef clients notify the server of their states and can query for the states of other nodes
    

    View Slide

18. Describe Infrastructure as Code
    httpd_service 'customers' do
    mpm 'prefork'
    action [:create, :start]
    end
    httpd_config 'customers' do
    instance 'customers'
    source 'customers.conf.erb'
    notifies :restart, 'httpd_service[customers]'
    end
    directory '/var/www/customers/public_html' do
    recursive true
    end
    

    View Slide

19. Test the Code
    describe 'apache::default' do
    context 'When all attributes are default, on an unspecified platform' do
    let(:chef_run) do
    runner = ChefSpec::ServerRunner.new
    runner.converge(described_recipe)
    end
    it 'converges successfully' do
    expect { chef_run }.to_not raise_error
    end
    it 'installs apache' do
    expect(chef_run).to install_package 'apache2'
    end
    end
    end
    

    View Slide

20. Version the Code & the Artifact
    name 'cmgw'
    maintainer 'Chef Software, Inc.'
    maintainer_email 'nharvey@chef.io'
    license 'apache2'
    description 'Installs/Configures cmgw'
    long_description 'Installs/Configures cmgw'
    version '0.1.0'
    

    View Slide

21. The game changer: rapid time to value
    Innovation
    Quality/
    Compliance
    Dynamic
    Infrastructure
    Infrastructure as Code
    Automate the Stack
    DevOps
    + +
    

    View Slide

22. DEVOPS
    A cultural and professional movement,
    focused on how we build and operate
    high velocity organizations, born from
    the experiences of its practitioners.
    

    View Slide

23. PEOPLE
    PRODUCTS
    COMPANIES
    

    View Slide

24. WE ARE LEAN
    •  Eliminate non-value-added action (Waste/Muda)
    •  Pull over Push
    •  Kaizen (Continuous Improvement)
    •  Kaikaku (Disruptive Change)
    •  Small Batch + Experimentation
    

    View Slide

25. View Slide

26. UBIQUITOUS
    WORKFLOW
    AUTOMATION
    

    View Slide

27. DIVERSITY
    

    View Slide

28. DevOps workflow & culture
    Eliminate silos and lower the overhead of IT operations and service
    management by supporting DevOps culture. Build communities.
    •  Unified workflow for application and infrastructure
    •  Integration with version control for dev and ops
    •  Support for automated testing of infrastructure and
    applications
    •  Integration of security and compliance into product
    development
    •  Advanced, high-velocity workflow
    

    View Slide

29. Continuous delivery of infrastructure & apps
    Implement a high-velocity software delivery pipeline that integrates
    application and infrastructure. Eliminate the risks incurred with large,
    infrequent releases.
    •  Rapid provisioning of dev and test environments
    •  Ensure consistency and repeatability of environments
    •  Unified pipeline for infrastructure, runtime
    environments and applications
    •  Support for large teams with multiple projects
    •  Advanced, high-velocity workflow
    

    View Slide

30. Security and compliance at velocity
    Regulatory compliance and security concerns are facts of life for
    every enterprise. At the same time, competitive pressures are
    increasing. Embed requirements into the software delivery pipeline.
    Code makes compliance at velocity possible.
    •  Embed compliance into the software delivery pipeline
    •  Automated checking of compliance criteria with
    analytics
    •  Structured review process during development
    •  Discovery and analysis
    •  Patch management and remediation
    

    View Slide

31. View Slide

32. MANAGE RISK
    •  Small batches, near term hypothesis
    •  Validation comes from customers
    •  Introduce near-term volatility to gain decreased
    long-term risk
    

    View Slide

33. View Slide

34. CONTINUOUS INTEGRATION
    •  Always integrate branches to master
    •  They should be short lived, iterative branches
    •  Fix the build when it goes red
    

    View Slide

35. THE FOUR-EYE RULE
    

    View Slide

36. WRITE TESTS
    •  Unit test (a single function)
    •  Integration tests (multiple classes/units)
    •  Functional tests (user-oriented, high-level, full
    stack)
    •  Smoke tests (quickly determine if the system is
    “working”)
    

    View Slide

37. ONE PATH FOR CHANGE
    •  The way change moves through your
    organization is fixed
    •  Designed to re-enforce your principles and aid
    flow
    •  Flexible at the level of execution
    

    View Slide

38. CODE GOES THROUGH THE
    SAME WORKFLOW
    Applications are code
    Infrastructure is code
    

    View Slide

39. The rewards?*
    *source: Dr. Nicole Forsgren research on DevOps
    More deployments
    Ship faster
    Faster MTTR after issues
    More profits, market share, and productivity
    Market cap goes up
    

    View Slide

40. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Collaborative Dev Production
    Chef Server
    Chef Server
    Chef Supermarket
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Deploy
    Test
    Chef Delivery!
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks
    

    View Slide

41. Demo Time
    

    View Slide

42. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    

    View Slide

43. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks
    

    View Slide

44. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Collaborative Dev
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Deploy
    Test
    Chef Delivery!
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks
    

    View Slide

45. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Submit
    Change
    

    View Slide

46. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Submit
    Change
    

    View Slide

47. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Submit
    Change
    Does this
    code change
    look good?
    

    View Slide

48. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Submit
    Change
    Does this
    code change
    look good?
    

    View Slide

49. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    

    View Slide

50. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?
    

    View Slide

51. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?
    

    View Slide

52. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?
    

    View Slide

53. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?
    

    View Slide

54. Shared Workflow
    Delivery’s pipeline is shared across projects and teams
    

    View Slide

55. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Collaborative Dev Production
    Chef Server
    Chef Server
    Chef Supermarket
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Deploy
    Test
    Chef Delivery!
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks
    

    View Slide

56. What questions can I answer for you?
    

    View Slide

57. Nathen Harvey
    • VP, Community Development at Chef
    • Co-host of the Food Fight Show Podcast
    • Co-organizer of DevOpsDC meetup
    • Occasional farmer – http://ei.chef.io
    • Love eggs – http://eggs.chef.io
    • @nathenharvey
    • nharvey@chef.io
    

    View Slide