$30 off During Our Annual Pro Sale. View Details »

Infrastructure as Code and Chef

Nathen Harvey
February 03, 2016

Infrastructure as Code and Chef

Every business is becoming a software business. The transformations underway require dynamic infrastructure, automating the stack, and changing the way value is delivered to your customers. This talk will look at the ways Chef helps you manage your infrastructure as code. We'll explore the workflow changes that Chef enables and end with a demonstration of the entire Chef workflow. Along the way, you'll learn about Chef, DevOps, Continuous Delivery, and Compliance.

This talk was given at the .NET Prague Meetup on 3 Feb 2016. http://www.meetup.com/Prague-NET-Meetup/events/228106089/

Nathen Harvey

February 03, 2016
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. Nathen Harvey | VP, Community Development | [email protected]

    View Slide

  2. EVERY business is a software business
    We’re going to be a software
    company with airplanes.
    – CIO, Alaska Airlines

    View Slide

  3. Quality and innovation, historically a tradeoff
    QUALITY/COMPLIANCE
    RATE OF INNOVATION

    View Slide

  4. Challenges
    Manual processes
    Weeks to setup new systems or software
    Legacy systems and tools
    Inflexible, hard-to-change hardware and software
    Organizational silos
    Unwieldy divisions of responsibility
    Infrequent, large releases
    Fear of deployment due to risk to SLAs
    Regulatory burdens
    Compliance bottleneck at the end of a project

    View Slide

  5. Idea Value
    æ
    S
    ƴ
    Ä
    Nj
    Manual processes
    Weeks to setup new systems or software
    Legacy systems and tools
    Inflexible, hard-to-change hardware and software
    Organizational silos
    Unwieldy divisions of responsibility
    Regulatory burdens
    Compliance bottleneck at the end of a project
    Infrequent, large releases
    Fear of deployment due to risk to SLAs
    Automation
    New systems and software updates deployed in minutes
    Solutions to the challenges

    View Slide

  6. Idea Value
    æ
    S
    ƴ
    Ä
    Nj
    Manual processes
    Weeks to setup new systems or software
    Legacy systems and tools
    Inflexible, hard-to-change hardware and software
    Organizational silos
    Unwieldy divisions of responsibility
    Regulatory burdens
    Compliance bottleneck at the end of a project
    Infrequent, large releases
    Fear of deployment due to risk to SLAs
    Dynamic infrastructure
    Easy migration to on-demand, cloud-based infrastructure and
    management of heterogeneous networks
    Automation
    New systems and software updates deployed in minutes
    Solutions to the challenges

    View Slide

  7. Idea Value
    æ
    S
    ƴ
    Ä
    Nj
    Manual processes
    Weeks to setup new systems or software
    Legacy systems and tools
    Inflexible, hard-to-change hardware and software
    Organizational silos
    Unwieldy divisions of responsibility
    Regulatory burdens
    Compliance bottleneck at the end of a project
    Infrequent, large releases
    Fear of deployment due to risk to SLAs
    Increased cooperation and trust
    Teams are aligned towards common goals
    Dynamic infrastructure
    Easy migration to on-demand, cloud-based infrastructure and
    management of heterogeneous networks
    Automation
    New systems and software updates deployed in minutes
    Solutions to the challenges

    View Slide

  8. Idea Value
    æ
    S
    ƴ
    Ä
    Nj
    Manual processes
    Weeks to setup new systems or software
    Legacy systems and tools
    Inflexible, hard-to-change hardware and software
    Organizational silos
    Unwieldy divisions of responsibility
    Regulatory burdens
    Compliance bottleneck at the end of a project
    Infrequent, large releases
    Fear of deployment due to risk to SLAs
    Increased cooperation and trust
    Teams are aligned towards common goals
    Dynamic infrastructure
    Easy migration to on-demand, cloud-based infrastructure and
    management of heterogeneous networks
    Automation
    New systems and software updates deployed in minutes
    Solutions to the challenges
    Continuous delivery of infrastructure and
    applications
    Safe, rapid delivery of incremental value

    View Slide

  9. Idea Value
    æ
    S
    ƴ
    Ä
    Nj
    Manual processes
    Weeks to setup new systems or software
    Legacy systems and tools
    Inflexible, hard-to-change hardware and software
    Organizational silos
    Unwieldy divisions of responsibility
    Regulatory burdens
    Compliance bottleneck at the end of a project
    Infrequent, large releases
    Fear of deployment due to risk to SLAs
    Compliance at velocity
    Integration of compliance into the workflow using automated
    tests
    Increased cooperation and trust
    Teams are aligned towards common goals
    Dynamic infrastructure
    Easy migration to on-demand, cloud-based infrastructure and
    management of heterogeneous networks
    Automation
    New systems and software updates deployed in minutes
    Continuous delivery of infrastructure and
    applications
    Safe, rapid delivery of incremental value
    Solutions to the challenges

    View Slide

  10. The game changer: rapid time to value
    Innovation
    Quality/
    Compliance
    Dynamic
    Infrastructure

    View Slide

  11. Dynamic infrastructure
    Migrate applications to the cloud and support hybrid and multi-cloud
    environments. Automate the management of heterogeneous
    networks, including legacy systems.
    •  Provisioning and setting up environments
    •  Dynamic scaling of compute resources
    •  Migrating legacy workloads to the cloud
    •  Multi cloud and hybrid cloud deployment
    •  Support for heterogeneous environments

    View Slide

  12. The game changer: rapid time to value
    Innovation
    Quality/
    Compliance
    Dynamic
    Infrastructure
    Infrastructure as Code
    Automate the Stack
    +

    View Slide

  13. Infrastructure as Code
    •  Programmatically provision and configure
    components

    View Slide

  14. Infrastructure as Code
    •  Programmatically provision and configure
    components
    •  Treat like any other code base

    View Slide

  15. Infrastructure as Code
    •  Programmatically provision and configure
    components
    •  Treat like any other code base
    •  Reconstruct business from code repository, data
    backup, and compute resources

    View Slide

  16. Automation
    Turn infrastructure into code—infrastructure as code is versionable,
    testable and repeatable. Manual processes become a thing of the
    past.
    •  Automated, full-stack application policies
    •  Package and service installation
    •  Versionable, testable, repeatable workflow
    •  Scalable application policies
    •  Management of interdependencies across nodes

    View Slide

  17. Chef Server
    Policy
    State
    State queries
    Servers, VMs, cloud instances,
    etc.
    running the Chef client
    •  The Chef server stores policy and configuration data
    •  The Chef client periodically runs on each node in the network
    •  Chef clients poll the server for the latest policies
    •  Chef clients notify the server of their states and can query for the states of other nodes

    View Slide

  18. Describe Infrastructure as Code
    httpd_service 'customers' do
    mpm 'prefork'
    action [:create, :start]
    end
    httpd_config 'customers' do
    instance 'customers'
    source 'customers.conf.erb'
    notifies :restart, 'httpd_service[customers]'
    end
    directory '/var/www/customers/public_html' do
    recursive true
    end

    View Slide

  19. Test the Code
    describe 'apache::default' do
    context 'When all attributes are default, on an unspecified platform' do
    let(:chef_run) do
    runner = ChefSpec::ServerRunner.new
    runner.converge(described_recipe)
    end
    it 'converges successfully' do
    expect { chef_run }.to_not raise_error
    end
    it 'installs apache' do
    expect(chef_run).to install_package 'apache2'
    end
    end
    end

    View Slide

  20. Version the Code & the Artifact
    name 'cmgw'
    maintainer 'Chef Software, Inc.'
    maintainer_email '[email protected]'
    license 'apache2'
    description 'Installs/Configures cmgw'
    long_description 'Installs/Configures cmgw'
    version '0.1.0'

    View Slide

  21. The game changer: rapid time to value
    Innovation
    Quality/
    Compliance
    Dynamic
    Infrastructure
    Infrastructure as Code
    Automate the Stack
    DevOps
    + +

    View Slide

  22. DEVOPS
    A cultural and professional movement,
    focused on how we build and operate
    high velocity organizations, born from
    the experiences of its practitioners.

    View Slide

  23. PEOPLE
    PRODUCTS
    COMPANIES

    View Slide

  24. WE ARE LEAN
    •  Eliminate non-value-added action (Waste/Muda)
    •  Pull over Push
    •  Kaizen (Continuous Improvement)
    •  Kaikaku (Disruptive Change)
    •  Small Batch + Experimentation

    View Slide

  25. View Slide

  26. UBIQUITOUS
    WORKFLOW
    AUTOMATION

    View Slide

  27. DIVERSITY

    View Slide

  28. DevOps workflow & culture
    Eliminate silos and lower the overhead of IT operations and service
    management by supporting DevOps culture. Build communities.
    •  Unified workflow for application and infrastructure
    •  Integration with version control for dev and ops
    •  Support for automated testing of infrastructure and
    applications
    •  Integration of security and compliance into product
    development
    •  Advanced, high-velocity workflow

    View Slide

  29. Continuous delivery of infrastructure & apps
    Implement a high-velocity software delivery pipeline that integrates
    application and infrastructure. Eliminate the risks incurred with large,
    infrequent releases.
    •  Rapid provisioning of dev and test environments
    •  Ensure consistency and repeatability of environments
    •  Unified pipeline for infrastructure, runtime
    environments and applications
    •  Support for large teams with multiple projects
    •  Advanced, high-velocity workflow

    View Slide

  30. Security and compliance at velocity
    Regulatory compliance and security concerns are facts of life for
    every enterprise. At the same time, competitive pressures are
    increasing. Embed requirements into the software delivery pipeline.
    Code makes compliance at velocity possible.
    •  Embed compliance into the software delivery pipeline
    •  Automated checking of compliance criteria with
    analytics
    •  Structured review process during development
    •  Discovery and analysis
    •  Patch management and remediation

    View Slide

  31. View Slide

  32. MANAGE RISK
    •  Small batches, near term hypothesis
    •  Validation comes from customers
    •  Introduce near-term volatility to gain decreased
    long-term risk

    View Slide

  33. View Slide

  34. CONTINUOUS INTEGRATION
    •  Always integrate branches to master
    •  They should be short lived, iterative branches
    •  Fix the build when it goes red

    View Slide

  35. THE FOUR-EYE RULE

    View Slide

  36. WRITE TESTS
    •  Unit test (a single function)
    •  Integration tests (multiple classes/units)
    •  Functional tests (user-oriented, high-level, full
    stack)
    •  Smoke tests (quickly determine if the system is
    “working”)

    View Slide

  37. ONE PATH FOR CHANGE
    •  The way change moves through your
    organization is fixed
    •  Designed to re-enforce your principles and aid
    flow
    •  Flexible at the level of execution

    View Slide

  38. CODE GOES THROUGH THE
    SAME WORKFLOW
    Applications are code
    Infrastructure is code

    View Slide

  39. The rewards?*
    *source: Dr. Nicole Forsgren research on DevOps
    More deployments
    Ship faster
    Faster MTTR after issues
    More profits, market share, and productivity
    Market cap goes up

    View Slide

  40. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Collaborative Dev Production
    Chef Server
    Chef Server
    Chef Supermarket
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Deploy
    Test
    Chef Delivery!
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks

    View Slide

  41. Demo Time

    View Slide

  42. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover

    View Slide

  43. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks

    View Slide

  44. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Collaborative Dev
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Deploy
    Test
    Chef Delivery!
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks

    View Slide

  45. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Submit
    Change

    View Slide

  46. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Submit
    Change

    View Slide

  47. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Submit
    Change
    Does this
    code change
    look good?

    View Slide

  48. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Submit
    Change
    Does this
    code change
    look good?

    View Slide

  49. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?

    View Slide

  50. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?

    View Slide

  51. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?

    View Slide

  52. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?

    View Slide

  53. Unified Pipeline Shape
    The stages are fixed, and each stage has a fixed set of phases!
    APPROVE DELIVER
    Lint
    Syntax
    Unit
    Security
    Quality
    Publish
    Lint
    Syntax
    Unit
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Provision
    Deploy
    Smoke
    Functional
    Submit
    Change
    Does this
    code change
    look good?
    Do we want
    to ship this?

    View Slide

  54. Shared Workflow
    Delivery’s pipeline is shared across projects and teams

    View Slide

  55. Chef Provides a Proven Approach to DevOps
    ...
    ...
    ...
    Targets/Workloads
    Collaborative Dev Production
    Chef Server
    Chef Server
    Chef Supermarket
    Assessment
    Chef Compliance
    Search
    ^ Audit
    Ǘ Discover
    Deploy
    Test
    Chef Delivery!
    Local Dev
    ê Model
    ƨ Build
    ¿ Test
    Chef DK
    Chef Client & Cookbooks

    View Slide

  56. What questions can I answer for you?

    View Slide

  57. Nathen Harvey
    • VP, Community Development at Chef
    • Co-host of the Food Fight Show Podcast
    • Co-organizer of DevOpsDC meetup
    • Occasional farmer – http://ei.chef.io
    • Love eggs – http://eggs.chef.io
    • @nathenharvey
    • [email protected]

    View Slide