$30 off During Our Annual Pro Sale. View Details »

Chef Compliance Workshop

Nathen Harvey
December 02, 2015

Chef Compliance Workshop

Beta version of a Chef Compliance Workshop

Nathen Harvey

December 02, 2015
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. Chef Compliance An Introduction Nathen Harvey - nharvey@chef.io @nathenharvey

  2. Objectives By the end of this workshop you will be

    able to: » Describe the capabilities of Chef Compliance » Configure Chef Compliance to scan nodes in your environment » Write custom Compliance policies using InSpec » Upload custom Compliance policies to your Chef Compliance server » Use InSpec-based compliance checks in your cookbook
  3. Chef Compliance - Scan Scan your entire infrastructure for risks

    and compliance issues
  4. Chef Compliance - Report Get reports on risks and issues

    classified by severity and impact levels
  5. Chef Compliance - Automate Build automated compliance testing and remediation

    into your pipeline
  6. Chef Compliance - Audit Get started quickly with pre-built profiles

    for CIS, Linux and Windows
  7. Lab 1 - Login to Chef Compliance server » Open

    Chef Compliance URL provided » Tell your browser it's OK to trust the untrusted certificate
  8. » Login with the credentials provided

  9. Lab 2 - Add Your Node » From the Dashboard,

    click the 'Add' button
  10. » Enter the IP address of your node » Enter

    an Environment (use your name)
  11. Using SSH as the access protocol: » set the Username

    » choose to login with a password
  12. » set the Password » Add the node

  13. Lab 3 - Verify Connectivity » From the Dashboard, select

    your node » Click the 'Connectivity' button
  14. » The status should be "Connection established"

  15. Chef Compliance - Scan Scan your entire infrastructure for risks

    and compliance issues
  16. Chef Compliance - Audit Get started quickly with pre-built profiles

    for CIS, Linux and Windows
  17. Lab 4 - Scan the SSH configuration » From the

    Dashboard, select your node » Click the 'Scan' button
  18. » Select only the base/ssh compliance profile » Press the

    'Scan now' button
  19. Chef Compliance - Report Get reports on risks and issues

    classified by severity and impact levels
  20. Lab 5 - Review the Compliance Report

  21. None
  22. Expand the 'Client: Set SSH protocol version to 2' control

  23. Lab 6 - Review a Critical Issue Let's look at

    that SSH control. » Navigate to the Compliance profiles » Click on "Basic SSH"
  24. » Find and expand the 'Client: Set SSH protocol version

    to 2' control
  25. Lab 7 - Remediate the Issue Login to your node

    $ ssh chef@52.90.226.238 chef@52.90.226.238's password: Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-48-generic x86_64) * Documentation: https://help.ubuntu.com/ System information as of Wed Dec 2 04:14:55 UTC 2015 System load: 0.02 Processes: 110 Usage of /: 41.2% of 7.74GB Users logged in: 1 Memory usage: 4% IP address for eth0: 172.31.1.118 Swap usage: 0% IP address for docker0: 172.17.42.1
  26. Generate an SSH cookbook chef@node$ mkdir -p cookbooks chef@node$ cd

    cookbooks
  27. Generate an SSH cookbook chef@node$ chef generate cookbook ssh Compiling

    Cookbooks... Recipe: code_generator::cookbook * directory[/home/chef/cookbooks/ssh] action create - create new directory /home/chef/cookbooks/ssh * template[/home/chef/cookbooks/ssh/metadata.rb] action create_if_missing - create new file /home/chef/cookbooks/ssh/metadata.rb - update content in file /home/chef/cookbooks/ssh/metadata.rb from none to 279f80 (diff output suppressed by config) * template[/home/chef/cookbooks/ssh/README.md] action create_if_missing - create new file /home/chef/cookbooks/ssh/README.md - update content in file /home/chef/cookbooks/ssh/README.md from none to 16927d (diff output suppressed by config) * cookbook_file[/home/chef/cookbooks/ssh/chefignore] action create - create new file /home/chef/cookbooks/ssh/chefignore - update content in file /home/chef/cookbooks/ssh/chefignore from none to 51b09a
  28. Create a Client recipe chef@node$ chef generate recipe ssh client

    Compiling Cookbooks... Recipe: code_generator::recipe * directory[./ssh/spec/unit/recipes] action create (up to date) * cookbook_file[./ssh/spec/spec_helper.rb] action create_if_missing (up to date) * template[./ssh/spec/unit/recipes/client_spec.rb] action create_if_missing - create new file ./ssh/spec/unit/recipes/client_spec.rb - update content in file ./ssh/spec/unit/recipes/client_spec.rb from none to 51dff0 (diff output suppressed by config) * template[./ssh/recipes/client.rb] action create - create new file ./ssh/recipes/client.rb - update content in file ./ssh/recipes/client.rb from none to 9c811a (diff output suppressed by config)
  29. Create a template file chef@node$ chef generate template ssh \

    ssh_config.erb -s /etc/ssh/ssh_config Compiling Cookbooks... Recipe: code_generator::template * directory[./ssh/templates/default] action create - create new directory ./ssh/templates/default * file[./ssh/templates/default/ssh_config.erb] action create - create new file ./ssh/templates/default/ssh_config.erb - update content in file ./ssh/templates/default/ssh_config.erb from none to 6005ad (diff output suppressed by config)
  30. Write the client recipe Open file: ~/cookbooks/ssh/recipes/client.rb template '/etc/ssh/ssh_config' do

    source 'ssh_config.erb' owner 'root' group 'root' mode '0644' end
  31. Lab 8 - Test Kitchen (1 of 4) Open file:

    ~/cookbooks/ssh/.kitchen.yml --- driver: name: docker use_sudo: false
  32. Lab 8 - Test Kitchen (2 of 4) Open file:

    ~/cookbooks/ssh/.kitchen.yml provisioner: name: chef_zero # Uncomment the following verifier to leverage Inspec instead of Busser (the # default verifier) # verifier: # name: inspec
  33. Lab 8 - Test Kitchen (3 of 4) Open file:

    ~/cookbooks/ssh/.kitchen.yml platforms: - name: ubuntu-14.04 # - name: centos-7.1
  34. Lab 8 - Test Kitchen (4 of 4) Open file:

    ~/cookbooks/ssh/.kitchen.yml suites: - name: client run_list: - recipe[ssh::client] attributes:
  35. Converge the Kitchen chef@node$ cd ~/cookbooks/ssh chef@node$ kitchen converge

  36. -----> Starting Kitchen (v1.4.2) /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/httpclient-2.6.0.1/lib/httpclient/webagent-cookie.rb:458: warning: already initialized constant HTTPClient::CookieManager

    /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/httpclient-2.6.0.1/lib/httpclient/cookie.rb:8: warning: previous definition of CookieManager was here -----> Creating <client-ubuntu-1404>... Sending build context to Docker daemon 63.49 kB Sending build context to Docker daemon Step 0 : FROM ubuntu:14.04 ---> ca4d7b1b9a51 Step 1 : RUN dpkg-divert --local --rename --add /sbin/initctl ---> Running in 5cd1cebf6812 Leaving 'local diversion of /sbin/initctl to /sbin/initctl.distrib' ---> 10d23f2f5ec2 Removing intermediate container 5cd1cebf6812 Step 2 : RUN ln -sf /bin/true /sbin/initctl ---> Running in 33a8c8ddf8f5 ---> 64fbee7d40c3 Removing intermediate container 33a8c8ddf8f5 Step 3 : ENV DEBIAN_FRONTEND noninteractive ---> Running in f232fdc644fd
  37. Lab 9 - Add InSpec Verification Add an InSpec test

    chef@node$ mkdir -p ~/cookbooks/ssh/test/integration/client/ inspec
  38. 1 of 2 Open file: ~/cookbooks/ssh/test/integration/client/inspec/ client_spec.rb control 'ssh-4' do

    impact 1.0 title 'Client: Set SSH protocol version to 2' desc " Set the SSH protocol version to 2. Don't use legacy insecure SSHv1 connections anymore. "
  39. 2 of 2 Open file: ~/cookbooks/ssh/test/integration/client/inspec/ client_spec.rb describe ssh_config do

    its('Protocol') { should eq('2') } end end Copy-and-paste the control from Chef Compliance, we want to test the same thing.
  40. Run InSpec from the Command Line » InSpec is an

    executable application. » InSpec can execute on remote hosts, including docker containers
  41. » What is your docker container id? chef@ndoe$ docker ps

    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 511b3fcb2777 af3815cee160:latest "/usr/sbin/sshd -D - 10 seconds ago Up 10 seconds 0.0.0.0:32773->22/tcp silly_davinci
  42. chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://container_id

  43. chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://88e57e403cd1 F Failures: 1)

    SSH Configuration Protocol should eq "2" Failure/Error: its('Protocol') { should eq('2') } expected: "2" got: nil (compared using ==) # ./test/integration/client/inspec/client_spec.rb:9:in `block (3 levels) in load' Finished in 0.2124 seconds (files took 0.37134 seconds to load) 1 example, 1 failure Failed examples: rspec # SSH Configuration Protocol should eq "2"
  44. Update the template Open file: ~/cookbooks/ssh/templates/default/ssh_config.erb Change # Protocol 2,1

    To Protocol 2
  45. Apply the change to the Test Kitchen chef@node$ cd ~/cookbooks/ssh

    chef@node$ kitchen converge -----> Converging <client-ubuntu-1404>... ... Converging 1 resources Recipe: ssh::client * template[/etc/ssh/ssh_config] action create - update content in file /etc/ssh/ssh_config from 6005ad to ed645d --- /etc/ssh/ssh_config 2015-08-18 02:14:14.000000000 +0000 +++ /etc/ssh/.ssh_config20151202-416-1p3jnk3 2015-12-02 07:35:17.734336383 +0000 @@ -37,7 +37,7 @@ # IdentityFile ~/.ssh/id_rsa # IdentityFile ~/.ssh/id_dsa # Port 22 -# Protocol 2,1 + Protocol 2 # Cipher 3des # Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc # MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160 Running handlers: Running handlers complete Chef Client finished, 1/1 resources updated in 01 seconds Finished converging <client-ubuntu-1404> (0m3.52s). -----> Kitchen is finished. (0m4.20s)
  46. Run InSpec from the Command Line chef@node$ inspec exec ~/cookbooks/ssh/test/integration/

    client/inspec/client_spec.rb -t docker://container_id
  47. chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://511b3fcb2777 . Finished in

    0.18559 seconds (files took 0.36615 seconds to load) 1 example, 0 failures
  48. Lab 10 - Apply the new SSH Policy Use chef-client

    to apply the new SSH policy chef@node$ cd ~/
  49. chef@node$ sudo chef-client --local-mode -r 'recipe[ssh::client]' Starting Chef Client, version

    12.5.1 resolving cookbooks for run list: ["ssh::client"] Synchronizing Cookbooks: - ssh (0.1.0) Compiling Cookbooks... Converging 1 resources Recipe: ssh::client * template[/etc/ssh/ssh_config] action create - update content in file /etc/ssh/ssh_config from 6005ad to ed645d --- /etc/ssh/ssh_config 2015-12-02 04:14:38.422336383 +0000 +++ /etc/ssh/.ssh_config20151202-14917-s8hgi2 2015-12-02 07:42:23.330336383 +0000 @@ -37,7 +37,7 @@ # IdentityFile ~/.ssh/id_rsa # IdentityFile ~/.ssh/id_dsa # Port 22 -# Protocol 2,1 + Protocol 2 # Cipher 3des # Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc # MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160 Running handlers: Running handlers complete Chef Client finished, 1/1 resources updated in 01 seconds
  50. Lab 11 - Scan the SSH configuration » From the

    Dashboard, select your node » Click the 'Scan' button
  51. » Select only the base/ssh compliance profile

  52. » Press the 'Scan now' button

  53. Lab 12 - Review the Compliance Report Verify the 'Client:

    Set SSH protocol version to 2' control is now 'Compliant'
  54. None
  55. Lab 13 - Create a Custom Compliance profile » Compliance

    Profiles are built using InSpec
  56. Create a directory for your profiles chef@node$ mkdir -p ~/compliance_profiles

    chef@node$ cd ~/compliance_profiles
  57. Is it a profile yet? » InSpec includes a command

    to check a profile chef@node mkdir -p USERNAME/tmp_profile_for_USERNAME
  58. chef@node$ inspec check USERNAME/tmp_profile_for_USERNAME E, [2015-12-02T07:58:03.197043 #15448] ERROR -- :

    Can't find metadata file username/tmp_profile_for_USERNAME/metadata.rb /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/inspec-0.9.2/lib/inspec/targets/folder.rb:23:in `resolve': Don't know how to handle folder username/tmp_profile_for_USERNAME/ (RuntimeError) ...
  59. Create the metadata file chef@node$ touch USERNAME/tmp_profile_for_USERNAME/ metadata.rb chef@node$ inspec

    check USERNAME/tmp_profile_for_USERNAME I, [2015-12-02T08:02:19.171190 #15458] INFO -- : Checking profile in USERNAME/tmp_profile_for_USERNAME E, [2015-12-02T08:02:19.171445 #15458] ERROR -- : No profile name defined W, [2015-12-02T08:02:19.171558 #15458] WARN -- : No version defined W, [2015-12-02T08:02:19.171676 #15458] WARN -- : No title defined W, [2015-12-02T08:02:19.171770 #15458] WARN -- : No maintainer defined /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/inspec-0.9.2/lib/inspec/profile.rb:104:in `check': undefined method `empty?' for nil:NilClass (NoMethodError)
  60. Address the ERRORs and WARNs Open file: ~/compliance_profiles/USERNAME/ tmp_profile_for_USERNAME/metadata.rb name

    'USERNAME/tmp_profile_for_USERNAME' version '0.1.0' title '/tmp Profile for USERNAME' maintainer 'YOUR NAME'
  61. Create a directory for the controls chef@node$ mkdir -p USERNAME/tmp_profile_for_USERNAME/test

  62. Open file: ~/compliance_profiles/USERNAME/ tmp_profile_for_USERNAME/test/tmp.rb title '/tmp profile for USERNAME' control

    "tmp-1.0" do # A unique ID for this control impact 0.3 # The criticality, if this control fails. title "Create /tmp directory" # A human-readable title desc "A /tmp directory must exist" describe file('/tmp') do # The actual test it { should be_directory } end end control "tmp-1.1" do impact 0.3 title "/tmp directory is owned by the root user" desc "The /tmp directory must be owned by the root user" describe file('/tmp') do it { should be_owned_by 'root' } end end
  63. chef@node$ inspec check USERNAME/tmp_profile_for_USERNAME I, [2015-12-02T08:42:30.432722 #16567] INFO -- :

    Checking profile in USERNAME/tmp_profile_for_USERNAME I, [2015-12-02T08:42:30.432937 #16567] INFO -- : Metadata OK. D, [2015-12-02T08:42:30.433053 #16567] DEBUG -- : Found 20 rules. D, [2015-12-02T08:42:30.433154 #16567] DEBUG -- : Verify all rules in USERNAME/tmp_profile_for_USERNAME/test/tmp.rb I, [2015-12-02T08:42:30.433256 #16567] INFO -- : Rule definitions OK.
  64. Test the profile locally chef@node$ inspec exec USERNAME/tmp_profile_for_USERNAME/ .. Finished

    in 0.0166 seconds (files took 0.24531 seconds to load) 2 examples, 0 failures
  65. Zip up the profile chef@node$ cd ~/compliance_profiles chef@node$ zip -r

    USERNAME.zip USERNAME
  66. SCP the .zip file to your LOCAL computer $ scp

    chef@IPADDRESS:~/compliance_profiles/USERNAME.zip . chef@52.90.226.238's password: USERNAME.zip 100% 1257 1.2KB/s 00:00
  67. Lab 14 - Upload the profile to Chef Compliance »

    From the Compliance page, press the 'Add profile' button
  68. » Select the USERNAME.zip file you created

  69. Lab 15 - Scan with the new profile » From

    the Dashboard, select your node » Click the 'Scan' button
  70. » Select only the chef/tmp_profile_for_USERNAME compliance profile

  71. » Press the 'Scan now' button

  72. Lab 16 - Review the Compliance Report

  73. Chef Compliance - Scan Scan your entire infrastructure for risks

    and compliance issues
  74. Chef Compliance - Report Get reports on risks and issues

    classified by severity and impact levels
  75. Chef Compliance - Automate Build automated compliance testing and remediation

    into your pipeline
  76. Chef Compliance - Audit Get started quickly with pre-built profiles

    for CIS, Linux and Windows
  77. Objectives By the end of this workshop you will be

    able to: » Describe the capabilities of Chef Compliance » Configure Chef Compliance to scan nodes in your environment » Write custom Compliance policies using InSpec » Upload custom Compliance policies to your Chef Compliance server » Use InSpec-based compliance checks in your cookbook
  78. What's next? » What questions can I answer for you?

    » Remediate more failures » Add additional tests
  79. Thank You! Nathen Harvey - nharvey@chef.io @nathenharvey