Chef Compliance Workshop

Beta version of a Chef Compliance Workshop

Transcript

1. Chef Compliance An Introduction Nathen Harvey - nharvey@chef.io @nathenharvey
2. Objectives By the end of this workshop you will be able to: » Describe the capabilities of Chef Compliance » Configure Chef Compliance to scan nodes in your environment » Write custom Compliance policies using InSpec » Upload custom Compliance policies to your Chef Compliance server » Use InSpec-based compliance checks in your cookbook
3. Chef Compliance - Scan Scan your entire infrastructure for risks and compliance issues
4. Chef Compliance - Report Get reports on risks and issues classified by severity and impact levels
5. Chef Compliance - Automate Build automated compliance testing and remediation into your pipeline
6. Chef Compliance - Audit Get started quickly with pre-built profiles for CIS, Linux and Windows
7. Lab 1 - Login to Chef Compliance server » Open Chef Compliance URL provided » Tell your browser it's OK to trust the untrusted certificate
8. » Login with the credentials provided
9. Lab 2 - Add Your Node » From the Dashboard, click the 'Add' button
10. » Enter the IP address of your node » Enter an Environment (use your name)
11. Using SSH as the access protocol: » set the Username » choose to login with a password
12. » set the Password » Add the node
13. Lab 3 - Verify Connectivity » From the Dashboard, select your node » Click the 'Connectivity' button
14. » The status should be "Connection established"
15. Chef Compliance - Scan Scan your entire infrastructure for risks and compliance issues
16. Chef Compliance - Audit Get started quickly with pre-built profiles for CIS, Linux and Windows
17. Lab 4 - Scan the SSH configuration » From the Dashboard, select your node » Click the 'Scan' button
18. » Select only the base/ssh compliance profile » Press the 'Scan now' button
19. Chef Compliance - Report Get reports on risks and issues classified by severity and impact levels
20. Lab 5 - Review the Compliance Report
22. Expand the 'Client: Set SSH protocol version to 2' control
23. Lab 6 - Review a Critical Issue Let's look at that SSH control. » Navigate to the Compliance profiles » Click on "Basic SSH"
24. » Find and expand the 'Client: Set SSH protocol version to 2' control
25. Lab 7 - Remediate the Issue Login to your node $ ssh chef@52.90.226.238 chef@52.90.226.238's password: Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-48-generic x86_64) * Documentation: https://help.ubuntu.com/ System information as of Wed Dec 2 04:14:55 UTC 2015 System load: 0.02 Processes: 110 Usage of /: 41.2% of 7.74GB Users logged in: 1 Memory usage: 4% IP address for eth0: 172.31.1.118 Swap usage: 0% IP address for docker0: 172.17.42.1
26. Generate an SSH cookbook chef@node$ mkdir -p cookbooks chef@node$ cd cookbooks
27. Generate an SSH cookbook chef@node$ chef generate cookbook ssh Compiling Cookbooks... Recipe: code_generator::cookbook * directory[/home/chef/cookbooks/ssh] action create - create new directory /home/chef/cookbooks/ssh * template[/home/chef/cookbooks/ssh/metadata.rb] action create_if_missing - create new file /home/chef/cookbooks/ssh/metadata.rb - update content in file /home/chef/cookbooks/ssh/metadata.rb from none to 279f80 (diff output suppressed by config) * template[/home/chef/cookbooks/ssh/README.md] action create_if_missing - create new file /home/chef/cookbooks/ssh/README.md - update content in file /home/chef/cookbooks/ssh/README.md from none to 16927d (diff output suppressed by config) * cookbook_file[/home/chef/cookbooks/ssh/chefignore] action create - create new file /home/chef/cookbooks/ssh/chefignore - update content in file /home/chef/cookbooks/ssh/chefignore from none to 51b09a
28. Create a Client recipe chef@node$ chef generate recipe ssh client Compiling Cookbooks... Recipe: code_generator::recipe * directory[./ssh/spec/unit/recipes] action create (up to date) * cookbook_file[./ssh/spec/spec_helper.rb] action create_if_missing (up to date) * template[./ssh/spec/unit/recipes/client_spec.rb] action create_if_missing - create new file ./ssh/spec/unit/recipes/client_spec.rb - update content in file ./ssh/spec/unit/recipes/client_spec.rb from none to 51dff0 (diff output suppressed by config) * template[./ssh/recipes/client.rb] action create - create new file ./ssh/recipes/client.rb - update content in file ./ssh/recipes/client.rb from none to 9c811a (diff output suppressed by config)
29. Create a template file chef@node$ chef generate template ssh \ ssh_config.erb -s /etc/ssh/ssh_config Compiling Cookbooks... Recipe: code_generator::template * directory[./ssh/templates/default] action create - create new directory ./ssh/templates/default * file[./ssh/templates/default/ssh_config.erb] action create - create new file ./ssh/templates/default/ssh_config.erb - update content in file ./ssh/templates/default/ssh_config.erb from none to 6005ad (diff output suppressed by config)
30. Write the client recipe Open file: ~/cookbooks/ssh/recipes/client.rb template '/etc/ssh/ssh_config' do source 'ssh_config.erb' owner 'root' group 'root' mode '0644' end
31. Lab 8 - Test Kitchen (1 of 4) Open file: ~/cookbooks/ssh/.kitchen.yml --- driver: name: docker use_sudo: false
32. Lab 8 - Test Kitchen (2 of 4) Open file: ~/cookbooks/ssh/.kitchen.yml provisioner: name: chef_zero # Uncomment the following verifier to leverage Inspec instead of Busser (the # default verifier) # verifier: # name: inspec
33. Lab 8 - Test Kitchen (3 of 4) Open file: ~/cookbooks/ssh/.kitchen.yml platforms: - name: ubuntu-14.04 # - name: centos-7.1
34. Lab 8 - Test Kitchen (4 of 4) Open file: ~/cookbooks/ssh/.kitchen.yml suites: - name: client run_list: - recipe[ssh::client] attributes:
35. Converge the Kitchen chef@node$ cd ~/cookbooks/ssh chef@node$ kitchen converge
36. -----> Starting Kitchen (v1.4.2) /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/httpclient-2.6.0.1/lib/httpclient/webagent-cookie.rb:458: warning: already initialized constant HTTPClient::CookieManager /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/httpclient-2.6.0.1/lib/httpclient/cookie.rb:8: warning: previous definition of CookieManager was here -----> Creating <client-ubuntu-1404>... Sending build context to Docker daemon 63.49 kB Sending build context to Docker daemon Step 0 : FROM ubuntu:14.04 ---> ca4d7b1b9a51 Step 1 : RUN dpkg-divert --local --rename --add /sbin/initctl ---> Running in 5cd1cebf6812 Leaving 'local diversion of /sbin/initctl to /sbin/initctl.distrib' ---> 10d23f2f5ec2 Removing intermediate container 5cd1cebf6812 Step 2 : RUN ln -sf /bin/true /sbin/initctl ---> Running in 33a8c8ddf8f5 ---> 64fbee7d40c3 Removing intermediate container 33a8c8ddf8f5 Step 3 : ENV DEBIAN_FRONTEND noninteractive ---> Running in f232fdc644fd
37. Lab 9 - Add InSpec Verification Add an InSpec test chef@node$ mkdir -p ~/cookbooks/ssh/test/integration/client/ inspec
38. 1 of 2 Open file: ~/cookbooks/ssh/test/integration/client/inspec/ client_spec.rb control 'ssh-4' do impact 1.0 title 'Client: Set SSH protocol version to 2' desc " Set the SSH protocol version to 2. Don't use legacy insecure SSHv1 connections anymore. "
39. 2 of 2 Open file: ~/cookbooks/ssh/test/integration/client/inspec/ client_spec.rb describe ssh_config do its('Protocol') { should eq('2') } end end Copy-and-paste the control from Chef Compliance, we want to test the same thing.
40. Run InSpec from the Command Line » InSpec is an executable application. » InSpec can execute on remote hosts, including docker containers
41. » What is your docker container id? chef@ndoe$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 511b3fcb2777 af3815cee160:latest "/usr/sbin/sshd -D - 10 seconds ago Up 10 seconds 0.0.0.0:32773->22/tcp silly_davinci
42. chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://container_id
43. chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://88e57e403cd1 F Failures: 1) SSH Configuration Protocol should eq "2" Failure/Error: its('Protocol') { should eq('2') } expected: "2" got: nil (compared using ==) # ./test/integration/client/inspec/client_spec.rb:9:in `block (3 levels) in load' Finished in 0.2124 seconds (files took 0.37134 seconds to load) 1 example, 1 failure Failed examples: rspec # SSH Configuration Protocol should eq "2"
44. Update the template Open file: ~/cookbooks/ssh/templates/default/ssh_config.erb Change # Protocol 2,1 To Protocol 2
45. Apply the change to the Test Kitchen chef@node$ cd ~/cookbooks/ssh chef@node$ kitchen converge -----> Converging <client-ubuntu-1404>... ... Converging 1 resources Recipe: ssh::client * template[/etc/ssh/ssh_config] action create - update content in file /etc/ssh/ssh_config from 6005ad to ed645d --- /etc/ssh/ssh_config 2015-08-18 02:14:14.000000000 +0000 +++ /etc/ssh/.ssh_config20151202-416-1p3jnk3 2015-12-02 07:35:17.734336383 +0000 @@ -37,7 +37,7 @@ # IdentityFile ~/.ssh/id_rsa # IdentityFile ~/.ssh/id_dsa # Port 22 -# Protocol 2,1 + Protocol 2 # Cipher 3des # Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc # MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160 Running handlers: Running handlers complete Chef Client finished, 1/1 resources updated in 01 seconds Finished converging <client-ubuntu-1404> (0m3.52s). -----> Kitchen is finished. (0m4.20s)
46. Run InSpec from the Command Line chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://container_id
47. chef@node$ inspec exec ~/cookbooks/ssh/test/integration/ client/inspec/client_spec.rb -t docker://511b3fcb2777 . Finished in 0.18559 seconds (files took 0.36615 seconds to load) 1 example, 0 failures
48. Lab 10 - Apply the new SSH Policy Use chef-client to apply the new SSH policy chef@node$ cd ~/
49. chef@node$ sudo chef-client --local-mode -r 'recipe[ssh::client]' Starting Chef Client, version 12.5.1 resolving cookbooks for run list: ["ssh::client"] Synchronizing Cookbooks: - ssh (0.1.0) Compiling Cookbooks... Converging 1 resources Recipe: ssh::client * template[/etc/ssh/ssh_config] action create - update content in file /etc/ssh/ssh_config from 6005ad to ed645d --- /etc/ssh/ssh_config 2015-12-02 04:14:38.422336383 +0000 +++ /etc/ssh/.ssh_config20151202-14917-s8hgi2 2015-12-02 07:42:23.330336383 +0000 @@ -37,7 +37,7 @@ # IdentityFile ~/.ssh/id_rsa # IdentityFile ~/.ssh/id_dsa # Port 22 -# Protocol 2,1 + Protocol 2 # Cipher 3des # Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc # MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160 Running handlers: Running handlers complete Chef Client finished, 1/1 resources updated in 01 seconds
50. Lab 11 - Scan the SSH configuration » From the Dashboard, select your node » Click the 'Scan' button
51. » Select only the base/ssh compliance profile
52. » Press the 'Scan now' button
53. Lab 12 - Review the Compliance Report Verify the 'Client: Set SSH protocol version to 2' control is now 'Compliant'
55. Lab 13 - Create a Custom Compliance profile » Compliance Profiles are built using InSpec
56. Create a directory for your profiles chef@node$ mkdir -p ~/compliance_profiles chef@node$ cd ~/compliance_profiles
57. Is it a profile yet? » InSpec includes a command to check a profile chef@node mkdir -p USERNAME/tmp_profile_for_USERNAME
58. chef@node$ inspec check USERNAME/tmp_profile_for_USERNAME E, [2015-12-02T07:58:03.197043 #15448] ERROR -- : Can't find metadata file username/tmp_profile_for_USERNAME/metadata.rb /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/inspec-0.9.2/lib/inspec/targets/folder.rb:23:in `resolve': Don't know how to handle folder username/tmp_profile_for_USERNAME/ (RuntimeError) ...
59. Create the metadata file chef@node$ touch USERNAME/tmp_profile_for_USERNAME/ metadata.rb chef@node$ inspec check USERNAME/tmp_profile_for_USERNAME I, [2015-12-02T08:02:19.171190 #15458] INFO -- : Checking profile in USERNAME/tmp_profile_for_USERNAME E, [2015-12-02T08:02:19.171445 #15458] ERROR -- : No profile name defined W, [2015-12-02T08:02:19.171558 #15458] WARN -- : No version defined W, [2015-12-02T08:02:19.171676 #15458] WARN -- : No title defined W, [2015-12-02T08:02:19.171770 #15458] WARN -- : No maintainer defined /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/inspec-0.9.2/lib/inspec/profile.rb:104:in `check': undefined method `empty?' for nil:NilClass (NoMethodError)
60. Address the ERRORs and WARNs Open file: ~/compliance_profiles/USERNAME/ tmp_profile_for_USERNAME/metadata.rb name 'USERNAME/tmp_profile_for_USERNAME' version '0.1.0' title '/tmp Profile for USERNAME' maintainer 'YOUR NAME'
61. Create a directory for the controls chef@node$ mkdir -p USERNAME/tmp_profile_for_USERNAME/test
62. Open file: ~/compliance_profiles/USERNAME/ tmp_profile_for_USERNAME/test/tmp.rb title '/tmp profile for USERNAME' control "tmp-1.0" do # A unique ID for this control impact 0.3 # The criticality, if this control fails. title "Create /tmp directory" # A human-readable title desc "A /tmp directory must exist" describe file('/tmp') do # The actual test it { should be_directory } end end control "tmp-1.1" do impact 0.3 title "/tmp directory is owned by the root user" desc "The /tmp directory must be owned by the root user" describe file('/tmp') do it { should be_owned_by 'root' } end end
63. chef@node$ inspec check USERNAME/tmp_profile_for_USERNAME I, [2015-12-02T08:42:30.432722 #16567] INFO -- : Checking profile in USERNAME/tmp_profile_for_USERNAME I, [2015-12-02T08:42:30.432937 #16567] INFO -- : Metadata OK. D, [2015-12-02T08:42:30.433053 #16567] DEBUG -- : Found 20 rules. D, [2015-12-02T08:42:30.433154 #16567] DEBUG -- : Verify all rules in USERNAME/tmp_profile_for_USERNAME/test/tmp.rb I, [2015-12-02T08:42:30.433256 #16567] INFO -- : Rule definitions OK.
64. Test the profile locally chef@node$ inspec exec USERNAME/tmp_profile_for_USERNAME/ .. Finished in 0.0166 seconds (files took 0.24531 seconds to load) 2 examples, 0 failures
65. Zip up the profile chef@node$ cd ~/compliance_profiles chef@node$ zip -r USERNAME.zip USERNAME
66. SCP the .zip file to your LOCAL computer $ scp chef@IPADDRESS:~/compliance_profiles/USERNAME.zip . chef@52.90.226.238's password: USERNAME.zip 100% 1257 1.2KB/s 00:00
67. Lab 14 - Upload the profile to Chef Compliance » From the Compliance page, press the 'Add profile' button
68. » Select the USERNAME.zip file you created
69. Lab 15 - Scan with the new profile » From the Dashboard, select your node » Click the 'Scan' button
70. » Select only the chef/tmp_profile_for_USERNAME compliance profile
71. » Press the 'Scan now' button
72. Lab 16 - Review the Compliance Report
73. Chef Compliance - Scan Scan your entire infrastructure for risks and compliance issues
74. Chef Compliance - Report Get reports on risks and issues classified by severity and impact levels
75. Chef Compliance - Automate Build automated compliance testing and remediation into your pipeline
76. Chef Compliance - Audit Get started quickly with pre-built profiles for CIS, Linux and Windows
77. Objectives By the end of this workshop you will be able to: » Describe the capabilities of Chef Compliance » Configure Chef Compliance to scan nodes in your environment » Write custom Compliance policies using InSpec » Upload custom Compliance policies to your Chef Compliance server » Use InSpec-based compliance checks in your cookbook
78. What's next? » What questions can I answer for you? » Remediate more failures » Add additional tests
79. Thank You! Nathen Harvey - nharvey@chef.io @nathenharvey