Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chef Vault

Chef Vault

Slides for a talk about Chef Vault

Joshua Timberman

June 16, 2014
Tweet

More Decks by Joshua Timberman

Other Decks in Technology

Transcript

  1. Ways of storing secrets… •Password managers (1password, lastpass, keepass) •GPG/PGP

    encrypted file •Firewall-protected databass •The Configuration Management Database
  2. Ways to Store Secrets in Chef Attributes Data Bag Items

    Encrypted Data Bag Items Chef Vault
  3. Secrets in Chef: Attributes ! # In a recipe or

    cookbook default attributes: node.default['mysql']['root_password'] = 'This-Is-Secure-Right?' ! # In a role: { "name": "mysql-server", "default_attributes": { "mysql": { "root_password": "This-is-Secure-Right?" } } }
  4. Secrets in Chef: Attributes % knife search node "mysql*password:*" -a

    mysql 1 items found ! chef-vault-examples: mysql: root_password: This-Is-Secure-Right?
  5. Secrets in Chef: Data Bag Items % knife show data_bags/secrets/mysql_root.json

    data_bags/secrets/mysql_root.json: { "id": "mysql_root", "password": "Is-This-More-Secure?" } ! % cat cookbooks/mysql/recipes/grants root_pw = data_bag_item('secrets', 'mysql_root') ! template '/etc/mysql/grants.sql' do variables :root_pw => root_pw end
  6. Secrets in Chef: Data Bag Items % knife show data_bags/secrets/mysql_root.json

    data_bags/secrets/mysql_root.json: { "id": "mysql_root", "password": "Is-This-More-Secure?" } ! % cat cookbooks/mysql/recipes/grants root_pw = data_bag_item('secrets', 'mysql_root') ! template '/etc/mysql/grants.sql' do variables :root_pw => root_pw end
  7. Secrets in Chef: Data Bag Items % knife show data_bags/secrets/mysql_root.json

    data_bags/secrets/mysql_root.json: { "id": "mysql_root", "password": "Is-This-More-Secure?" } ! % cat cookbooks/mysql/recipes/grants root_pw = data_bag_item('secrets', 'mysql_root') ! template '/etc/mysql/grants.sql' do variables :root_pw => root_pw end
  8. Secrets in Chef: Data Bag Items % knife show data_bags/secrets/mysql_root.json

    data_bags/secrets/mysql_root.json: { "id": "mysql_root", "password": "Is-This-More-Secure?" } ! % cat cookbooks/mysql/recipes/grants root_pw = data_bag_item('secrets', 'mysql_root') ! template '/etc/mysql/grants.sql' do variables :root_pw => root_pw end
  9. Problems with Data Bags •Still plain text •Still easy to

    search •Can use RBAC... •But /search doesn't have RBAC :( •Using Hosted Chef?
  10. Chef Server RBAC •Enterprise Chef has RBAC •Remove read permissions

    from certain groups/users •Remove read permissions from certain API clients (nodes)
  11. Secrets in Chef: Encrypted Data Bags •Added in Chef 10

    •Data bag items encrypted at rest •Encrypted with a secret (phrase or file) •Hosted Chef is the primary use case
  12. Encrypted Data Bags: Local files % cat secret-key -----BEGIN RSA

    PRIVATE KEY—— ... some stuff in a random key i made for examples -----END RSA PRIVATE KEY----- ! % knife show data_bags/secrets/mysql_root.json data_bags/secrets/mysql_root.json: { "id": "mysql_root", "password": "Now-we-re-getting-somewhere!" }
  13. Encrypted Data Bags: Knife % knife data bag from file

    -h knife data bag from file BAG FILE|FOLDER [FILE|FOLDER..] (options) -s, --secret The secret key to use to encrypt data bag item values --secret-file SECRET_FILE A file containing the secret key to use to encrypt data bag item values
  14. Encrypted Data Bags: Upload % knife data bag from file

    secrets \ data_bags/secrets/mysql_root.json --secret-file secret-key ! Updated data_bag_item[secrets::mysql_root]
  15. Encrypted Data Bags: Show % knife show data_bags/secrets/mysql_root.json data_bags/secrets/mysql_root.json: {

    "id": "mysql_root", "password": { "encrypted_data": "aCztoQZuBvPQopBgxY32AjAVQ8QXxcaj0VkuIxO/ YZxkVNgClcJv3RpDvLFF\nZT6s\n", "iv": "nYTlXs9tCMOQw2ikgXCwrA==\n", "version": 1, "cipher": "aes-256-cbc" } }
  16. Encrypted Data Bags: Decrypt % knife data bag show secrets

    mysql_root \ --secret-file secret-key id: mysql_root password: Now-we-re-getting-somewhere!
  17. Wait... how do we get the secret? % grep secret

    /etc/chef/client.rb encrypted_data_bag_secret "/etc/chef/encrypted_data_bag_secret"
  18. The secret must be present... •Built into a machine image

    •Copied with SCP/Rsync •Put into place with knife bootstrap...
  19. Problems with Encrypted Items... •They're encrypted, so that's better •But

    now we have a key distribution problem • how do we share the secret key with others? • oops. •There's no centralized point of control •The key lives on the systems that need it...
  20. "That's why storing encryption keys on the same system where

    the protected data resides violates all of the core principles of data protection." ! - Patrick Townsend Townsend Security ! http://web.townsendsecurity.com/bid/23881/PCI-DSS-2-0- and-Encryption-Key-Management
  21. How do you manage secrets with Chef? attributes... data bags...

    encrypted data bags... Dennis Hill https://flic.kr/p/Lzpyo
  22. How does it work? • Chef Client: Private RSA key

    (/etc/chef/client.pem) • Chef Server: Public RSA key • Chef Vault: Generates a Shared Secret • Chef Vault: Encrypts w/ Public Keys • API Clients (nodes) • Users • Search Query
  23. Chef Vault: Gem and Knife Plugin % gem install chef-vault

    % knife vault ** VAULT COMMANDS ** knife vault create VAULT ITEM VALUES (options) knife vault decrypt VAULT ITEM [VALUES] (options) knife vault delete VAULT ITEM (options) knife vault edit VAULT ITEM (options) knife vault remove VAULT ITEM VALUES (options) knife vault rotate all keys knife vault rotate keys VAULT ITEM (options) knife vault show VAULT ITEM [VALUES] (options)
  24. Chef Vault: Create % knife vault create secrets mysql_root \

    -J data_bags/secrets/mysql_root_unencrypted.json \ -A jtimberman -S 'role:mysql-server'
  25. Chef Vault: Show % knife show data_bags/secrets/mysql_root.json data_bags/secrets/mysql_root.json: { "id":

    "mysql_root", "password": { "encrypted_data": "naJm6HHz0ti7kzVJxFpwnmvR2/ u38yYinBpfUXUsERJQBwtfLjY2wnrLo0S3\nxO2G\n", "iv": "A0RjHlcOer+K0XT2BSI5KQ==\n", "version": 1, "cipher": "aes-256-cbc" } }
  26. Chef Vault: Show Keys % knife data bag show secrets

    mysql_root_keys admins: jtimberman chef-vault-examples: KqshDPU49J217p/xH0HAIy5Ik5j/cHflM0QYuhAtFOuWqUh4ZkV983148UcT r3hCjhQ9MkE+5NOz3CMRiOv5p1gGJ17HaM3Vtn6UY3AIN8A5biZHJPsO47uk AZyn0gsxzys561lmCAaZA3T/F/TGlv2dppf/iPmjtCX/84barkcO1F8qPrUt 94s4+LrVGconqQs3YmhdNKzfnO1im82CHZyv7DM2D2hmojGl6Re1K5UYbnvz Lo/Circ5+dninO8JXbBqvX1pNwbflWf1p7J+KqJvEnY2ZwortXNNN9FABH9+ 34WAzpZLVfICGvKuMSrofRGDfXa9Vp7Aw44cbxu1tQ== ! clients: chef-vault-examples id: mysql_root_keys jtimberman: dlmahCPrKrMF0Vh35rzOzT57hNAohC2iIU32Iit/TwFjMiGZSB0GGEXi6R7y oXjlJX1Dusqk8R4zoYWdxIvjcgWoyFTtpC1QM2ArOPiV2FMQNur+OB3RmbRM fTxygbF84PGxPHPd17pZP+hBvlxfv4sARUAfxVXr9uQIc9i14uVsZozIZ8KP YE305QuUwPhi/vAIf2Z8GAVN7X2qN4AnWKRq8dowCID67f8NdvXiwCzWQT7N pFjRSa2DdE2BeAq77YsLmdph3wd5Qg+uViQqcMnxMx0KHZOcUvWvkXUZjvCt U5bOdQCZYrI4mmMCftKN4XVKFzbzlbH1FEKmwDfwyw== ! search_query: role:mysql-server
  27. Chef Vault: Show (Decrypt) % knife vault show secrets mysql_root

    id: mysql_root password: Now-we-re-getting-somewhere!
  28. Chef Vault: Who Has Access? % knife search secrets 'id:mysql_root_keys'

    -a clients 1 items found ! data_bag_item_secrets_mysql_root_keys: clients: chef-vault-examples ! % knife search secrets 'id:mysql_root_keys' -a admins 1 items found ! data_bag_item_secrets_mysql_root_keys: admins: jtimberman
  29. Chef Vault: Update is Additive % knife vault update secrets

    mysql_root -S 'role:mysql- server' ! % knife search secrets 'id:mysql_root_keys' -a clients 1 items found ! data_bag_item_secrets_mysql_root_keys: clients: chef-vault-examples second-node
  30. Chef Vault: Edit % knife vault show secrets mysql_root id:

    mysql_root password: Now-we-re-getting-somewhere! ! % knife vault edit secrets mysql_root ! % knife vault show secrets mysql_root id: mysql_root password: Time-to-update-passwords?
  31. Chef Vault: Rotate % knife data bag show secrets mysql_root

    id: mysql_root password: encrypted_data: B7uY/ zhphnyd0T7OVFRC0n0PNneCBG5Gcx3MbN6KYwS0MiAfQT9wAyuWzG2EKReB !
  32. Chef Vault: Rotate % knife data bag show secrets mysql_root

    id: mysql_root password: encrypted_data: B7uY/ zhphnyd0T7OVFRC0n0PNneCBG5Gcx3MbN6KYwS0MiAfQT9wAyuWzG2EKReB ! % knife vault rotate keys secrets mysql_root
  33. Chef Vault: Rotate % knife data bag show secrets mysql_root

    id: mysql_root password: encrypted_data: ZR73Skl9CntebESAPyj8MxLuLb0TXDj/6A/PSUar +eqyDy4wKW8Bvknv8Ckr+EJ7 ! % knife vault show secrets mysql_root id: mysql_root password: Time-to-update-passwords?
  34. chef-vault Cookbook # in your cookbook's metadata depends 'chef-vault' !

    # in your recipe include_recipe 'chef-vault' ! root_pw = chef_vault_item('secrets', 'mysql_root')
  35. Chef Servers Generate Secrets ubuntu@chef-server:~$ sudo chef-server-ctl reconfigure ... *

    execute[/opt/chef-server/embedded/bin/psql --port 5432 -d 'opscode_chef' -c "CREATE USER opscode_chef WITH SUPERUSER ENCRYPTED PASSWORD '0f3ca56ae76af84181'"] action run * template[/var/opt/chef-server/erchef/etc/app.config] action create + %% Database connection parameters + {db_host, "127.0.0.1"}, + {db_port, 5432}, + {db_user, "opscode_chef"}, + {db_pass, "0f3ca56ae76af84181"}, + {db_name, "opscode_chef" }, + {idle_check, 10000},
  36. /etc/chef-server/chef-server-secrets.json ubuntu@chef-server:~$ sudo grep sql_password /etc/chef- server/chef-server-secrets.json "sql_password": "0f3ca56ae76af84181", !

    ubuntu@chef-server:~$ ls /etc/chef-server/ admin.pem chef-server-running.json chef-server- secrets.json chef-validator.pem chef-webui.pem ! ubuntu@chef-server:~$ scp -r /etc/chef-server root@chef- server-replica
  37. chef_vault_secret resource data = JSON.parse(IO.read('/etc/chef-server/chef-server- secrets.json')) ! chef_vault_secret 'store-chef-server-secrets' do

    data_bag 'chef-server-secrets' raw_data 'data' => data admins 'jtimberman' search 'role:chef-server' end
  38. Generate and store a password... require 'secure_random' secure_pw = SecureRandom.hex(48)

    ! chef_vault_secret 'mysql_root' do data_bag 'secrets' raw_data 'password' => secure_pw admins 'jtimberman' search 'role:mysql_server' end
  39. Share Secrets with a Coworker % cat secret_data.json { "id":

    "secret_data", "data": { "file_contents": "---- BEGIN SUPER SECRET FILE...." }, } ! % knife vault create secrets secret_data \ -J secret_data.json -A your_coworker
  40. Good Ideas •Configure knife (knife[:vault_mode] = 'client') •Restrict access as

    needed (by user, node) •Rotate keys when necessary • on a schedule, when admins leave, etc. •Use the chef-vault cookbook!
  41. Chef Vault Benefits •Encrypted data bag item •No "shared secret"

    to copy •Flexible library and tooling •Easily revoke access to items •Easily rotate the shared secret •Chef Software uses this in production
  42. Things to Know •Chef Vault 3 in the works •Citadel:

    an alternative for AWS / S3 • https://github.com/poise/citadel •Cheffish & Chef Metal • https://github.com/opscode/cheffish • https://github.com/opscode/chef-metal