Secrets Management for Development & Operations

Originally presented at DevOpsDays Istanbul, 2021.

Your company tells you they want to secure your application and infrastructure with a secrets manager. How should you use it, and how will it affect your code? In this session, you’ll learn how to refactor your applications, delivery pipelines, local development, and infrastructure to accommodate rotating secrets. You'll take away the best practices and patterns for integrating with a secrets manager while minimizing interruptions to your development workflow.

Transcript

1. DevOpsDays Istanbul September 2021 
 
 @joatmon08 Secrets Management for Development & Operations 1
2. Secrets Passwords, API tokens, SSL Certificates, or any other sensitive information used to access something else. 2
3. We have a secrets manager! 3
4. It will help us securely access, store, revoke, and rotate secrets. 4
5. Immutability & ephemerality in security changes development and operations workflows. 5
6. How does it affect my development or operations workflows? 6
7. What do I need to know to adopt a secrets manager? 7
8. Developer Advocate at HashiCorp 
 she/her 
 joatmon08.github.io 
 @joatmon08 Rosemary Wang 8
9. What you need to know ▪ Organizing your secrets manager ▪ Comparing patterns for secrets injection ▪ Adding a secrets manager to your: – Delivery pipelines – Infrastructure – Local development – Applications
10. Organize your secrets manager 10
11. Authentication Try using protocols or built-in methods TARGET SECRETS MANAGER /BUSINESS/SECRET AUTHENTICATE OIDC/JWT KUBERNETES CLOUD PROVIDERS TOKENS (AVOID USERNAME/PASSWORD!) 11
12. Access Control Implement least- privilege access by: business domain, environment, and usage TARGET (DEV) SECRETS MANAGER /BUSINESS/SECRET ALLOW READ FROM /BUSINESS/DEV/SECRET ADMINISTRATORS ALLOW WRITE TO /BUSINESS/* TARGET (PROD) 12 ALLOW READ FROM /BUSINESS/PROD/SECRET
13. Reasonable TTLs You don’t need a database password rotated every five seconds. TARGET SECRETS MANAGER /BUSINESS/SECRET AUTHENTICATE GET SET RENEWAL INTERVAL TO BE AT LEAST HALF THE TTL OF THE SECRET. 13
14. 1. Use built-in authentication protocols 2. Set up access control for secrets 3. Configure a reasonable time-to-live 14
15. Compare secrets injection patterns 15
16. Direct Use script or client library. TARGET SECRETS MANAGER /BUSINESS/SECRET AUTHENTICATE GET 16
17. Sidecar Separate async process. TARGET READ FILE SECRETS MANAGER /BUSINESS/SECRET SIDECAR WRITE AUTHENTICATE GET 17
18. Add a secrets manager 18
19. Delivery Pipelines Use secrets for deployment 19
20. UNIT TESTS INTEGRATION TESTS DEPLOY TO DEV GET SECRETS FOR DEV SECRETS MANAGER /BUSINESS/SECRET INTEGRATION TESTS DEPLOY TO PROD GET SECRETS FOR PROD SECRETS MANAGER /BUSINESS/SECRET Direct Consider… • Separate by manager versus path • Only allow certain CI runners to authenticate 20
21. Infrastructure Defining secrets “as code” 21
22. CODE EDITOR resource "aws_db_instance" "products" { allocated_storage = 1 0 engine = "postgres " engine_version = "11.6 " instance_class = "db.t3.micro " name = "products " identifier = "${var.name}-products " username = var.database_usernam e password = var.database_passwor d } 22
23. PASS TO INFRASTRUCTURE AS CODE GET “BOOTSTRAP” DATABASE PASSWORD SECRETS MANAGER /BUSINESS/SECRET CONFIGURE DATABASE ROTATE “BOOTSTRAP” DATABASE PASSWORD Consider… • Configuration drift versus security • Secret storage in infrastructure as code 23
24. Local development Retrieving secrets for testing 24
25. SECRETS MANAGER /BUSINESS/SECRET SECURE ACCESS MANAGEMENT GET CREDENTIALS FROM SECRETS MANAGER LOG INTO DATABASE Consider… • Tracking human versus service access • Evaluate TTL for development testing 25
26. Applications “Inject” secrets while running 26
27. APPLICATION USES CLIENT LIBRARY TO GET SECRET SECRETS MANAGER /BUSINESS/SECRET Direct START APPLICATION NEW SECRET? RESTART APPLICATION. You must… • Implement application reload or separate thread to get secrets • Account for connection failure to secrets manager (in your application code) 27
28. Spring Cloud Add to bootstrap.yml spring.cloud.vault : authentication: APPROL E app-role : role-id: REDACTE D secret-id: REDACTE D mysql : enabled: tru e role: readonl y backend: mysq l username-property: spring.datasource.usernam e password-property: spring.datasource.password CODE EDITOR
29. .NET Example Add configuration provider public VaultConfigurationProvider(VaultOptions config ) { _config = config ; var vaultClientSettings = new VaultClientSettings ( _config.Address , new AppRoleAuthMethodInfo(_config.Role , _config.Secret ) ) ; _client = new VaultClient(vaultClientSettings) ; } CODE EDITOR
30. .NET Example Add extension method public static class VaultExtension s { public static IConfigurationBuilder AddVault ( this IConfigurationBuilder configuration , Action<VaultOptions> options ) { var vaultOptions = new VaultConfigurationSource(options) ; configuration.Add(vaultOptions) ; return configuration ; } } CODE EDITOR
31. CONFIGURATION FILE SECRETS MANAGER /BUSINESS/SECRET SIDECAR PROCESS Sidecar APPLICATION READS CONFIGURATION FROM FILE CHANGE TO FILE? RESTART APPLICATION. START APPLICATION You must… • Configure separate process • Access control to file • Configure application reload if file changes (Spring, .NET) 31
32. What you need to know ▪ Organizing a secrets manager ▪ Comparing patterns for secrets injection ▪ Adding a secrets manager to your: – Delivery pipelines – Infrastructure – Local development – Applications
33. Learn more… ▪ Delivery pipeline: youtu.be/qgF7XquqVSA ▪ Application + secrets management deep dive: youtu.be/gO4i_s0h1uo ▪ Application Client Libraries (Tutorials & Getting Started) – learn.hashicorp.com/collections/vault/app-integration – cloud.spring.io/spring-cloud-vault/reference/html/ ▪ Kubernetes + Vault – learn.hashicorp.com/tutorials/vault/kubernetes-sidecar – learn.hashicorp.com/tutorials/vault/kubernetes-secret-store-driver 33
34. joatmon08.github.io 
 Rosemary Wang 
 @joatmon08 Thank you! 34