Automating AWS for fun and profit

Now days most people find themselves running multiple AWS accounts, supporting their development and operations. With our explosive growth to over 100 plus AWS accounts, there have been new challenges;
Examples being: Instance tagging, Rightsizing, Unused resources and the one that tops itself almost weekly Security!

The presentation will aim to share experience and background working with open source tools, the newest Cloud-Custodian to tackle said issues via automation and give back time to the challenges you'd rather be focusing on!

Transcript

1. Automating AWS for fun and profit Andrew Boundy - Security Engineer @SEEK.com.au
2. Problems • One Hundred AWS Accounts • Developers have full admin access • Ownership = ??
4. We Automate!
5. Cloud Custodian A rules engine for infrastructure management. Defined policies to enable well managed cloud infrastructure, that's both secure and cost optimized. ➢ ➢ ➢ ➢ ➢
6. OK, So what’s the use case? DevOps ➢ Tagging ➢ Rightsizing ➢ Service Limit ➢ Garbage Collection Security ➢ Compliance ➢ Data Security ➢ Public Resources
7. Devops
8. Auto Tagging Tag any new instances missing an owner tag - name: ec2-auto-tag-user resource: ec2 mode: type: cloudtrail role: role/custodian events: - RunInstances filters: - "tag:Owner": not-null actions: - type: auto-tag-user tag: Owner principal_id_tag: CreatorId
9. Missing Tags Stop all instances missing tags after two days - name: ec2-tag-compliance-mark resource: ec2 filters: - "tag:c7n_status": absent - "tag:Owner": absent - "tag:CostCenter": absent - "tag:Project": absent actions: - type: mark-for-op op: stop days: 2
10. Service Limits Find any service at more than 50% limit - name: account-service-limits resource: account filters: - type: service-limit threshold: 70 actions: - type: request-limit-increase percent-increase: 25
11. Rightsizing Find databases using over 90% of their allocated storage, and resize them to have an additional 30% storage - name: rds-storage-resize resource: rds filters: - type: metrics name: FreeStorageSpace percent-attr: AllocatedStorage attr-multiplier: 1073741824 value: 90 op: greater-than actions: - type: resize percent: 30
12. Unused Resources Mark any unattached EBS volumes for deletion in 30 days - name: ebs-mark-unattached-deletion resource: ebs filters: - Attachments: [] - "tag:maid_status": absent actions: - type: mark-for-op op: delete days: 30
13. Security
14. Compliance Terminate all non encrypted EBS volumes upon creation - name: ebs-terminate-unencrypted resource: ebs mode: type: cloudtrail events: - CreateVolume filters: - Encrypted: false actions: - delete
15. Data Security Remove s3 bucket permissions, public without tags - name: s3-remove-public resource: s3 filters: - type: global-grants - "tag:Public": absent actions: - type: delete-global-grants
16. Products with Public Access ➢ ➢ ➢ ➢ ➢ ➢ ➢
17. Supporting Features
18. Reporting ➢ ➢ ➢ ➢
19. Notifications!
20. Q&A