Serverless Ops - Bye-bye Bash, Hello Go!

Learn how to use Go to automate tasks like security incident response, disaster recovery plan testing, and test environment population – all without running a single server! By creating AWS Lambda functions using Go you can minimize your own management overhead while creating tools that are more readable, fault tolerant, testable, and performant than shell scripts.

In this talk I walk through various common operations tasks and show how Go and AWS Lambda can be the two of the most flexible tools in your tool box. I also show how to get started quickly to solve almost any operations issue while still maintaining good software development practices.

Transcript

1. © 2020, Amazon Web Services, Inc. or its Affiliates. Rob Sutter GoDays Berlin – 23 January 2020 Serverless Ops Bye-bye Bash, Hello Go!
2. © 2020, Amazon Web Services, Inc. or its Affiliates. What’s in it for you? • Use the power of serverless to keep the rest of your resources running • Example of testing your apps ops • A pattern for orchestrating IT operations workflows in the cloud • A complete serverless IT operations workflow with code in Go • Q&A
3. © 2020, Amazon Web Services, Inc. or its Affiliates. Agenda • Where’s my shell?! • But there are still servers! • Why serverless for ops? • Why choose Go for ops? • Testing our apps ops • General pattern • Example – AWS Backup DynamoDB rotator • Q&A
4. © 2020, Amazon Web Services, Inc. or its Affiliates. Who am I? Rob Sutter – robsut@amazon.com, @rts_rob • Senior Developer Advocate – Serverless • Gopher (and Scala type – shhh…) • Previously: • Co-founded WorkFone, a SaaS startup • Infrastructure at Le Tote, an e-commerce startup • Consulting, government, odd jobs here and there • The Florida State University, Management Information Systems ‘05
5. © 2020, Amazon Web Services, Inc. or its Affiliates. Where’s my shell?!
6. © 2020, Amazon Web Services, Inc. or its Affiliates. Where’s my shell?! • Instances are hidden behind bastion servers and SSH tunnels • Containers are even trickier with sidecars and other workarounds • With managed services the shell concept doesn’t even exist! Photo by Ahmed Sobah on Unsplash
7. © 2020, Amazon Web Services, Inc. or its Affiliates. But there are still servers!
8. © 2020, Amazon Web Services, Inc. or its Affiliates. No infrastructure provisioning, no management Automatic scaling Pay for value Highly available and secure What is serverless?
9. © 2020, Amazon Web Services, Inc. or its Affiliates. Why serverless for ops?
10. © 2020, Amazon Web Services, Inc. or its Affiliates. Comparison of operational responsibility AWS Lambda Serverless functions AWS Fargate Serverless containers ECS/EKS Container-management as a service EC2 Infrastructure-as-a-Service More opinionated Less opinionated AWS manages Customer manages • Data source integrations • Physical hardware, software, networking, and facilities • Provisioning • Application code • Container orchestration, provisioning • Cluster scaling • Physical hardware, host OS/kernel, networking, and facilities • Application code • Data source integrations • Security config and updates, network config, management tasks • Container orchestration control plane • Physical hardware software, networking, and facilities • Application code • Data source integrations • Work clusters • Security config and updates, network config, firewall, management tasks • Physical hardware software, networking, and facilities • Application code • Data source integrations • Scaling • Security config and updates, network config, management tasks • Provisioning, managing scaling and patching of servers
11. © 2020, Amazon Web Services, Inc. or its Affiliates. Why serverless for ops? • Operations and automation tasks are irregularly executed • Runs in same the platform as your infrastructure • Takes advantage of service integrations • Fine-grained access permissions • Logging
12. © 2020, Amazon Web Services, Inc. or its Affiliates. Why choose Go for ops?
13. © 2020, Amazon Web Services, Inc. or its Affiliates. Why choose Go for ops? • Long support life • Concurrency • Type safety • Developer joy!
14. © 2020, Amazon Web Services, Inc. or its Affiliates. Testing our apps ops
15. © 2020, Amazon Web Services, Inc. or its Affiliates. Write real tests • Use dependency injection to mock cloud services
16. © 2020, Amazon Web Services, Inc. or its Affiliates. General pattern
17. © 2020, Amazon Web Services, Inc. or its Affiliates. State machine-based workflow management Initiate ops event, e.g., backup, restore, scale Check for completion Sleep for a defined period with back-off Return to sleep state or notify on completion
18. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Step Functions • Coordinate multiple services into serverless workflows • Like the func main() of a procedural serverless app • Error handling • Retries and backoff
19. © 2020, Amazon Web Services, Inc. or its Affiliates. Example AWS Backup DynamoDB rotator
20. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Backup DynamoDB rotator
21. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Backup DynamoDB rotator https://github.com/awslabs/aws-backup-dynamodb-rotator AWS Cloud AWS Step Functions AWS Lambda Lambda function Amazon DynamoDB AWS Backup Amazon Simple Notification Service Amazon DynamoDB
22. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Backup DynamoDB rotator https://github.com/awslabs/aws-backup-dynamodb-rotator
23. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Backup DynamoDB rotator https://github.com/awslabs/aws-backup-dynamodb-rotator
24. © 2020, Amazon Web Services, Inc. or its Affiliates. Q&A Rob Sutter @rts_rob
25. © 2020, Amazon Web Services, Inc. or its Affiliates. Thank you!