Serverless Ops – Bye-bye Bash, Hello Go! - Rob Sutter - Amazon

Learn how to use Go to automate tasks like security incident response, disaster recovery plan testing, and test environment population – all without running a single server! By creating AWS Lambda functions using Go you can minimize your own management overhead while creating tools that are more readable, fault tolerant, testable, and performant than shell scripts.

In this talk I'll walk through various common operations tasks and show how Go and AWS Lambda can be the two of the most flexible tools in your tool box. I'll also show how to get started quickly to solve almost any operations issue while still maintaining good software development practices.

This session is for anyone looking to simplify their operations responsibilities or interested in learning more about serverless functions and Go. Basic familiarity with Go is helpful, but not required.

Transcript

1. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Rob Sutter
   GoDays Berlin – 23 January 2020
   Serverless Ops
   Bye-bye Bash, Hello Go!
   

   View Slide

2. © 2020, Amazon Web Services, Inc. or its Affiliates.
   What’s in it for you?
   • Use the power of serverless to keep the rest of your resources running
   • Example of testing your apps ops
   • A pattern for orchestrating IT operations workflows in the cloud
   • A complete serverless IT operations workflow with code in Go
   • Q&A
   

   View Slide

3. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Agenda
   • Where’s my shell?!
   • But there are still servers!
   • Why serverless for ops?
   • Why choose Go for ops?
   • Testing our apps ops
   • General pattern
   • Example – AWS Backup DynamoDB rotator
   • Q&A
   

   View Slide

4. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Who am I?
   Rob Sutter – robsut@amazon.com, @rts_rob
   • Senior Developer Advocate – Serverless
   • Gopher (and Scala type – shhh…)
   • Previously:
   • Co-founded WorkFone, a SaaS startup
   • Infrastructure at Le Tote, an e-commerce startup
   • Consulting, government, odd jobs here and there
   • The Florida State University, Management Information Systems ‘05
   

   View Slide

5. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Where’s my shell?!
   

   View Slide

6. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Where’s my shell?!
   • Instances are hidden behind
   bastion servers and SSH tunnels
   • Containers are even trickier with
   sidecars and other workarounds
   • With managed services the shell
   concept doesn’t even exist!
   Photo by Ahmed Sobah on Unsplash
   

   View Slide

7. © 2020, Amazon Web Services, Inc. or its Affiliates.
   But there are still servers!
   

   View Slide

8. © 2020, Amazon Web Services, Inc. or its Affiliates.
   No infrastructure provisioning,
   no management
   Automatic scaling
   Pay for value Highly available and secure
   What is serverless?
   

   View Slide

9. © 2020, Amazon Web Services, Inc. or its Affiliates.
   Why serverless for ops?
   

   View Slide

10. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Comparison of operational responsibility
    AWS Lambda
    Serverless functions
    AWS Fargate
    Serverless containers
    ECS/EKS
    Container-management as a service
    EC2
    Infrastructure-as-a-Service
    More opinionated
    Less opinionated
    AWS manages Customer manages
    • Data source integrations
    • Physical hardware, software, networking,
    and facilities
    • Provisioning
    • Application code
    • Container orchestration, provisioning
    • Cluster scaling
    • Physical hardware, host OS/kernel,
    networking, and facilities
    • Application code
    • Data source integrations
    • Security config and updates, network config,
    management tasks
    • Container orchestration control plane
    • Physical hardware software,
    networking, and facilities
    • Application code
    • Data source integrations
    • Work clusters
    • Security config and updates, network config,
    firewall, management tasks
    • Physical hardware software,
    networking, and facilities
    • Application code
    • Data source integrations
    • Scaling
    • Security config and updates, network config,
    management tasks
    • Provisioning, managing scaling and
    patching of servers
    

    View Slide

11. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Why serverless for ops?
    • Operations and automation tasks are irregularly executed
    • Runs in same the platform as your infrastructure
    • Takes advantage of service integrations
    • Fine-grained access permissions
    • Logging
    

    View Slide

12. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Why choose Go for ops?
    

    View Slide

13. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Why choose Go for ops?
    • Long support life
    • Concurrency
    • Type safety
    • Developer joy!
    

    View Slide

14. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Testing our apps ops
    

    View Slide

15. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Write real tests
    • Use dependency injection to mock cloud services
    

    View Slide

16. © 2020, Amazon Web Services, Inc. or its Affiliates.
    General pattern
    

    View Slide

17. © 2020, Amazon Web Services, Inc. or its Affiliates.
    State machine-based workflow management
    Initiate ops event, e.g., backup, restore, scale
    Check for completion
    Sleep for a defined period with back-off
    Return to sleep state or notify on completion
    

    View Slide

18. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Step Functions
    • Coordinate multiple services into
    serverless workflows
    • Like the func main() of a
    procedural serverless app
    • Error handling
    • Retries and backoff
    

    View Slide

19. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Example
    AWS Backup DynamoDB rotator
    

    View Slide

20. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Backup DynamoDB rotator
    

    View Slide

21. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Backup DynamoDB rotator
    https://github.com/awslabs/aws-backup-dynamodb-rotator
    AWS Cloud
    AWS Step Functions AWS Lambda
    Lambda function
    Amazon DynamoDB
    AWS Backup Amazon Simple
    Notification Service
    Amazon DynamoDB
    

    View Slide

22. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Backup DynamoDB rotator
    https://github.com/awslabs/aws-backup-dynamodb-rotator
    

    View Slide

23. © 2020, Amazon Web Services, Inc. or its Affiliates.
    AWS Backup DynamoDB rotator
    https://github.com/awslabs/aws-backup-dynamodb-rotator
    

    View Slide

24. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Q&A
    Rob Sutter
    @rts_rob
    

    View Slide

25. © 2020, Amazon Web Services, Inc. or its Affiliates.
    Thank you!
    

    View Slide