Real use-cases of using AWS Lambda for infrastructure tasks

In this talk, we will look at the application of the serverless approach on the example of AWS Lambda to solve various practical problems related to the infrastructure. It will cover both the practical application of AWS Lambda and the issues of testing, deployment and monitoring of serverless applications.

Examples of problems that we solve Lambda’oh:
– Keeping the AMI up to date (solution similar to Packer from HashiCorp, but to our specifications).
– Sending feedback from CI to code review.
– Disable unused EC2 instances.
– Microservice for collecting metrics, etc.

Transcript

1. !1 Artem Nikitin Software Engineer @elastic Real usage of AWS Lambda for infrastructure tasks
2. !2 Agenda What are we talking about today? What is serverless / AWS Lambda? 1 Why and how we use it 2 DevOps for Lambda 3
3. !3
4. !4 https://thenewstack.io/ebooks/serverless/guide-to-serverless-technologies
5. !5 History • 2012
 Why The Future Of Software And Apps Is Serverless 
 https://readwrite.com/2012/10/15/why-the-future-of-software-and-apps-is- serverless/ • 2014
 AWS announced Lambda
 https://aws.amazon.com/blogs/aws/run-code-cloud/ Or how it all started
6. !6 Serverless • PaaS 2.0 ?! • FaaS
 • A Serverless solution is one that costs you nothing to run if nobody is using it (excluding data storage) What is it?
7. !7 Lambda • Ideal example of 12 factor app • New pricing model • New approach to infrastructure/ops It's not that new actually
8. !8 Lambda • Autoscaling • Load balancing • Fault tolerance • Security Not just an environment
9. !9 Lambda • Logs (via CloudWatch) • Tracing (via X-Ray) • Monitoring Not just an infrastructure
10. !10 Lambda • Supports C#, NodeJS, Python, Java, Go, Ruby • Lambda@Edge
 • Integrated with many AWS services • HTTP via API Gateway • Run in VPC • 3rd party Lambda’s via Serverless Application Repository More technical details
11. !11
12. !12 Why Lambda? • Complex CI, mostly in AWS • We needed an infrastructure to support CI
 • https://2018.heisenbug-piter.ru/en/talks/2018/spb/ 4wqvzujdickc2awoms6kgi/ 
 • https://devopsconf.io/moscow/2018/abstracts/3735
13. !13 Our use cases • Posting feedback from CI to code review
14. !14
15. !15 Our use cases • Posting feedback from CI to code review • Updating AMI (like Packer from HashiCorp but with our specifics)
16. !16
17. !17 Our use cases • Posting feedback from CI to code review • Updating AMI (like Packer from HashiCorp but with our specifics) • Collecting metrics from test apps
18. !18
19. !19 Our use cases • Posting feedback from CI to code review • Updating AMI (like Packer from HashiCorp but with our specifics) • Collecting metrics from test apps • And much more smaller tasks
20. !20 What else can be done • Usual things like microservices • Glue for AWS services
 • https://github.com/AutoSpotting/AutoSpotting • Massive scaling of E2E tests • And much more..
21. !21
22. !22 https://thenewstack.io/ebooks/serverless/guide-to-serverless-technologies
23. !23 Development • AWS SDK • Serverless Framework, APEX • https://github.com/artemnikitin/delex-code-example
24. !24 Development • Cold start problem • No connection pooling • Lambda’s from Java to Go -> 2x money savings • 250K+ requests and 180K seconds in October for 3.16 USD*
25. !25 Testing • Unit tests • AWS SAM CLI • Test in production
26. !26 Deployment • Manual • AWS SAM CLI
 • Terraform
27. !27 Deployment • Only one way for release and versioning • Many modern practices are unusable • API Gateway with separated release process
28. !28 Debugging • AWS SAM CLI • No proper solution for production :(
29. !29
30. !30 Thank you!!! hi@artemnikitin.com artemnikitin artemnikitin