Real use-cases of using AWS Lambda for infrastructure tasks

by Artem Nikitin

Embed

Start on current slide

Slide 1

Slide 1 text

!1 Artem Nikitin Software Engineer @elastic Real usage of AWS Lambda for infrastructure tasks

Slide 2

Slide 2 text

!2 Agenda What are we talking about today? What is serverless / AWS Lambda? 1 Why and how we use it 2 DevOps for Lambda 3

Slide 3

Slide 3 text

Slide 4

Slide 4 text

!4 https://thenewstack.io/ebooks/serverless/guide-to-serverless-technologies

Slide 5

Slide 5 text

!5 History • 2012  Why The Future Of Software And Apps Is Serverless   https://readwrite.com/2012/10/15/why-the-future-of-software-and-apps-is- serverless/ • 2014  AWS announced Lambda  https://aws.amazon.com/blogs/aws/run-code-cloud/ Or how it all started

Slide 6

Slide 6 text

!6 Serverless • PaaS 2.0 ?! • FaaS  • A Serverless solution is one that costs you nothing to run if nobody is using it (excluding data storage) What is it?

Slide 7

Slide 7 text

!7 Lambda • Ideal example of 12 factor app • New pricing model • New approach to infrastructure/ops It's not that new actually

Slide 8

Slide 8 text

!8 Lambda • Autoscaling • Load balancing • Fault tolerance • Security Not just an environment

Slide 9

Slide 9 text

!9 Lambda • Logs (via CloudWatch) • Tracing (via X-Ray) • Monitoring Not just an infrastructure

Slide 10

Slide 10 text

!10 Lambda • Supports C#, NodeJS, Python, Java, Go, Ruby • Lambda@Edge  • Integrated with many AWS services • HTTP via API Gateway • Run in VPC • 3rd party Lambda’s via Serverless Application Repository More technical details

Slide 11

Slide 11 text

!11

Slide 12

Slide 12 text

!12 Why Lambda? • Complex CI, mostly in AWS • We needed an infrastructure to support CI  • https://2018.heisenbug-piter.ru/en/talks/2018/spb/ 4wqvzujdickc2awoms6kgi/   • https://devopsconf.io/moscow/2018/abstracts/3735

Slide 13

Slide 13 text

!13 Our use cases • Posting feedback from CI to code review

Slide 14

Slide 14 text

!14

Slide 15

Slide 15 text

!15 Our use cases • Posting feedback from CI to code review • Updating AMI (like Packer from HashiCorp but with our specifics)

Slide 16

Slide 16 text

!16

Slide 17

Slide 17 text

!17 Our use cases • Posting feedback from CI to code review • Updating AMI (like Packer from HashiCorp but with our specifics) • Collecting metrics from test apps

Slide 18

Slide 18 text

!18

Slide 19

Slide 19 text

!19 Our use cases • Posting feedback from CI to code review • Updating AMI (like Packer from HashiCorp but with our specifics) • Collecting metrics from test apps • And much more smaller tasks

Slide 20

Slide 20 text

!20 What else can be done • Usual things like microservices • Glue for AWS services  • https://github.com/AutoSpotting/AutoSpotting • Massive scaling of E2E tests • And much more..

Slide 21

Slide 21 text

!21

Slide 22

Slide 22 text

!22 https://thenewstack.io/ebooks/serverless/guide-to-serverless-technologies

Slide 23

Slide 23 text

!23 Development • AWS SDK • Serverless Framework, APEX • https://github.com/artemnikitin/delex-code-example

Slide 24

Slide 24 text

!24 Development • Cold start problem • No connection pooling • Lambda’s from Java to Go -> 2x money savings • 250K+ requests and 180K seconds in October for 3.16 USD*

Slide 25

Slide 25 text

!25 Testing • Unit tests • AWS SAM CLI • Test in production

Slide 26

Slide 26 text

!26 Deployment • Manual • AWS SAM CLI  • Terraform

Slide 27

Slide 27 text

!27 Deployment • Only one way for release and versioning • Many modern practices are unusable • API Gateway with separated release process