Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Going Serverless with Python

Marc Aubé
February 24, 2021

Going Serverless with Python

I have a confession to make, I don't fully grok AWS and all its services. But that doesn't stop me from shipping code to production on my own! In this introductory talk, I'll show some tools you can use to automate the deployment of a Python app to the cloud, without ever worrying about managing or scaling servers.

Companion code: https://github.com/marcaube/confoo-2021-serverless-python

Marc Aubé

February 24, 2021
Tweet

More Decks by Marc Aubé

Other Decks in Programming

Transcript

  1. Today's plan 1. What is serverless? 2. A few options

    in Python 3. Serverless Hello World 4. Event-driven service 5. Where do I go from there?
  2. Function as a Service (FaaS), a.k.a Serverless ‣ reduced time

    to production ‣ ease of maintenance ‣ on-demand scaling ‣ pay-per-use
  3. What kind of workload is right for Serverless? ‣ Chat

    bots & IoT ‣ Client-heavy applications ‣ Batch processing ‣ Stateless microservices
  4. What's not a good fit? ‣ Long-running background jobs ‣

    Memory-bound computing ‣ Apps with high-perf requirements (e.g. trading)
  5. Current Best Practices ‣ Assumes you are familiar with AWS

    ‣ Difficult for a dev to get started ‣ Focused on the infrastructure to run the code
  6. What we'll do instead ‣ Leverage frameworks to do the

    heavy lifting ‣ Focus on the code while generating the infrastructure ‣ Learn AWS by using the services
  7. Configure the AWS cli $ aws configure AWS Access Key

    ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: ca-central-1 Default output format [None]: json
  8. $ tree -a . ├── .chalice │ └── config.json ├──

    .gitignore ├── app.py └── requirements.txt 1 directory, 4 files
  9. hello-world-dev IAM role { "Version": "2012-10-17", "Statement": [ { "Effect":

    "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:*:logs:*:*:*" } ] }
  10. The plan 1. event-driven instead of REST 2. read/write to

    S3 bucket 3. process images using a Python library
  11. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  12. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  13. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  14. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  15. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  16. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  17. Deploy the new function $ chalice deploy Creating deployment package.

    Updating policy for IAM role: hello-world-dev Creating lambda function: hello-world-dev-resize_image Configuring S3 events in bucket mybucket to function hello-world-dev-resize_image Updating lambda function: hello-world-dev Updating rest API Resources deployed: - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev-resize_image - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev - Rest API URL: https://y25aa7x0qg.execute-api.ca-central-1.amazonaws.com/api/
  18. Deploy the new function $ chalice deploy Creating deployment package.

    Updating policy for IAM role: hello-world-dev Creating lambda function: hello-world-dev-resize_image Configuring S3 events in bucket mybucket to function hello-world-dev-resize_image Updating lambda function: hello-world-dev Updating rest API Resources deployed: - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev-resize_image - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev - Rest API URL: https://y25aa7x0qg.execute-api.ca-central-1.amazonaws.com/api/
  19. Going Further... ‣ custom domain ‣ security ‣ testing ‣

    observability (logs, metrics, error tracking) ‣ keeping your function warm to avoid cold-starts ‣ going beyond the execution time limit
  20. Links ‣ Github repository with code from this talks ‣

    Chalice website and Documentation ‣ A sample TODO application