Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Going Serverless with Python

5e6bcf291601ee2e0faf35b30a839cb6?s=47 Marc Aubé
February 24, 2021

Going Serverless with Python

I have a confession to make, I don't fully grok AWS and all its services. But that doesn't stop me from shipping code to production on my own! In this introductory talk, I'll show some tools you can use to automate the deployment of a Python app to the cloud, without ever worrying about managing or scaling servers.

Companion code: https://github.com/marcaube/confoo-2021-serverless-python

5e6bcf291601ee2e0faf35b30a839cb6?s=128

Marc Aubé

February 24, 2021
Tweet

Transcript

  1. None
  2. Today's plan 1. What is serverless? 2. A few options

    in Python 3. Serverless Hello World 4. Event-driven service 5. Where do I go from there?
  3. What is serverless? (and why should I care?)

  4. Function as a Service (FaaS), a.k.a Serverless ‣ reduced time

    to production ‣ ease of maintenance ‣ on-demand scaling ‣ pay-per-use
  5. "No server is easier to manage than no server." —

    Werner Vogel, Amazon CTO
  6. What kind of workload is right for Serverless?

  7. What kind of workload is right for Serverless? ‣ Chat

    bots & IoT ‣ Client-heavy applications ‣ Batch processing ‣ Stateless microservices
  8. What's not a good fit?

  9. What's not a good fit? ‣ Long-running background jobs ‣

    Memory-bound computing ‣ Apps with high-perf requirements (e.g. trading)
  10. Python's Serverless Landscape

  11. Current Best Practices

  12. Current Best Practices ‣ Assumes you are familiar with AWS

    ‣ Difficult for a dev to get started ‣ Focused on the infrastructure to run the code
  13. What we'll do instead ‣ Leverage frameworks to do the

    heavy lifting ‣ Focus on the code while generating the infrastructure ‣ Learn AWS by using the services
  14. Serverless Framework

  15. Zappa

  16. Chalice

  17. Hello World!

  18. Add user

  19. Set permissions

  20. Review your choices

  21. Download your Acess key and Secret

  22. Install the AWS CLI1 $ brew install awscli 1 https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html

  23. Configure the AWS cli $ aws configure AWS Access Key

    ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: ca-central-1 Default output format [None]: json
  24. Configure the AWS cli (alternate) # ~/.aws/credentials [chalice-hello] aws_access_key_id=AKIAIOSFODNN7EXAMPLE aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

  25. Install Chalice $ pip install chalice

  26. Create a project $ chalice new-project helloworld

  27. $ tree -a . ├── .chalice │ └── config.json ├──

    .gitignore ├── app.py └── requirements.txt 1 directory, 4 files
  28. app.py

  29. Export your AWS profile $ export AWS_PROFILE=chalice-hello

  30. None
  31. None
  32. hello-world-dev IAM role { "Version": "2012-10-17", "Statement": [ { "Effect":

    "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:*:logs:*:*:*" } ] }
  33. None
  34. None
  35. None
  36. None
  37. None
  38. Let's make something useful!

  39. The plan 1. event-driven instead of REST 2. read/write to

    S3 bucket 3. process images using a Python library
  40. Create a bucket $ aws s3 mb s3://confoo2021-hello-serverless

  41. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  42. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  43. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  44. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  45. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  46. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w')

    as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
  47. Deploy the new function $ chalice deploy Creating deployment package.

    Updating policy for IAM role: hello-world-dev Creating lambda function: hello-world-dev-resize_image Configuring S3 events in bucket mybucket to function hello-world-dev-resize_image Updating lambda function: hello-world-dev Updating rest API Resources deployed: - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev-resize_image - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev - Rest API URL: https://y25aa7x0qg.execute-api.ca-central-1.amazonaws.com/api/
  48. Deploy the new function $ chalice deploy Creating deployment package.

    Updating policy for IAM role: hello-world-dev Creating lambda function: hello-world-dev-resize_image Configuring S3 events in bucket mybucket to function hello-world-dev-resize_image Updating lambda function: hello-world-dev Updating rest API Resources deployed: - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev-resize_image - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev - Rest API URL: https://y25aa7x0qg.execute-api.ca-central-1.amazonaws.com/api/
  49. Upload an image $ aws s3 cp truck.jpg s3://confoo2021-hello-serverless/images/truck.jpg

  50. Check the corresponding thumbnail

  51. Our Architecture

  52. Going Further... ‣ custom domain ‣ security ‣ testing ‣

    observability (logs, metrics, error tracking) ‣ keeping your function warm to avoid cold-starts ‣ going beyond the execution time limit
  53. Example code

  54. Slides

  55. Links ‣ Github repository with code from this talks ‣

    Chalice website and Documentation ‣ A sample TODO application
  56. Thank you!