Going Serverless with Python

I have a confession to make, I don't fully grok AWS and all its services. But that doesn't stop me from shipping code to production on my own! In this introductory talk, I'll show some tools you can use to automate the deployment of a Python app to the cloud, without ever worrying about managing or scaling servers.

Companion code: https://github.com/marcaube/confoo-2021-serverless-python

Transcript

2. Today's plan 1. What is serverless? 2. A few options in Python 3. Serverless Hello World 4. Event-driven service 5. Where do I go from there?
3. What is serverless? (and why should I care?)
4. Function as a Service (FaaS), a.k.a Serverless ‣ reduced time to production ‣ ease of maintenance ‣ on-demand scaling ‣ pay-per-use
5. "No server is easier to manage than no server." — Werner Vogel, Amazon CTO
6. What kind of workload is right for Serverless?
7. What kind of workload is right for Serverless? ‣ Chat bots & IoT ‣ Client-heavy applications ‣ Batch processing ‣ Stateless microservices
8. What's not a good fit?
9. What's not a good fit? ‣ Long-running background jobs ‣ Memory-bound computing ‣ Apps with high-perf requirements (e.g. trading)
10. Python's Serverless Landscape
11. Current Best Practices
12. Current Best Practices ‣ Assumes you are familiar with AWS ‣ Difficult for a dev to get started ‣ Focused on the infrastructure to run the code
13. What we'll do instead ‣ Leverage frameworks to do the heavy lifting ‣ Focus on the code while generating the infrastructure ‣ Learn AWS by using the services
14. Serverless Framework
15. Zappa
16. Chalice
17. Hello World!
18. Add user
19. Set permissions
20. Review your choices
21. Download your Acess key and Secret
22. Install the AWS CLI1 $ brew install awscli 1 https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html
23. Configure the AWS cli $ aws configure AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: ca-central-1 Default output format [None]: json
24. Configure the AWS cli (alternate) # ~/.aws/credentials [chalice-hello] aws_access_key_id=AKIAIOSFODNN7EXAMPLE aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
25. Install Chalice $ pip install chalice
26. Create a project $ chalice new-project helloworld
27. $ tree -a . ├── .chalice │ └── config.json ├── .gitignore ├── app.py └── requirements.txt 1 directory, 4 files
28. app.py
29. Export your AWS profile $ export AWS_PROFILE=chalice-hello
32. hello-world-dev IAM role { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "arn:*:logs:*:*:*" } ] }
38. Let's make something useful!
39. The plan 1. event-driven instead of REST 2. read/write to S3 bucket 3. process images using a Python library
40. Create a bucket $ aws s3 mb s3://confoo2021-hello-serverless
41. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w') as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
42. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w') as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
43. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w') as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
44. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w') as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
45. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w') as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
46. @app.on_s3_event( bucket='confoo2021-hello-serverless', events=['s3:ObjectCreated:Put'], prefix='images/', suffix='.jpg' ) def resize_image(event): with tempfile.NamedTemporaryFile('w') as f: s3.download_file(event.bucket, event.key, f.name) im = Image.open(f.name) im.thumbnail((250, 250)) im.save(f.name, "JPEG", quality=80) s3.upload_file(f.name, event.bucket, f'thumbnails/{event.key}')
47. Deploy the new function $ chalice deploy Creating deployment package. Updating policy for IAM role: hello-world-dev Creating lambda function: hello-world-dev-resize_image Configuring S3 events in bucket mybucket to function hello-world-dev-resize_image Updating lambda function: hello-world-dev Updating rest API Resources deployed: - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev-resize_image - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev - Rest API URL: https://y25aa7x0qg.execute-api.ca-central-1.amazonaws.com/api/
48. Deploy the new function $ chalice deploy Creating deployment package. Updating policy for IAM role: hello-world-dev Creating lambda function: hello-world-dev-resize_image Configuring S3 events in bucket mybucket to function hello-world-dev-resize_image Updating lambda function: hello-world-dev Updating rest API Resources deployed: - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev-resize_image - Lambda ARN: arn:aws:lambda:ca-central-1:867045000300:function:hello-world-dev - Rest API URL: https://y25aa7x0qg.execute-api.ca-central-1.amazonaws.com/api/
49. Upload an image $ aws s3 cp truck.jpg s3://confoo2021-hello-serverless/images/truck.jpg
50. Check the corresponding thumbnail
51. Our Architecture
52. Going Further... ‣ custom domain ‣ security ‣ testing ‣ observability (logs, metrics, error tracking) ‣ keeping your function warm to avoid cold-starts ‣ going beyond the execution time limit
53. Example code
54. Slides
55. Links ‣ Github repository with code from this talks ‣ Chalice website and Documentation ‣ A sample TODO application
56. Thank you!