Serverless with Python

Talk at PyConES 2017

Transcript

1. Serverless with Python Look ma, no servers! Alejandro Guirao @lekum github.com/lekum lekum.org
2. What is Serverless? • No servers? • FaaS (functions as a service) • Elasticity
3. Hosted solutions
4. The pioneer: AWS Lambda S3 bucket DynamoDB table Kinesis Streams SNS Topic SES e-mail Cognito CloudFormation stack CloudWatch logs & events CodeCommit Config Alexa skill Lex API Gateway AWS Lambda Event sources
5. AWS Lambda limits
6. AWS Lambda: Manual deploy def my_handler(event, context): message = 'Hello {} {}!'.format(event['first_name'], event['last_name']) return { 'message' : message }
7. Context object get_remaining_time_in_millis() function_name function_version invoked_function_arn memory_limit_in_mb aws_request_id log_group_name log_stream_name identity client_context
8. AWS Lambda: Manual deploy aws lambda create-function \ --region eu-west-1 \ --function-name HelloPython \ --zip-file fileb://deployment-package.zip \ --role role-arn \ --handler hello_python.my_handler \ --runtime python3.6 \ --timeout 15 \ --memory-size 512
9. AWS Lambda: Manual deploy $ aws lambda invoke \ --function-name <function-arn> \ --payload '{"first_name":"Margaret", "last_name":"Hamilton"}' out.json $ cat out.json {"message": "Hello Margaret Hamilton!"}
10. Testing AWS Lambda • Decouple business logic • Test with sample events • Use lambci/lambda docker run -v $PWD:/var/task lambci/lambda:python3.6 my_module.my_handler
11. AWS Lambda: chalice • Microframework for API Gateway + Lambda functions • Flask-like • A CLI that creates and configures the AWS resources needed (Lambda, API Gateway, IAM…) and makes and uploads the deployment package • Officially supported by AWS
12. AWS Lambda: chalice $ pip install chalice $ chalice new-project helloworld && cd helloworld $ cat app.py from chalice import Chalice app = Chalice(app_name='helloworld') @app.route('/', methods=['POST']) def greeting(): body = app.current_request.json_body message = 'Hello {} {}!'.format(body['first_name'], body['last_name']) return { "message" : message } $ chalice deploy --dev
13. AWS Lambda: zappa • Zappa is a tool for deploying your Python WSGI apps to AWS • Manages the lifecycle of the AWS resources and your application versioning (via S3) • Advanced features as remote invocations or scheduling configuration and async task execution $ zappa init $ zappa deploy
14. Use case: CARTO + Matplotlib + Amazon AWS
15. Other hosted solutions • Azure Functions • IBM BlueMix Cloud Functions (based on OpenWhisk) • Iron.io (based on Docker containers) • Google cloud functions does NOT yet support Python
16. Do-It-Yourself
17. DIY: Iron functions • Based on Docker containers: stdin and stdout as communication mechanism • Pulls images from a Docker registry (Dockerhub) • Configuration based on a func.yml file • Creation of a function: $ fn build && fn push $ fn apps create helloapp && fn routes create helloapp /hello
18. DIY: Iron functions import os import sys import json if not os.isatty(sys.stdin.fileno()): obj = json.loads(sys.stdin.read()) first_name = obj.get("first_name") last_name = obj.get("last_name") message = 'Hello {} {}!'.format(first_name, last_name) print(json.dumps({"message": message}))
19. DIY: Apache OpenWhisk Complex Architecture: • Event-based programming with triggers, rules and actions $ wsk action create hello hello.py --kind python:3 $ wsk action invoke hello --param first_name Margaret --param last_name Hamilton --result
20. DIY: OpenFaas • Works on top of Docker Swarm or Kubernetes • Autoscales to meet the concurrency needed • Configuration via .yaml or UI provider: name: faas gateway: http://127.0.0.1:8080 functions: hello: lang: Dockerfile handler: ./handler image: alexellis2/faas-hello-python3
21. DIY: fission • Runs on Kubernetes (locally on Minikube) • 100 msec cold-start latencies $ fission function create --name hello --env python3 --code hello.py $ fission route add --function hello --url /hello import json from flask import request def main(): data = json.loads(request.get_data().decode()) message = 'Hello {} {}!'.format(data['first_name'], data['last_name']) return json.dumps({"message": message}
22. Happy hacking! Alejandro Guirao @lekum lekum.org lekum/serverless-with-python