Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless with Python

Serverless with Python

Talk at PyConES 2017


Alejandro Guirao Rodríguez

September 24, 2017


  1. Serverless with Python Look ma, no servers! Alejandro Guirao @lekum

    github.com/lekum lekum.org
  2. What is Serverless? • No servers? • FaaS (functions as

    a service) • Elasticity
  3. Hosted solutions

  4. The pioneer: AWS Lambda S3 bucket DynamoDB table Kinesis Streams

    SNS Topic SES e-mail Cognito CloudFormation stack CloudWatch logs & events CodeCommit Config Alexa skill Lex API Gateway AWS Lambda Event sources
  5. AWS Lambda limits

  6. AWS Lambda: Manual deploy def my_handler(event, context): message = 'Hello

    {} {}!'.format(event['first_name'], event['last_name']) return { 'message' : message }
  7. Context object get_remaining_time_in_millis() function_name function_version invoked_function_arn memory_limit_in_mb aws_request_id log_group_name log_stream_name

    identity client_context
  8. AWS Lambda: Manual deploy aws lambda create-function \ --region eu-west-1

    \ --function-name HelloPython \ --zip-file fileb://deployment-package.zip \ --role role-arn \ --handler hello_python.my_handler \ --runtime python3.6 \ --timeout 15 \ --memory-size 512
  9. AWS Lambda: Manual deploy $ aws lambda invoke \ --function-name

    <function-arn> \ --payload '{"first_name":"Margaret", "last_name":"Hamilton"}' out.json $ cat out.json {"message": "Hello Margaret Hamilton!"}
  10. Testing AWS Lambda • Decouple business logic • Test with

    sample events • Use lambci/lambda docker run -v $PWD:/var/task lambci/lambda:python3.6 my_module.my_handler
  11. AWS Lambda: chalice • Microframework for API Gateway + Lambda

    functions • Flask-like • A CLI that creates and configures the AWS resources needed (Lambda, API Gateway, IAM…) and makes and uploads the deployment package • Officially supported by AWS
  12. AWS Lambda: chalice $ pip install chalice $ chalice new-project

    helloworld && cd helloworld $ cat app.py from chalice import Chalice app = Chalice(app_name='helloworld') @app.route('/', methods=['POST']) def greeting(): body = app.current_request.json_body message = 'Hello {} {}!'.format(body['first_name'], body['last_name']) return { "message" : message } $ chalice deploy --dev
  13. AWS Lambda: zappa • Zappa is a tool for deploying

    your Python WSGI apps to AWS • Manages the lifecycle of the AWS resources and your application versioning (via S3) • Advanced features as remote invocations or scheduling configuration and async task execution $ zappa init $ zappa deploy
  14. Use case: CARTO + Matplotlib + Amazon AWS

  15. Other hosted solutions • Azure Functions • IBM BlueMix Cloud

    Functions (based on OpenWhisk) • Iron.io (based on Docker containers) • Google cloud functions does NOT yet support Python
  16. Do-It-Yourself

  17. DIY: Iron functions • Based on Docker containers: stdin and

    stdout as communication mechanism • Pulls images from a Docker registry (Dockerhub) • Configuration based on a func.yml file • Creation of a function: $ fn build && fn push $ fn apps create helloapp && fn routes create helloapp /hello
  18. DIY: Iron functions import os import sys import json if

    not os.isatty(sys.stdin.fileno()): obj = json.loads(sys.stdin.read()) first_name = obj.get("first_name") last_name = obj.get("last_name") message = 'Hello {} {}!'.format(first_name, last_name) print(json.dumps({"message": message}))
  19. DIY: Apache OpenWhisk Complex Architecture: • Event-based programming with triggers,

    rules and actions $ wsk action create hello hello.py --kind python:3 $ wsk action invoke hello --param first_name Margaret --param last_name Hamilton --result
  20. DIY: OpenFaas • Works on top of Docker Swarm or

    Kubernetes • Autoscales to meet the concurrency needed • Configuration via .yaml or UI provider: name: faas gateway: functions: hello: lang: Dockerfile handler: ./handler image: alexellis2/faas-hello-python3
  21. DIY: fission • Runs on Kubernetes (locally on Minikube) •

    100 msec cold-start latencies $ fission function create --name hello --env python3 --code hello.py $ fission route add --function hello --url /hello import json from flask import request def main(): data = json.loads(request.get_data().decode()) message = 'Hello {} {}!'.format(data['first_name'], data['last_name']) return json.dumps({"message": message}
  22. Happy hacking! Alejandro Guirao @lekum lekum.org lekum/serverless-with-python