$30 off During Our Annual Pro Sale. View Details »

Serverless with Python

Serverless with Python

Talk at PyConES 2017

Alejandro Guirao Rodríguez

September 24, 2017

More Decks by Alejandro Guirao Rodríguez

Other Decks in Programming


  1. Serverless with Python Look ma, no servers! Alejandro Guirao @lekum

    github.com/lekum lekum.org
  2. What is Serverless? • No servers? • FaaS (functions as

    a service) • Elasticity
  3. Hosted solutions

  4. The pioneer: AWS Lambda S3 bucket DynamoDB table Kinesis Streams

    SNS Topic SES e-mail Cognito CloudFormation stack CloudWatch logs & events CodeCommit Config Alexa skill Lex API Gateway AWS Lambda Event sources
  5. AWS Lambda limits

  6. AWS Lambda: Manual deploy def my_handler(event, context): message = 'Hello

    {} {}!'.format(event['first_name'], event['last_name']) return { 'message' : message }
  7. Context object get_remaining_time_in_millis() function_name function_version invoked_function_arn memory_limit_in_mb aws_request_id log_group_name log_stream_name

    identity client_context
  8. AWS Lambda: Manual deploy aws lambda create-function \ --region eu-west-1

    \ --function-name HelloPython \ --zip-file fileb://deployment-package.zip \ --role role-arn \ --handler hello_python.my_handler \ --runtime python3.6 \ --timeout 15 \ --memory-size 512
  9. AWS Lambda: Manual deploy $ aws lambda invoke \ --function-name

    <function-arn> \ --payload '{"first_name":"Margaret", "last_name":"Hamilton"}' out.json $ cat out.json {"message": "Hello Margaret Hamilton!"}
  10. Testing AWS Lambda • Decouple business logic • Test with

    sample events • Use lambci/lambda docker run -v $PWD:/var/task lambci/lambda:python3.6 my_module.my_handler
  11. AWS Lambda: chalice • Microframework for API Gateway + Lambda

    functions • Flask-like • A CLI that creates and configures the AWS resources needed (Lambda, API Gateway, IAM…) and makes and uploads the deployment package • Officially supported by AWS
  12. AWS Lambda: chalice $ pip install chalice $ chalice new-project

    helloworld && cd helloworld $ cat app.py from chalice import Chalice app = Chalice(app_name='helloworld') @app.route('/', methods=['POST']) def greeting(): body = app.current_request.json_body message = 'Hello {} {}!'.format(body['first_name'], body['last_name']) return { "message" : message } $ chalice deploy --dev
  13. AWS Lambda: zappa • Zappa is a tool for deploying

    your Python WSGI apps to AWS • Manages the lifecycle of the AWS resources and your application versioning (via S3) • Advanced features as remote invocations or scheduling configuration and async task execution $ zappa init $ zappa deploy
  14. Use case: CARTO + Matplotlib + Amazon AWS

  15. Other hosted solutions • Azure Functions • IBM BlueMix Cloud

    Functions (based on OpenWhisk) • Iron.io (based on Docker containers) • Google cloud functions does NOT yet support Python
  16. Do-It-Yourself

  17. DIY: Iron functions • Based on Docker containers: stdin and

    stdout as communication mechanism • Pulls images from a Docker registry (Dockerhub) • Configuration based on a func.yml file • Creation of a function: $ fn build && fn push $ fn apps create helloapp && fn routes create helloapp /hello
  18. DIY: Iron functions import os import sys import json if

    not os.isatty(sys.stdin.fileno()): obj = json.loads(sys.stdin.read()) first_name = obj.get("first_name") last_name = obj.get("last_name") message = 'Hello {} {}!'.format(first_name, last_name) print(json.dumps({"message": message}))
  19. DIY: Apache OpenWhisk Complex Architecture: • Event-based programming with triggers,

    rules and actions $ wsk action create hello hello.py --kind python:3 $ wsk action invoke hello --param first_name Margaret --param last_name Hamilton --result
  20. DIY: OpenFaas • Works on top of Docker Swarm or

    Kubernetes • Autoscales to meet the concurrency needed • Configuration via .yaml or UI provider: name: faas gateway: functions: hello: lang: Dockerfile handler: ./handler image: alexellis2/faas-hello-python3
  21. DIY: fission • Runs on Kubernetes (locally on Minikube) •

    100 msec cold-start latencies $ fission function create --name hello --env python3 --code hello.py $ fission route add --function hello --url /hello import json from flask import request def main(): data = json.loads(request.get_data().decode()) message = 'Hello {} {}!'.format(data['first_name'], data['last_name']) return json.dumps({"message": message}
  22. Happy hacking! Alejandro Guirao @lekum lekum.org lekum/serverless-with-python