Serverless with Python

Talk at PyConES 2017

Transcript

1. Serverless with Python
   Look ma, no servers!
   Alejandro Guirao
   @lekum
   github.com/lekum
   lekum.org
   

   View Slide

2. What is Serverless?
   ● No servers?
   ● FaaS (functions as a service)
   ● Elasticity
   

   View Slide

3. Hosted solutions
   

   View Slide

4. The pioneer: AWS Lambda
   S3 bucket
   DynamoDB table
   Kinesis Streams
   SNS Topic
   SES e-mail
   Cognito
   CloudFormation
   stack
   CloudWatch
   logs & events
   CodeCommit
   Config
   Alexa skill
   Lex
   API Gateway
   AWS Lambda
   Event sources
   

   View Slide

5. AWS Lambda limits
   

   View Slide

6. AWS Lambda: Manual deploy
   def my_handler(event, context):
   message = 'Hello {} {}!'.format(event['first_name'],
   event['last_name'])
   return { 'message' : message }
   

   View Slide

7. Context object
   get_remaining_time_in_millis()
   function_name
   function_version
   invoked_function_arn
   memory_limit_in_mb
   aws_request_id
   log_group_name
   log_stream_name
   identity
   client_context
   

   View Slide

8. AWS Lambda: Manual deploy
   aws lambda create-function \
   --region eu-west-1 \
   --function-name HelloPython \
   --zip-file fileb://deployment-package.zip \
   --role role-arn \
   --handler hello_python.my_handler \
   --runtime python3.6 \
   --timeout 15 \
   --memory-size 512
   

   View Slide

9. AWS Lambda: Manual deploy
   $ aws lambda invoke \
   --function-name \
   --payload '{"first_name":"Margaret", "last_name":"Hamilton"}'
   out.json
   $ cat out.json
   {"message": "Hello Margaret Hamilton!"}
   

   View Slide

10. Testing AWS Lambda
    ● Decouple business logic
    ● Test with sample events
    ● Use lambci/lambda
    docker run -v $PWD:/var/task lambci/lambda:python3.6
    my_module.my_handler
    

    View Slide

11. AWS Lambda: chalice
    ● Microframework for API Gateway + Lambda functions
    ● Flask-like
    ● A CLI that creates and configures the AWS resources
    needed (Lambda, API Gateway, IAM…) and makes and
    uploads the deployment package
    ● Officially supported by AWS
    

    View Slide

12. AWS Lambda: chalice
    $ pip install chalice
    $ chalice new-project helloworld && cd helloworld
    $ cat app.py
    from chalice import Chalice
    app = Chalice(app_name='helloworld')
    @app.route('/', methods=['POST'])
    def greeting():
    body = app.current_request.json_body
    message = 'Hello {} {}!'.format(body['first_name'], body['last_name'])
    return { "message" : message }
    $ chalice deploy --dev
    

    View Slide

13. AWS Lambda: zappa
    ● Zappa is a tool for deploying your Python WSGI apps to
    AWS
    ● Manages the lifecycle of the AWS resources and your
    application versioning (via S3)
    ● Advanced features as remote invocations or scheduling
    configuration and async task execution
    $ zappa init
    $ zappa deploy
    

    View Slide

14. Use case: CARTO + Matplotlib + Amazon
    AWS
    

    View Slide

15. Other hosted solutions
    ● Azure Functions
    ● IBM BlueMix Cloud Functions (based on OpenWhisk)
    ● Iron.io (based on Docker containers)
    ● Google cloud functions does NOT yet support Python
    

    View Slide

16. Do-It-Yourself
    

    View Slide

17. DIY: Iron functions
    ● Based on Docker containers: stdin and stdout as communication
    mechanism
    ● Pulls images from a Docker registry (Dockerhub)
    ● Configuration based on a func.yml file
    ● Creation of a function:
    $ fn build && fn push
    $ fn apps create helloapp && fn routes create helloapp /hello
    

    View Slide

18. DIY: Iron functions
    import os
    import sys
    import json
    if not os.isatty(sys.stdin.fileno()):
    obj = json.loads(sys.stdin.read())
    first_name = obj.get("first_name")
    last_name = obj.get("last_name")
    message = 'Hello {} {}!'.format(first_name, last_name)
    print(json.dumps({"message": message}))
    

    View Slide

19. DIY: Apache OpenWhisk
    Complex Architecture: ● Event-based programming with
    triggers, rules and actions
    $ wsk action create hello hello.py
    --kind python:3
    $ wsk action invoke hello --param
    first_name Margaret --param last_name
    Hamilton --result
    

    View Slide

20. DIY: OpenFaas
    ● Works on top of Docker Swarm or Kubernetes
    ● Autoscales to meet the concurrency needed
    ● Configuration via .yaml or UI
    provider:
    name: faas
    gateway: http://127.0.0.1:8080
    functions:
    hello:
    lang: Dockerfile
    handler: ./handler
    image: alexellis2/faas-hello-python3
    

    View Slide

21. DIY: fission
    ● Runs on Kubernetes (locally on Minikube)
    ● 100 msec cold-start latencies
    $ fission function create --name hello --env python3 --code hello.py
    $ fission route add --function hello --url /hello
    import json
    from flask import request
    def main():
    data = json.loads(request.get_data().decode())
    message = 'Hello {} {}!'.format(data['first_name'], data['last_name'])
    return json.dumps({"message": message}
    

    View Slide

22. Happy hacking!
    Alejandro Guirao
    @lekum
    lekum.org
    lekum/serverless-with-python
    

    View Slide