Microservices and serverless in python projects

Monoliths, microservices and now Serverless. Function as a Service (FaaS) platforms give us new ways to attack old problems. The possibility of executing functions as a service allows designing scalable and highly parallel applications, but on the other hand, this kind of applications require a particular programming style. For example, bundling dependencies and managing state is not trivial.

However, there are plenty of tools and frameworks to help you code serverless applications with Python, and once you get started it is not complicated.

In this talk we will mention the advantages of Serverless and we will focus on the situations in which we can introduce it into our Python projects. We will use AWS Lambda for the examples.

Transcript

1. Microservices and Serverless in
   Python projects
   José Manuel Ortega
   Europython 2018
   @jmortegac
   v 0.5
   

   View Slide

2. Agenda
   •Microservices in python
   •Introducing Serverless and Function as a Service
   •Python frameworks for AWS
   •AWS Lambda functions with zappa and chalice
   •Deploy AWS lambda functions from aws
   console
   2
   Microservices and Serverless in Python projects
   

   View Slide

3. Microservices vs Serverless
   3
   Microservices and Serverless in Python projects
   

   View Slide

4. Microservices
   4
   Microservices and Serverless in Python projects
   

   View Slide

5. Asynchronous calls with asyncio and aiohttp
   5
   Microservices and Serverless in Python projects
   

   View Slide

6. REST API Development
   6
   Microservices and Serverless in Python projects
   

   View Slide

7. Performance
   7
   Microservices and Serverless in Python projects
   8% less memory
   6% faster response times
   

   View Slide

8. Microservices with graphql
   8
   Microservices and Serverless in Python projects
   

   View Slide

9. 9
   Microservices and Serverless in Python projects
   

   View Slide

10. 10
    Microservices and Serverless in Python projects
    

    View Slide

11. 11
    Microservices and Serverless in Python projects
    

    View Slide

12. 12
    Microservices and Serverless in Python projects
    

    View Slide

13. 13
    Microservices and Serverless in Python projects
    SERVER
    

    View Slide

14. 14
    Microservices and Serverless in Python projects
    CLIENT
    

    View Slide

15. Microservices benefits
    •Separation of concerns
    •Services are decoupled from each other
    •Managing smaller projects
    •More scaling and deployment options
    15
    Microservices and Serverless in Python projects
    

    View Slide

16. 16
    Microservices and Serverless in Python projects
    Serverless
    

    View Slide

17. Serverless architecture
    •FaaS - Function as a Service
    •Fully managed computing
    • Provisioning
    • Scalability
    • Monitoring
    • Logging
    •Deploy your code
    •Pay only for actual usage
    17
    Microservices and Serverless in Python projects
    

    View Slide

18. Serverless architecture
    18
    Microservices and Serverless in Python projects
    

    View Slide

19. Serverless uses cases
    ➢ REST API
    • Stateless services and microservices
    • Suitable for Chat bots
    ➢ Events
    • File processing (S3 event) & Data ingestion
    • Data/Stream processing
    • Incidents handling (CloudWatch event log)
    • IoT
    ➢ Scheduled tasks
    • Monitoring, load testing
    • Periodical jobs
    19
    Microservices and Serverless in Python projects
    

    View Slide

20. 20
    Microservices and Serverless in Python projects
    

    View Slide

21. Serverless benefits
    •No server management
    •Automatic scaling and load balancing
    •Lower infrastructure costs
    •Flexibility and high availability
    •Infrastructure managed by service provider
    21
    Microservices and Serverless in Python projects
    

    View Slide

22. Serverless drawbacks
    • The tools around the deployment automation of
    serverless functions are still in development.
    •There is no control over containers when the
    execution environments are created or
    destroyed
    •Debugging, Deploying and monitoring
    22
    Microservices and Serverless in Python projects
    

    View Slide

23. Cloud providers
    •AWS
    •Microsoft Azure
    •Cloud platform
    •OpenWhisk(OS)
    •Kubeless(OS)
    23
    Microservices and Serverless in Python projects
    

    View Slide

24. Aws lambda
    24
    Microservices and Serverless in Python projects
    

    View Slide

25. 25
    Microservices and Serverless in Python projects
    

    View Slide

26. Aws lambda functions
    26
    Microservices and Serverless in Python projects
    

    View Slide

27. Aws lambda functions
    27
    Microservices and Serverless in Python projects
    

    View Slide

28. Create lambda function with awscli
    28
    Microservices and Serverless in Python projects
    $ aws lambda create-function \
    --region eu-west-1 \
    --function-name MyHandler\
    --zip-file fileb://handler.zip \
    --role arn:aws:iam::XXX:role/MyLambdaRole \
    --vpc-config SubnetIds=XXX,SecurityGroupIds=XXX \
    --handler handler.handler \
    --runtime python3.6 \
    --profile personal \
    --timeout 10 \
    --memory-size 512
    

    View Slide

29. 29
    Microservices and Serverless in Python projects
    

    View Slide

30. 30
    Microservices and Serverless in Python projects
    

    View Slide

31. Frameworks
    31
    Microservices and Serverless in Python projects
    python-λ
    

    View Slide

32. 32
    Microservices and Serverless in Python projects
    

    View Slide

33. 33
    Microservices and Serverless in Python projects
    

    View Slide

34. Zappa architecture
    34
    Microservices and Serverless in Python projects
    

    View Slide

35. Zappa
    35
    Microservices and Serverless in Python projects
    

    View Slide

36. 36
    Microservices and Serverless in Python projects
    

    View Slide

37. 37
    Microservices and Serverless in Python projects
    

    View Slide

38. 38
    Microservices and Serverless in Python projects
    

    View Slide

39. 39
    Microservices and Serverless in Python projects
    # zappa_settings.json
    {
    "dev": {
    "aws_region": "us-east-1",
    "django_settings": "hello.settings",
    "profile_name": "default",
    "project_name": "hello",
    "runtime": "python3.6",
    "s3_bucket": "zappa-huyg6op0s"
    }
    }
    

    View Slide

40. Zappa deploy
    40
    Microservices and Serverless in Python projects
    $ zappa deploy
    •Zips code and dependencies
    •Create AWS Lambda and deploys the zip
    •Creates endpoint on API Gateway and
    links to AWS Lambda
    

    View Slide

41. Zappa deploy
    41
    Microservices and Serverless in Python projects
    

    View Slide

42. Zappa
    42
    Microservices and Serverless in Python projects
    

    View Slide

43. Zappa Asynchronous Task
    43
    Microservices and Serverless in Python projects
    

    View Slide

44. Chalice
    44
    Microservices and Serverless in Python projects
    •Python Serverless Microframework for AWS
    •Each endpoint is a separate function
    

    View Slide

45. Chalice
    45
    Microservices and Serverless in Python projects
    

    View Slide

46. Chalice
    46
    Microservices and Serverless in Python projects
    

    View Slide

47. Chalice example
    47
    Microservices and Serverless in Python projects
    

    View Slide

48. Chalice methods
    48
    Microservices and Serverless in Python projects
    Resource HTTP Verb AWS Lambda
    /talks GET get_talks
    /talk POST add_new_talk
    /talks/{ID} PUT update_talk
    /talks/{ID} DELETE delete_talk
    

    View Slide

49. Chalice methods
    49
    Microservices and Serverless in Python projects
    

    View Slide

50. Chalice options
    50
    Microservices and Serverless in Python projects
    

    View Slide

51. Chalice deploy
    51
    Microservices and Serverless in Python projects
    Updating IAM policy.
    Updating lambda function...
    Regen deployment package...
    Sending changes to lambda.
    API Gateway rest API already found.
    Deploying to: dev
    

    View Slide

52. 52
    Microservices and Serverless in Python projects
    

    View Slide

53. https://github.com/lambci/docker-lambda
    53
    Microservices and Serverless in Python projects
    

    View Slide

54. http://serverlesscalc.com/
    54
    Microservices and Serverless in Python projects
    

    View Slide

55. 55
    Microservices and Serverless in Python projects
    

    View Slide

56. https://github.com/serverless/examples
    56
    Microservices and Serverless in Python projects
    

    View Slide

57. References
    57
    Microservices and Serverless in Python projects
    •https://aws.amazon.com/blogs/compute/microservic
    es-without-the-servers
    •https://github.com/Miserlou/Zappa
    •https://github.com/pmuens/awesome-serverless
    •https://github.com/aws/chalice
    •https://chalice.readthedocs.io/en/latest
    

    View Slide

58. 58
    Microservices and Serverless in Python projects
    Serverless architecture is the
    next generation of cloud
    evolution
    

    View Slide

59. Thank you!
    José Manuel Ortega
    jmortega.github.io
    

    View Slide