Microservices and serverless in python projects

Monoliths, microservices and now Serverless. Function as a Service (FaaS) platforms give us new ways to attack old problems. The possibility of executing functions as a service allows designing scalable and highly parallel applications, but on the other hand, this kind of applications require a particular programming style. For example, bundling dependencies and managing state is not trivial.

However, there are plenty of tools and frameworks to help you code serverless applications with Python, and once you get started it is not complicated.

In this talk we will mention the advantages of Serverless and we will focus on the situations in which we can introduce it into our Python projects. We will use AWS Lambda for the examples.

Transcript

1. Microservices and Serverless in Python projects José Manuel Ortega Europython 2018 @jmortegac v 0.5
2. Agenda •Microservices in python •Introducing Serverless and Function as a Service •Python frameworks for AWS •AWS Lambda functions with zappa and chalice •Deploy AWS lambda functions from aws console 2 Microservices and Serverless in Python projects
3. Microservices vs Serverless 3 Microservices and Serverless in Python projects
4. Microservices 4 Microservices and Serverless in Python projects
5. Asynchronous calls with asyncio and aiohttp 5 Microservices and Serverless in Python projects
6. REST API Development 6 Microservices and Serverless in Python projects
7. Performance 7 Microservices and Serverless in Python projects 8% less memory 6% faster response times
8. Microservices with graphql 8 Microservices and Serverless in Python projects
9. 9 Microservices and Serverless in Python projects
10. 10 Microservices and Serverless in Python projects
11. 11 Microservices and Serverless in Python projects
12. 12 Microservices and Serverless in Python projects
13. 13 Microservices and Serverless in Python projects SERVER
14. 14 Microservices and Serverless in Python projects CLIENT
15. Microservices benefits •Separation of concerns •Services are decoupled from each other •Managing smaller projects •More scaling and deployment options 15 Microservices and Serverless in Python projects
16. 16 Microservices and Serverless in Python projects Serverless
17. Serverless architecture •FaaS - Function as a Service •Fully managed computing • Provisioning • Scalability • Monitoring • Logging •Deploy your code •Pay only for actual usage 17 Microservices and Serverless in Python projects
18. Serverless architecture 18 Microservices and Serverless in Python projects
19. Serverless uses cases ➢ REST API • Stateless services and microservices • Suitable for Chat bots ➢ Events • File processing (S3 event) & Data ingestion • Data/Stream processing • Incidents handling (CloudWatch event log) • IoT ➢ Scheduled tasks • Monitoring, load testing • Periodical jobs 19 Microservices and Serverless in Python projects
20. 20 Microservices and Serverless in Python projects
21. Serverless benefits •No server management •Automatic scaling and load balancing •Lower infrastructure costs •Flexibility and high availability •Infrastructure managed by service provider 21 Microservices and Serverless in Python projects
22. Serverless drawbacks • The tools around the deployment automation of serverless functions are still in development. •There is no control over containers when the execution environments are created or destroyed •Debugging, Deploying and monitoring 22 Microservices and Serverless in Python projects
23. Cloud providers •AWS •Microsoft Azure •Cloud platform •OpenWhisk(OS) •Kubeless(OS) 23 Microservices and Serverless in Python projects
24. Aws lambda 24 Microservices and Serverless in Python projects
25. 25 Microservices and Serverless in Python projects
26. Aws lambda functions 26 Microservices and Serverless in Python projects
27. Aws lambda functions 27 Microservices and Serverless in Python projects
28. Create lambda function with awscli 28 Microservices and Serverless in Python projects $ aws lambda create-function \ --region eu-west-1 \ --function-name MyHandler\ --zip-file fileb://handler.zip \ --role arn:aws:iam::XXX:role/MyLambdaRole \ --vpc-config SubnetIds=XXX,SecurityGroupIds=XXX \ --handler handler.handler \ --runtime python3.6 \ --profile personal \ --timeout 10 \ --memory-size 512
29. 29 Microservices and Serverless in Python projects
30. 30 Microservices and Serverless in Python projects
31. Frameworks 31 Microservices and Serverless in Python projects python-λ
32. 32 Microservices and Serverless in Python projects
33. 33 Microservices and Serverless in Python projects
34. Zappa architecture 34 Microservices and Serverless in Python projects
35. Zappa 35 Microservices and Serverless in Python projects
36. 36 Microservices and Serverless in Python projects
37. 37 Microservices and Serverless in Python projects
38. 38 Microservices and Serverless in Python projects
39. 39 Microservices and Serverless in Python projects # zappa_settings.json { "dev": { "aws_region": "us-east-1", "django_settings": "hello.settings", "profile_name": "default", "project_name": "hello", "runtime": "python3.6", "s3_bucket": "zappa-huyg6op0s" } }
40. Zappa deploy 40 Microservices and Serverless in Python projects $ zappa deploy <env> •Zips code and dependencies •Create AWS Lambda and deploys the zip •Creates endpoint on API Gateway and links to AWS Lambda
41. Zappa deploy 41 Microservices and Serverless in Python projects
42. Zappa 42 Microservices and Serverless in Python projects
43. Zappa Asynchronous Task 43 Microservices and Serverless in Python projects
44. Chalice 44 Microservices and Serverless in Python projects •Python Serverless Microframework for AWS •Each endpoint is a separate function
45. Chalice 45 Microservices and Serverless in Python projects
46. Chalice 46 Microservices and Serverless in Python projects
47. Chalice example 47 Microservices and Serverless in Python projects
48. Chalice methods 48 Microservices and Serverless in Python projects Resource HTTP Verb AWS Lambda /talks GET get_talks /talk POST add_new_talk /talks/{ID} PUT update_talk /talks/{ID} DELETE delete_talk
49. Chalice methods 49 Microservices and Serverless in Python projects
50. Chalice options 50 Microservices and Serverless in Python projects
51. Chalice deploy 51 Microservices and Serverless in Python projects Updating IAM policy. Updating lambda function... Regen deployment package... Sending changes to lambda. API Gateway rest API already found. Deploying to: dev
52. 52 Microservices and Serverless in Python projects
53. https://github.com/lambci/docker-lambda 53 Microservices and Serverless in Python projects
54. http://serverlesscalc.com/ 54 Microservices and Serverless in Python projects
55. 55 Microservices and Serverless in Python projects
56. https://github.com/serverless/examples 56 Microservices and Serverless in Python projects
57. References 57 Microservices and Serverless in Python projects •https://aws.amazon.com/blogs/compute/microservic es-without-the-servers •https://github.com/Miserlou/Zappa •https://github.com/pmuens/awesome-serverless •https://github.com/aws/chalice •https://chalice.readthedocs.io/en/latest
58. 58 Microservices and Serverless in Python projects Serverless architecture is the next generation of cloud evolution
59. Thank you! José Manuel Ortega jmortega.github.io