Python Serverless Microservices - Workshop PyCon SE 2020

Transcript

1. Python Serverless Microservices Workshop Nilo Ney Coutinho Menezes (nilo@logikraft.be)

2. Plan • Short introduction to serverless • Examples: 1. Simple

   Example 2. Show IP and country 3. URL Shortener

3. Serverless Application Framework • Written in JavaScript, but works with

   multiple languages • Uses Node, Python and Chocolatey (alternative Windows Package Manager) • Deploys the serverless infrastructure in the cloud. Compatible with Amazon Web Services, Google Cloud, Microsoft Azure and others • Simplifies configuration and application updates • Configuration files written in YAML • https://www.serverless.com/

4. Server less? • We know the server still there •

   But we don’t have to keep it running • We don’t have to update it frequently • We can have more as we need them (autoscaling) • We only pay if we use them (per call, memory and run time) • So, it is an industry term to say somebody else is taking care of the server for you.

5. Microservices • Microservices are an architecture for development and deployment

   of new systems • It antagonizes the traditional monolithic architecture that deploys the full system in a single application • Microservices means we should split our systems into smaller ones • Smaller parts are easy to understand, can be deployed independently • Share no database with other parts • Share data and services via a well-defined API • Can be rewritten one by one (less coupling)

6. Lambdas • Lambdas are Functions as a Service (FaaS) provided

   by Amazon Web Services (AWS) • We will be using AWS as our example provider • You need to have an account to test the code shown in this presentation • You can create a free account on their web site: https://aws.amazon.com/ • If it is your first time on AWS, I strongly suggest you check their web site and documentation, as you will be paying for the resources used beyond the free tier. • To start using AWS with Python, install their command-line client: pip install awscli

7. Python Serverless Microservices • Because we use Python • Because

   we use AWS Lambdas, FaaS, serverless • And we would divide each API call in a different lambda, grouped in a domain, following the directives of Microservices and Domain-Driven Design (DDD). Check these books for more info:

8. 1 – Simple example

9. Installing • Download and install node.js from https://nodejs.org/en/ • Install

   the Serverless Application Framework: npm install –g serverless • Let’s created the application stub type: serverless • Try to answer the questions as in the image below:

10. The stub application: handler.py • Please rename the file to

    hello.py

11. Serverless.yml • This is all the configuration we need for

    our simple function. • It says we are going to deploy a Python 3.8 Lambda Function • It defines the handler (the module and function name) • It also specifies the URL path to use and the http method

12. Deploying • Serverless will deploy all we need with a

    single command sls deploy • Your urls will be different.

13. Visiting our new page Go to: https://dg68p0g319.execute-api.eu-west-1.amazonaws.com/dev/pyconse2020/hello Or https://bit.ly/PyConSE-Hello

14. The super user-friendly page :-D

15. What is in this json? • In the input field,

    all parameters passed to your function • Copy and paste it to your preferred text editor that supports JSON. Reformat the code to make it easier to read. • You get user location with IP and country information • The URL and path used on this call • Any parameter used (query string and path) if any • You can access all these values as in simple Python dictionary

16. What do we have now

17. 2 – Showing IP and Country

18. Showing our IP and country • Using just the headers

    available to our lambda, let’s display a simple page with the callers IP and country. • We need to return a html page now. • Better use templates to offload the work to Jinja2 pip install jinja2 • We also must tell serverless to deploy or modules with the lambda function sls plugin install -n serverless-python- requirements

19. Show me the code

20. Let’s test it

21. 3 – URL Shortener

22. URL Shortener • I suppose you will get a short

    domain name :-D • Simple service  Get URL  Return link with short code  When the short code is accessed: - Count access (increment database field) - Redirect user to the URL - Codes will be deleted after 2 days

23. What we will do • We will reuse the same

    function to process GET and POST requests • GET will show a simple form with a field to enter the URL • POST will process the user input, create a record in the database and return a page with the links created. • Another function will get the code and redirect to the user url, incrementing the counter.

24. We need a table • Let’s ask serverless to create

    a table for us.

25. The Database • We are using DynamoDB • It is

    a key value database in AWS • It is also serverless and we pay for the storage and requests made • Super fast • Scalable • Does not support SQL • Really odd at the beginning

26. Our table • We designed a very simple table with

    a hash key (primary key) • All other fields are free, no schema • Using PynamoDB to interact with DynamoDB API

27. PynamoDB Operations Checking if a key already exists Creating a

    new record Querying data Updating the counter

28. What did we do?

29. In serverless • We create two functions • One for

    create that handles GETs and POST • Another one that handles GETs • Look how their urls are different. The function is chosen by pattern matching • In the useURL, the {short_key} is a parameter passed to the lambda in the event • We are using layers for dependencies • And a role to have access to the database

30. How does it look?

31. Wait, no huge framework? • Nope • You have a

    very simple function deployed to the cloud • You choose the batteries • You can combine multiple Python libraries depending on your needs • Leverage the cloud provider services like databases, api gateways, queues, etc

32. Advantages • Simple, you can read the function from top

    to bottom, no hidden magic code • Cheap, millions of invocations for less than 1 euro • Pays as you go (no upfront costs, if you don’t use, you won’t pay either) • Functions can be deployed independently • No downtime, old version keeps running until the new one is deployed

33. Disadvantages • As any new thing can be difficult at

    the beginning • You get dependent to a cloud provider • More tools to use • Lack of framework to start with

34. What’s next • Add more functions • Play with AWS

    Cognito to support authenticated users • Read about lambda layers • Other frameworks to help with Lambdas and Python:  AWS Chalice https://github.com/aws/chalice  AWS SAM https://aws.amazon.com/serverless/sam/

35. Thank you • All the code is hosted on GitHub

    https://github.com/lskbr/pyconse2020 • You may find this presentation on SpeakerDeck https://speakerdeck.com/lskbr • If you have any questions, do not hesitate to contact me: nilo@logikraft.be @lskbr on Twitter and Telegram