Going Serverless Effectively Using Cloud Run

Slide 1

Slide 1 text

Going Serverless Eﬀectively (Using Cloud Run) Aditya Satrya Head of Engineering at Jabar Digital Service

Slide 2

Slide 2 text

What is Serverless 2 No infra management Fully managed security Pay-per-usage

Slide 3

Slide 3 text

3 Focus on your code, not infrastructure

Slide 4

Slide 4 text

Full Stack Serverless in GCP 4

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Serverless Compute Options 8 Serverless Functions Serverless Applications Serverless Containers Cloud Functions App Engine Cloud Run Deploy snippet of code Deploy app code Deploy Containers

Slide 9

Slide 9 text

How to choose? 9

Slide 10

Slide 10 text

10 Cloud Functions Good ﬁt for: ● Event-driven applications ● “Glueing” systems together ● Simple API Bad ﬁt for: ● Stateful systems ● Large, complex APIs ● Require a high level of control ● Need custom runtimes or binaries

Slide 11

Slide 11 text

Example: Data e-KTP Data Extractor Source: https:/ /medium.com/@imrenagi/ 11

Slide 12

Slide 12 text

12 App Engine Good ﬁt for: ● Stateless applications ● Rapidly developing CRUD-heavy applications ● App composed of a few services ● Deploying complex APIs Bad ﬁt for: ● Stateful systems ● Slow startup time ● Require protocol other than HTTP

Slide 13

Slide 13 text

13 Cloud Run Good ﬁt for: ● Stateless services that are easily containerized ● Event-driven applications ● Require custom dependencies Bad ﬁt for: ● Highly stateful systems ● Require protocols other than HTTP

Slide 14

Slide 14 text

14 Cloud Function App Engine Cloud Run Deployment unit Code snippet App code Container Custom URL No Yes Yes Run any language No No Yes Background process No Yes No

Slide 15

Slide 15 text

15 Serverless is not for every case

Slide 16

Slide 16 text

Disadvantages of Serverless 16 ▷ Testing and debugging become harder ▷ may cost more to run long-running processes ▷ Performance may be affected ▷ Vendor lock-in is a risk

Slide 17

Slide 17 text

How to choose serverless or not? 17

Slide 18

Slide 18 text

How to choose serverless option? 18

Slide 19

Slide 19 text

Cloud Run 19

Slide 20

Slide 20 text

What is Cloud Run? 20 Custom/given domain TLS certiﬁcate Autoscale Pay while request handled

Slide 21

Slide 21 text

Cloud Run: Concepts 21

Slide 22

Slide 22 text

Cloud Run: Concepts 22 ▷ Containers ▷ Resource model ▷ Concurrency ▷ Autoscaling

Slide 23

Slide 23 text

Containers 23

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Resource Model 25 Service Revision Instance

Slide 26

Slide 26 text

Concurrency 26

Slide 27

Slide 27 text

27 400 clients are making 3 requests per second to a Cloud Run Concurrency = 1 Concurrency = 80

Slide 28

Slide 28 text

Limitation of concurrency settings considerations 28 ▷ CPU and memory usage per request ▷ Application design: ○ can handle multiple requests at the same time? ▷ Lower concurrency ~ lower scaling performance

Slide 29

Slide 29 text

Autoscaling 29 Max simultaneous requests = concurrency per instance * max instances Limitation considerations: ● cost control ● compatibility with backing resources

Slide 30

Slide 30 text

Container Runtime Contract 30 ▷ Stateless ▷ Listen for HTTP request on $PORT ▷ Startup times < 4 mins ▷ No CPU outside request scope

Slide 31

Slide 31 text

Cloud Run: Use Cases 31 Public ● Website ● API endpoint ● Mobile/IoT backend ● Webhook Private ● Microservices ● Async tasks

Slide 32

Slide 32 text

Cloud Run: Development Tips 32

Slide 33

Slide 33 text

On Migrating Existing Service 33 Modify your code to: ▷ Listen on the port defined by PORT environment variable ▷ Remove reliance on local filesystem for persistent storage ▷ Add a Dockerfile ▷ Send logs to stdout, stderr, or /var/log

Slide 34

Slide 34 text

Writing Eﬀective Service 34 ▷ Avoiding background activities ▷ Deleting temporary ﬁles ▷ Do not let service crash on errors

Slide 35

Slide 35 text

Optimizing Performance 35 ▷ Minimize image size ○ Use lean base image (exp: alpine, scratch) ○ Multi stage build image ▷ Minimize entrypoint execution time ▷ Use dependency wisely ▷ Use lazy initialization for global variables

Slide 36

Slide 36 text

Fine-Tuning Concurrency 36 ▷ Find the maximum stable concurrency level: ○ Set expected_concurrency value ○ Load test your system ○ If poorly, reduce expected_concurrency ○ If well, increase expected_concurrency ○ Continue iterating until you ﬁnd the maximum stable concurrency ▷ Also increase memory_limit when increasing expected_concurrency

Slide 37

Slide 37 text

Demo 37

Slide 38

Slide 38 text

Demo 1: Code, Build, Deploy easily using Cloud Build and Cloud Run 38 ▷ Create Python Hello World application ▷ Create Dockerﬁle ▷ Build and push image using Cloud Build ▷ Deploy to Cloud Run ▷ Access your production-ready live app on provided domain

Slide 39

Slide 39 text

Demo 2: Find stable concurrency level using Load Testing 39 ▷ Set max instance = 1 ▷ Load test with 2500 concurrent users ▷ See number of users vs response time, it’s not stable ▷ Set max instance = 1000 ▷ Load test with 2500 concurrent users ▷ See number of users vs response time, it’s stable

Slide 40

Slide 40 text

References 40 ▷ https://cloud.google.com/serverless-options/ ▷ https://cloud.google.com/hosting-options/ ▷ https://bravenewgeek.com/serverless-on-gcp/ ▷ https://cloud.google.com/appengine/docs ▷ https://cloud.google.com/functions/docs/ ▷ https://cloud.google.com/run/docs/ ▷ https://github.com/ahmetb/cloud-run-faq ▷ https://cloud.google.com/blog/products/gcp/time-to-hello-world-vms-vs-containers -vs-paas-vs-faas ▷ https://cloud.google.com/blog/products/gcp/choosing-the-right-compute-option-in- gcp-a-decision-tree ▷ https://docs.microsoft.com/en-us/azure/architecture/guide/technology-choices/com pute-decision-tree ▷ https://www.cloudflare.com/learning/serverless/why-use-serverless/ ▷ https://medium.com/@PaulDJohnston/when-not-to-use-serverless-jeff-6d054d0e7098 ▷ https://www.youtube.com/watch?v=xVuuvZkYiNM&t=2538s ▷ https://medium.com/@imrenagi/ekstraksi-informasi-e-ktp-dengan-google-cloud-funct ion-dan-cloud-vision-api-4655db21d084