Pic-a-Daily Serverless Workshop

Pic-a-Daily Serverless Workshop

A presentation showing different serverless options on Google Cloud along with a workshop

Ce3e7802c882fca3f9160768f57ae116?s=128

Mete Atamel

March 26, 2020
Tweet

Transcript

  1. Proprietary + Confidential Guillaume Laforge — @glaforge Mete Atamel —

    @meteatamel
  2. Proprietary + Confidential GDG Cloud Nantes GDG Cloud Paris GDG

    Cloud Lyon GDG Lille Serverless Days Paris GDG Istanbul GDG Cloud Istanbul
  3. Proprietary + Confidential Meeting will be recorded and your name

    is in the chat Please mute yourself Ask questions in the chat, but feel free to unmute if stuck
  4. 01 The Pic-a-Daily Serverless Workshop: What you’re going to build!

  5. A photo-sharing application

  6. A photo-sharing application

  7. Final architecture diagram

  8. Four code labs g.co/codelabs/serverless-workshop

  9. Four code labs Lab 1 — Store and analyse pictures

    Storing pictures in Cloud Storage, trigger a Cloud Functions instance Calling the Cloud Vision API for analysis Storing metadata in Cloud Firestore Lab 3 — Run containers on schedule At a regular interval, with Cloud Scheduler Invoking a Cloud Run instance, to create a collage of recent pictures found in Cloud Firestore Storing collage in Cloud Storage Lab 2 — Create thumbnails Cloud Storage notifications sent to Pub/Sub Triggering a Cloud Run instance via Pub/Sub Creating a thumbnail with ImageMagick, and storing back into Cloud Storage Lab 4 — Create a web frontend Creating a web application with App Engine Listing pictures from Cloud Firestore Displaying pictures & collage from Cloud Storage g.co/codelabs/serverless-workshop
  10. 1. Gets credits for Google Cloud Platform Trial Billing Account

    2. Create or pick an existing Google Cloud project for the workshop 3. Make sure the project is using the Google Cloud Platform Trial Billing Account
  11. Go to link: https://gcpcredits.com/hannah Login with Google account In the

    end, you should see the billing account with credits
  12. In Google Cloud Console, go to Billing → Account Management

    Make sure your project is using the Google Cloud Platform Trial Billing Account
  13. 02 The Serverless spectrum on Google Cloud Platform

  14. Proprietary + Confidential Operational Model Programming Model No Infra Management

    Managed Security Pay only for usage Service-based Event-driven Stateless Serverless
  15. Proprietary + Confidential Virtual Machine OS Application Runtime Application Virtual

    Hardware Dedicated Server OS Application Runtime Application Virtual Hardware Containers OS Application Runtime Application Virtual Hardware Functions Functions Functions Application Platform OS Application Runtime Application Virtual Hardware Functions Serverless OS Application Runtime Application Virtual Hardware Functions Infrastructure Container Platform Function Compute Engine Kubernetes Engine App Engine Cloud Functions On-Premises Serverless Zone Physical Hardware Cloud Run
  16. Proprietary + Confidential Serverless functions Serverless web applications Serverless HTTP

    workloads Source code-based event driven functions Source code-based web apps and API backends HTTP containers fully managed HTTP containers on GKE cluster Cloud Functions App Engine Cloud Run Serverless compute options
  17. 03 Cloud Functions

  18. Dedicated Server Dedicated Server OS Application Runtime Application Virtual Hardware

    Functions Physical Hardware OS Application Runtime Application Hardware Functions Dedicated Server OS Application Runtime Application Hardware Functions On-Premises Virtual Machine OS Application Runtime Application Hardware Functions Infrastructure Compute Engine Containers OS Application Runtime Application Hardware Functions Container Kubernetes Engine Application Platform OS Application Runtime Application Hardware Functions Platform App Engine Serverless OS Application Runtime Application Hardware Functions Function Cloud Functions Highly customizable Highly managed
  19. Back end code functions that automatically run in response to

    a trigger event Microservices Cloud "glue" Node.js, Python, Go, Java Cloud Functions
  20. Cloud Functions as cloud glue Access 20+ Google services from

    GCF Cloud Storage Cloud Pub/Sub HTTPS Firebase Cloud Scheduler Cloud Tasks
  21. Good fit “Serverless” Using Pub/Sub and/or Cloud Storage Don't want

    to think about runtime env Data transformations (ETL) Cloud-based HTTP glue and webhooks Constraints Runtimes: Node.js, Python, Go, Java Function level granularity Must interact via events No custom domain name Good fit? Constraints?
  22. Why choose... Cloud Functions Background tasks are less sensitive to

    latency Single-purpose, reusable code Triggered by an event Retries Node.js, Python, Go, Javaalpha
  23. Four code labs Lab 1 — Store and analyse pictures

    Storing pictures in Cloud Storage, trigger a Cloud Functions instance Calling the Cloud Vision API for analysis Storing metadata in Cloud Firestore Lab 3 — Run containers on schedule At a regular interval, with Cloud Scheduler Invoking a Cloud Run instance, to create a collage of recent pictures found in Cloud Firestore Storing collage in Cloud Storage Lab 2 — Create thumbnails Cloud Storage notifications sent to Pub/Sub Triggering a Cloud Run instance via Pub/Sub Creating a thumbnail with ImageMagick, and storing back into Cloud Storage Lab 4 — Create a web frontend Creating a web application with App Engine Listing pictures from Cloud Firestore Displaying pictures & collage from Cloud Storage g.co/codelabs/serverless-workshop
  24. 04 Cloud Run

  25. Containers Any language Any library Any binary Ecosystem of base

    images .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1
  26. Containers Flexibility Serverless Velocity

  27. Cloud Run Fully managed, deploy your workloads and don’t see

    the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative Everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative.
  28. Container to production in seconds Natively Serverless One experience, where

    you want it Cloud Run Bringing serverless to containers
  29. HTTPS Endpoint Public • Website • API endpoint Private •

    Internal services • Async tasks • Mobile backend • Webhook
  30. Container contract Listen on 0.0.0.0 on port $PORT (default 8080)

    HTTP server must start < 4 min (timeout → 504) Request time < 15 min (default → 5 min) Stateless (in-memory file system, doesn’t persist) Computation only within request (No background activity)
  31. Container resources 1 vCPU per container instance (configurable to 2vCPU)

    256 MiB of memory up to a max of 2 GiB (configurable) 80 concurrent requests per container (configurable 1-80) 1000 max containers by default (configurable 1-1000) Access to a Metadata Server Sandboxed by gVisor
  32. Pay per use CPU / Memory / Requests 100ms

  33. Billable time Instance Billable Time Request 1 Start Request 1

    End Request 2 Start Request 2 End Instance Time Billable Non-billable
  34. Concurrency: up to 80 concurrent requests concurrency = 1 concurrency

    = 80
  35. Pub/Sub triggered internal services Cloud Run Cloud Pub/Sub Queue Queue

    Queue
  36. Storage triggered internal services Cloud Run Cloud Pub/Sub Queue Queue

    Queue Cloud Storage
  37. Scheduled services Command Line Interface (CLI) User Interface (UI) Scheduler

    API Cloud Run Cloud Scheduler
  38. Services part of async tasks Cloud Tasks user_registration user_levelcompleted user_inapppurchase

    user_statechange Daily activity metrics service User profile service Payment processing service Game state service
  39. Proprietary + Confidential Good fit? Constraints? Good fit Stateless HTTP

    request-response workloads Scale: way up, down to zero, bursty Specific runtime requirements: language, dependencies, etc Constraints Must use containers Decide on build process (Cloud Build, etc)
  40. Four code labs Lab 1 — Store and analyse pictures

    Storing pictures in Cloud Storage, trigger a Cloud Functions instance Calling the Cloud Vision API for analysis Storing metadata in Cloud Firestore Lab 3 — Run containers on schedule At a regular interval, with Cloud Scheduler Invoking a Cloud Run instance, to create a collage of recent pictures found in Cloud Firestore Storing collage in Cloud Storage Lab 2 — Create thumbnails Cloud Storage notifications sent to Pub/Sub Triggering a Cloud Run instance via Pub/Sub Creating a thumbnail with ImageMagick, and storing back into Cloud Storage Lab 4 — Create a web frontend Creating a web application with App Engine Listing pictures from Cloud Firestore Displaying pictures & collage from Cloud Storage g.co/codelabs/serverless-workshop
  41. Four code labs Lab 1 — Store and analyse pictures

    Storing pictures in Cloud Storage, trigger a Cloud Functions instance Calling the Cloud Vision API for analysis Storing metadata in Cloud Firestore Lab 3 — Run containers on schedule At a regular interval, with Cloud Scheduler Invoking a Cloud Run instance, to create a collage of recent pictures found in Cloud Firestore Storing collage in Cloud Storage Lab 2 — Create thumbnails Cloud Storage notifications sent to Pub/Sub Triggering a Cloud Run instance via Pub/Sub Creating a thumbnail with ImageMagick, and storing back into Cloud Storage Lab 4 — Create a web frontend Creating a web application with App Engine Listing pictures from Cloud Firestore Displaying pictures & collage from Cloud Storage g.co/codelabs/serverless-workshop
  42. 05 App Engine

  43. Dedicated Server Dedicated Server OS Application Runtime Application Virtual Hardware

    Functions Physical Hardware OS Application Runtime Application Hardware Functions Dedicated Server OS Application Runtime Application Hardware Functions Serverless OS Application Runtime Application Hardware Functions Function Cloud Functions On-Premises Virtual Machine OS Application Runtime Application Hardware Functions Infrastructure Compute Engine Containers OS Application Runtime Application Hardware Functions Container Kubernetes Engine Application Platform OS Application Runtime Application Hardware Functions Platform App Engine Highly customizable Highly managed
  44. What is App Engine? Serverless, but for your whole web

    app and not just single functions. App Engine goal: let developers focus on code, GCP handle the rest Can scale very quickly Open-source, idiomatic experience Use any library, extension, binary, or framework Python, Java, Node.js, PHP, Ruby, Go
  45. App Engine Standard (2nd gen) • Open-source, idiomatic experience •

    Use any extension, binary, or framework • Supported Runtimes ◦ Python 3.7 (3.8 in beta) ◦ Java 11 ◦ Node.js 10 (12 in beta) ◦ PHP 7.3 ◦ Go 1.12 App Engine Flexible • Runs Docker containers • Open source runtimes for Node.js, Ruby, Java, Python, Go, PHP, .NET • Also, if it runs in a Docker container, you can run it ← nearly everything ⚠Not serverless!
  46. App Engine runtimes Standard Standard 2nd generation Flexible Python 2.X,

    Java, PHP, Go Node.js 10, Python 3.7, Go 1.12, PHP 7.3, Java 11 Node.js, Ruby, Java, Python, Go, PHP, .NET & custom container images Sandboxed processes Runs on managed VMs GAE specific APIs + GCP APIs Use GCP APIs directly No binary modules / libraries any extension, binary, or framework open-source, idiomatic experience
  47. Good fit HTTP/S request-response Stateless serving applications Scaling to high

    traffic Constraints Standard (1st gen) • Runtimes for Python, Java, PHP, Go, Node.js Standard (2nd gen) • Can use use binary extensions • Python, Node.js, Java, Go, Ruby, PHP Flexible runtimes • Inherit Docker constraints • Not best for very low traffic sites (no scale to 0) • Slow deploy times Good fit? Constraints?
  48. Why choose... App Engine The right abstraction: use a web

    framework that supports routes and HTTP methods Web traffic requires minimal latency Custom domains Java, Python, Node.js, Go, PHP, Ruby
  49. Concurrency & scaling Traffic Traffic •Scale up/down •Scale-up may lead

    to "cold starts" •One vs. multiple concurrent requests •Traffic splitting across versions •CDN static asset serving
  50. Four code labs Lab 1 — Store and analyse pictures

    Storing pictures in Cloud Storage, trigger a Cloud Functions instance Calling the Cloud Vision API for analysis Storing metadata in Cloud Firestore Lab 3 — Run containers on schedule At a regular interval, with Cloud Scheduler Invoking a Cloud Run instance, to create a collage of recent pictures found in Cloud Firestore Storing collage in Cloud Storage Lab 2 — Create thumbnails Cloud Storage notifications sent to Pub/Sub Triggering a Cloud Run instance via Pub/Sub Creating a thumbnail with ImageMagick, and storing back into Cloud Storage Lab 4 — Create a web frontend Creating a web application with App Engine Listing pictures from Cloud Firestore Displaying pictures & collage from Cloud Storage g.co/codelabs/serverless-workshop
  51. Serverless Workshop Quest - 5 labs, 5 hours For more

    Serverless labs you can enroll in the Serverless Workshop Quest. After completing the first lab you will gain free access to Qwiklabs for 30 days. 1. Follow the provided link: https://google.qwiklabs.com/quests/98?qlcampaign=5m-GCPSW-98 2. You will be prompted to log into Qwiklabs. If you did not already have a Qwiklabs account, you’ll need to create one. Creating an account is always free. 3. Then, click on the “Enroll in this Quest” button. 4. Perform any lab from the Quest. 5. After completion, when you click “End lab”, you will get a free pass in your account to take the lab. 6. You will be able to take the lab free of charge.
  52. Proprietary + Confidential Thank you! Guillaume Laforge — @glaforge Mete

    Atamel — @meteatamel