Fastly Yamagoya Meetup: Leveraging Cloud Portability with Fastly

Slide 1

Slide 1 text

Leveraging Cloud Portability with Fastly Jun Sakata / @sakajunquality Ubie, inc. October 23, 2019. Fastly Yamagoya Meetup

Slide 2

Slide 2 text

Ubie, Inc. - Medical Startup - Founded in 2017 - 30+ employee - 15+ engineers - Business - Clinical decision making support - Operation efficiency support

Slide 3

Slide 3 text

Agenda How Fastly helps - Migrating services across vendors - Migrating services within a cloud - Making more portability in architecture

Slide 4

Slide 4 text

- Kubernetes: Open-source container orchestration platform. - Istio: Open-source Service Mesh platform. - Envoy: Open-source service proxy. Used in Istio dataplane. - GKE: GCP’s managed version of GKE. - Istio on GKE: GKE’s addon to install Istio. Terminologies

Slide 5

Slide 5 text

Migrating services across vendors

Slide 6

Slide 6 text

Migrating services across vendors - OnPrem to cloud - One cloud to another - etc...

Slide 7

Slide 7 text

1st Migration in Ubie - Migrated from Heroku to GCP - Microservices w/ Kubernetes (GKE) - Data and ML services - Google’s Startup Support - etc...

Slide 8

Slide 8 text

1st Migration in Ubie - Two existing service - Frontend service - Old backend service - Older than the company! - One new service - New backend service - in Kotlin

Slide 9

Slide 9 text

1st Migration in Ubie Frontend Old Backend Frontend Old Backend New Backend

Slide 10

Slide 10 text

Why Fastly? - Fast Configuration Activation - L7 Load Balancing - CDN Interconnect with GCP

Slide 11

Slide 11 text

Fast Configuration Activation - DNS could take much time - Required a quick rollout and rollback - Not enough e2e test (at that time) - Personally not enough knowledge with applications (at that time) - Rolled-back once actually

Slide 12

Slide 12 text

L7 Load Balancing - Outgoing traffic is sometimes limited in medical institutions - e.g. Domain name, IP address etc… - Meanwhile we want to use multiple services - Several backends are running and called from client-side

Slide 13

Slide 13 text

L7 Load Balancing - With Fastly, we’re using L7 path based routing Service A Service B Service C / /log /static Client

Slide 14

Slide 14 text

L7 Load Balancing - With Fastly, we’re using L7 path based routing - Can choose the best backend Service A Service B Service C / /log /static Client

Slide 15

Slide 15 text

How we migrated Frontend Old Backend Frontend Old Backend New Backend

Slide 16

Slide 16 text

How we migrated Frontend Old Backend Frontend Old Backend New Backend Changing the origin

Slide 17

Slide 17 text

Good points with Fastly - Quick Rolled-out/back - Dev Experience - Support

Slide 18

Slide 18 text

Dev Experience - Logging - Access log export to Google BigQuery - Most of logs are stored in BigQuery - Configuration via API - terraform

Slide 19

Slide 19 text

Support Case - Satisfactional technical support - Both in English and Japanese

Slide 20

Slide 20 text

Migrating services within a cloud

Slide 21

Slide 21 text

https://twitter.com/kelseyhightower/status/935252923721793536

Slide 22

Slide 22 text

Architecture Migration - Increase in # of users - Increase in # of services - Increase in # of developers - Changing softwares - Insufficient design - etc...

Slide 23

Slide 23 text

2nd Migration in Ubie - Changing Kubernetes (GKE) cluster - Installing Istio - Public to Private Cluster - Some new features - Changing logical deployment - namespaces

Slide 24

Slide 24 text

Istio - Open-source Service Mesh platform - Originally from Google and etc… - Service Discovery / Traffic Control / Observability

Slide 25

Slide 25 text

Why Istio? - Single Ingress-Gateway - More and more services are deployed - 2 services -> ~10 services - Client-side load balancing and its telemetry - Internal traffic has increased

Slide 26

Slide 26 text

Why Istio? - Ingress Gateway Service A Service B Service C Service A Service B Service C Ingress Gateway ... ... Redundant origin config... Single origin config Separated Ingress LBs...

Slide 27

Slide 27 text

Why Istio? - Internal traffic Service A Service B Service A Service B ILB It used to be using internal GCLB Need an additional configurations...

Slide 28

Slide 28 text

Why Istio? - Internal traffic

Slide 29

Slide 29 text

Logical Deployment - namespace = virtual separation of cluster - Frequently used with access control - namespace is separated into teams at first - => startup’s teams change often!! - Change to 1 namespace for 1 service

Slide 30

Slide 30 text

2nd Migrations namespace Services Services Service Service Separate namespaces Istio installed

Slide 31

Slide 31 text

2nd Migrations namespace Services Services Service Service Changed several backends to single backend (Istio Ingress Gateway)

Slide 32

Slide 32 text

Good points with Fastly - Quick Rolled-out/back - Dev Experience - Support - Security

Slide 33

Slide 33 text

Security - WAF - Same rule for different backend infra - +2 - Customer on-boarding process - Not white-boxed rules - Access Restriction with VCL - Restrictions are not affected by backend infra

Slide 34

Slide 34 text

Making more portability

Slide 35

Slide 35 text

The “God” Cluster - Create a single big cluster - Maintain it carefully and nicely

Slide 36

Slide 36 text

The “God” Cluster - Pros - Resource Utilization - Less things to consider e.g. CI/CD Canary release ... - Less things to manage - Cons - Version updates - Testing new features - Cluster outage directly goes to service disruption

Slide 37

Slide 37 text

Updates - Kubernetes (OSS) - Quortaly major updates - Security Patches - GKE (Managed) - OSS + GCP specific updates - Istio (OSS) - Frequent updates - Istio on GKE (Half-managed)

Slide 38

Slide 38 text

Updates - Kubernetes (OSS) - Quortaly major updates - Security Patches - GKE (Managed) - OSS + GCP specific updates - Istio (OSS) - Frequent updates - Istio on GKE (Half-managed) So many chance to destroy the whole service!!

Slide 39

Slide 39 text

3rd Migration in Ubie - Migrating to multi-clusters - Like a East-West traffic - Migrating to OSS Istio - “Istio on GKE” has limitations… - Test alpha/experimental features in production safely

Slide 40

Slide 40 text

3rd Migration in Ubie Service Service Service Service Tenant 1 Tenant 2 Round Robin Do not allow cross-cluster internal requests

Slide 41

Slide 41 text

If a disruption happens in either of clusters... Service Service Service Service Tenant 1 Tenant 2

Slide 42

Slide 42 text

Remove the whole cluster Service Service Service Service Tenant 1 Tenant 2 The whole service keeps running with the rest of clusters!

Slide 43

Slide 43 text

Traffic Management - External Requests - Internal Requests

Slide 44

Slide 44 text

External Requests - Load Balance with Fastly - Origin to Istio Ingress Gateway Origin configuration

Slide 45

Slide 45 text

External Requests Still working with the following problems... - Health Check - Kubernetes cluster level? - Each service level? - Session Stickiness - Definition of client... - Canary-based rollouts

Slide 46

Slide 46 text

Not Multi Cluster Ingress with GKE? - https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress - Ingress = L7 LB, Istio use L4 LB - Still in beta - Small chance to use another cloud

Slide 47

Slide 47 text

Internal Requests - Client-side load balancing with envoy in Istio - All the internal requests stays in the cluster

Slide 48

Slide 48 text

Good points with Fastly - Quick Rolled-out/back - Dev Experience - Support - Security - Custom VCL

Slide 49

Slide 49 text

Custom VCL - Implement domain specific features - e.g. sticky load balancing, WAF bypassing etc...

Slide 50

Slide 50 text

Access Log - Fastly’s external access log - + envoy’s internal access log

Slide 51

Slide 51 text

4th Migration in Ubie - Frontend performance? maybe

Slide 52

Slide 52 text

Takeaways

Slide 53

Slide 53 text

Takeaways - With Fastly, it’s really easy to migrate services and improve architecture. - Even if architecture design is not good enough - In Ubie, external traffic management w/ Fastly + internal traffic management w/ Istio combination works well

Slide 54

Slide 54 text

So many Good points with Fastly! - Quick Rolled-out/back - Dev Experience - Support - Security - Custom VCL

Slide 55

Slide 55 text

Thank You