Slide 1
Slide 1 text
Leveraging Cloud Portability
with Fastly
Jun Sakata / @sakajunquality
Ubie, inc.
October 23, 2019. Fastly Yamagoya Meetup
Slide 2
Slide 2 text
Ubie, Inc.
- Medical Startup
- Founded in 2017
- 30+ employee
- 15+ engineers
- Business
- Clinical decision making support
- Operation efficiency support
Slide 3
Slide 3 text
Agenda
How Fastly helps
- Migrating services across vendors
- Migrating services within a cloud
- Making more portability in architecture
Slide 4
Slide 4 text
- Kubernetes: Open-source container orchestration platform.
- Istio: Open-source Service Mesh platform.
- Envoy: Open-source service proxy. Used in Istio dataplane.
- GKE: GCP’s managed version of GKE.
- Istio on GKE: GKE’s addon to install Istio.
Terminologies
Slide 5
Slide 5 text
Migrating services across vendors
Slide 6
Slide 6 text
Migrating services across vendors
- OnPrem to cloud
- One cloud to another
- etc...
Slide 7
Slide 7 text
1st Migration in Ubie
- Migrated from Heroku to GCP
- Microservices w/ Kubernetes (GKE)
- Data and ML services
- Google’s Startup Support
- etc...
Slide 8
Slide 8 text
1st Migration in Ubie
- Two existing service
- Frontend service
- Old backend service
- Older than the company!
- One new service
- New backend service
- in Kotlin
Slide 9
Slide 9 text
1st Migration in Ubie
Frontend
Old
Backend
Frontend
Old
Backend
New
Backend
Slide 10
Slide 10 text
Why Fastly?
- Fast Configuration Activation
- L7 Load Balancing
- CDN Interconnect with GCP
Slide 11
Slide 11 text
Fast Configuration Activation
- DNS could take much time
- Required a quick rollout and rollback
- Not enough e2e test (at that time)
- Personally not enough knowledge with applications (at that time)
- Rolled-back once actually
Slide 12
Slide 12 text
L7 Load Balancing
- Outgoing traffic is sometimes limited in medical institutions
- e.g. Domain name, IP address etc…
- Meanwhile we want to use multiple services
- Several backends are running and called from client-side
Slide 13
Slide 13 text
L7 Load Balancing
- With Fastly, we’re using L7 path based routing
Service A
Service B
Service C
/
/log
/static
Client
Slide 14
Slide 14 text
L7 Load Balancing
- With Fastly, we’re using L7 path based routing
- Can choose the best backend
Service A
Service B
Service C
/
/log
/static
Client
Slide 15
Slide 15 text
How we migrated
Frontend
Old
Backend
Frontend
Old
Backend
New
Backend
Slide 16
Slide 16 text
How we migrated
Frontend
Old
Backend
Frontend
Old
Backend
New
Backend
Changing the origin
Slide 17
Slide 17 text
Good points with Fastly
- Quick Rolled-out/back
- Dev Experience
- Support
Slide 18
Slide 18 text
Dev Experience
- Logging
- Access log export to Google BigQuery
- Most of logs are stored in BigQuery
- Configuration via API
- terraform
Slide 19
Slide 19 text
Support Case
- Satisfactional technical support
- Both in English and Japanese
Slide 20
Slide 20 text
Migrating services within a cloud
Slide 21
Slide 21 text
https://twitter.com/kelseyhightower/status/935252923721793536
Slide 22
Slide 22 text
Architecture Migration
- Increase in # of users
- Increase in # of services
- Increase in # of developers
- Changing softwares
- Insufficient design
- etc...
Slide 23
Slide 23 text
2nd Migration in Ubie
- Changing Kubernetes (GKE) cluster
- Installing Istio
- Public to Private Cluster
- Some new features
- Changing logical deployment
- namespaces
Slide 24
Slide 24 text
Istio
- Open-source Service Mesh platform
- Originally from Google and etc…
- Service Discovery / Traffic Control / Observability
Slide 25
Slide 25 text
Why Istio?
- Single Ingress-Gateway
- More and more services are deployed
- 2 services -> ~10 services
- Client-side load balancing and its telemetry
- Internal traffic has increased
Slide 26
Slide 26 text
Why Istio? - Ingress Gateway
Service A
Service B
Service C
Service A
Service B
Service C
Ingress
Gateway
... ...
Redundant origin
config... Single origin config
Separated Ingress
LBs...
Slide 27
Slide 27 text
Why Istio? - Internal traffic
Service A Service B
Service A Service B
ILB
It used to be using internal GCLB
Need an additional configurations...
Slide 28
Slide 28 text
Why Istio? - Internal traffic
Slide 29
Slide 29 text
Logical Deployment
- namespace = virtual separation of cluster
- Frequently used with access control
- namespace is separated into teams at first
- => startup’s teams change often!!
- Change to 1 namespace for 1 service
Slide 30
Slide 30 text
2nd Migrations
namespace
Services
Services
Service Service
Separate
namespaces
Istio installed
Slide 31
Slide 31 text
2nd Migrations
namespace
Services
Services
Service Service
Changed several
backends to single
backend (Istio Ingress
Gateway)
Slide 32
Slide 32 text
Good points with Fastly
- Quick Rolled-out/back
- Dev Experience
- Support
- Security
Slide 33
Slide 33 text
Security
- WAF
- Same rule for different backend infra
- +2
- Customer on-boarding process
- Not white-boxed rules
- Access Restriction with VCL
- Restrictions are not affected by backend infra
Slide 34
Slide 34 text
Making more portability
Slide 35
Slide 35 text
The “God” Cluster
- Create a single big cluster
- Maintain it carefully and nicely
Slide 36
Slide 36 text
The “God” Cluster
- Pros
- Resource Utilization
- Less things to consider e.g. CI/CD Canary release ...
- Less things to manage
- Cons
- Version updates
- Testing new features
- Cluster outage directly goes to service disruption
Slide 37
Slide 37 text
Updates
- Kubernetes (OSS)
- Quortaly major updates
- Security Patches
- GKE (Managed)
- OSS + GCP specific updates
- Istio (OSS)
- Frequent updates
- Istio on GKE (Half-managed)
Slide 38
Slide 38 text
Updates
- Kubernetes (OSS)
- Quortaly major updates
- Security Patches
- GKE (Managed)
- OSS + GCP specific updates
- Istio (OSS)
- Frequent updates
- Istio on GKE (Half-managed)
So many chance to destroy the whole service!!
Slide 39
Slide 39 text
3rd Migration in Ubie
- Migrating to multi-clusters
- Like a East-West traffic
- Migrating to OSS Istio
- “Istio on GKE” has limitations…
- Test alpha/experimental features in production safely
Slide 40
Slide 40 text
3rd Migration in Ubie
Service Service
Service Service
Tenant 1
Tenant 2
Round Robin
Do not allow
cross-cluster
internal requests
Slide 41
Slide 41 text
If a disruption happens in either of clusters...
Service Service
Service Service
Tenant 1
Tenant 2
Slide 42
Slide 42 text
Remove the whole cluster
Service Service
Service Service
Tenant 1
Tenant 2
The whole service
keeps running with the
rest of clusters!
Slide 43
Slide 43 text
Traffic Management
- External Requests
- Internal Requests
Slide 44
Slide 44 text
External Requests
- Load Balance with Fastly
- Origin to Istio Ingress Gateway
Origin configuration
Slide 45
Slide 45 text
External Requests
Still working with the following problems...
- Health Check
- Kubernetes cluster level?
- Each service level?
- Session Stickiness
- Definition of client...
- Canary-based rollouts
Slide 46
Slide 46 text
Not Multi Cluster Ingress with GKE?
- https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress
- Ingress = L7 LB, Istio use L4 LB
- Still in beta
- Small chance to use another cloud
Slide 47
Slide 47 text
Internal Requests
- Client-side load balancing with envoy in Istio
- All the internal requests stays in the cluster
Slide 48
Slide 48 text
Good points with Fastly
- Quick Rolled-out/back
- Dev Experience
- Support
- Security
- Custom VCL
Slide 49
Slide 49 text
Custom VCL
- Implement domain specific features
- e.g. sticky load balancing, WAF bypassing etc...
Slide 50
Slide 50 text
Access Log
- Fastly’s external access log
- + envoy’s internal access log
Slide 51
Slide 51 text
4th Migration in Ubie
- Frontend performance? maybe
Slide 52
Slide 52 text
Takeaways
Slide 53
Slide 53 text
Takeaways
- With Fastly, it’s really easy to migrate services and improve
architecture.
- Even if architecture design is not good enough
- In Ubie, external traffic management w/ Fastly + internal traffic
management w/ Istio combination works well
Slide 54
Slide 54 text
So many Good points with Fastly!
- Quick Rolled-out/back
- Dev Experience
- Support
- Security
- Custom VCL
Slide 55
Slide 55 text
Thank You