Fastly Yamagoya Meetup: Leveraging Cloud Portability with Fastly

Fastly Yamagoya Meetup: Leveraging Cloud Portability with Fastly

923237754890d536819892ad42451555?s=128

sakajunquality

October 23, 2019
Tweet

Transcript

  1. Leveraging Cloud Portability with Fastly Jun Sakata / @sakajunquality Ubie,

    inc. October 23, 2019. Fastly Yamagoya Meetup
  2. Ubie, Inc. - Medical Startup - Founded in 2017 -

    30+ employee - 15+ engineers - Business - Clinical decision making support - Operation efficiency support
  3. Agenda How Fastly helps - Migrating services across vendors -

    Migrating services within a cloud - Making more portability in architecture
  4. - Kubernetes: Open-source container orchestration platform. - Istio: Open-source Service

    Mesh platform. - Envoy: Open-source service proxy. Used in Istio dataplane. - GKE: GCP’s managed version of GKE. - Istio on GKE: GKE’s addon to install Istio. Terminologies
  5. Migrating services across vendors

  6. Migrating services across vendors - OnPrem to cloud - One

    cloud to another - etc...
  7. 1st Migration in Ubie - Migrated from Heroku to GCP

    - Microservices w/ Kubernetes (GKE) - Data and ML services - Google’s Startup Support - etc...
  8. 1st Migration in Ubie - Two existing service - Frontend

    service - Old backend service - Older than the company! - One new service - New backend service - in Kotlin
  9. 1st Migration in Ubie Frontend Old Backend Frontend Old Backend

    New Backend
  10. Why Fastly? - Fast Configuration Activation - L7 Load Balancing

    - CDN Interconnect with GCP
  11. Fast Configuration Activation - DNS could take much time -

    Required a quick rollout and rollback - Not enough e2e test (at that time) - Personally not enough knowledge with applications (at that time) - Rolled-back once actually
  12. L7 Load Balancing - Outgoing traffic is sometimes limited in

    medical institutions - e.g. Domain name, IP address etc… - Meanwhile we want to use multiple services - Several backends are running and called from client-side
  13. L7 Load Balancing - With Fastly, we’re using L7 path

    based routing Service A Service B Service C / /log /static Client
  14. L7 Load Balancing - With Fastly, we’re using L7 path

    based routing - Can choose the best backend Service A Service B Service C / /log /static Client
  15. How we migrated Frontend Old Backend Frontend Old Backend New

    Backend
  16. How we migrated Frontend Old Backend Frontend Old Backend New

    Backend Changing the origin
  17. Good points with Fastly - Quick Rolled-out/back - Dev Experience

    - Support
  18. Dev Experience - Logging - Access log export to Google

    BigQuery - Most of logs are stored in BigQuery - Configuration via API - terraform
  19. Support Case - Satisfactional technical support - Both in English

    and Japanese
  20. Migrating services within a cloud

  21. https://twitter.com/kelseyhightower/status/935252923721793536

  22. Architecture Migration - Increase in # of users - Increase

    in # of services - Increase in # of developers - Changing softwares - Insufficient design - etc...
  23. 2nd Migration in Ubie - Changing Kubernetes (GKE) cluster -

    Installing Istio - Public to Private Cluster - Some new features - Changing logical deployment - namespaces
  24. Istio - Open-source Service Mesh platform - Originally from Google

    and etc… - Service Discovery / Traffic Control / Observability
  25. Why Istio? - Single Ingress-Gateway - More and more services

    are deployed - 2 services -> ~10 services - Client-side load balancing and its telemetry - Internal traffic has increased
  26. Why Istio? - Ingress Gateway Service A Service B Service

    C Service A Service B Service C Ingress Gateway ... ... Redundant origin config... Single origin config Separated Ingress LBs...
  27. Why Istio? - Internal traffic Service A Service B Service

    A Service B ILB It used to be using internal GCLB Need an additional configurations...
  28. Why Istio? - Internal traffic

  29. Logical Deployment - namespace = virtual separation of cluster -

    Frequently used with access control - namespace is separated into teams at first - => startup’s teams change often!! - Change to 1 namespace for 1 service
  30. 2nd Migrations namespace Services Services Service Service Separate namespaces Istio

    installed
  31. 2nd Migrations namespace Services Services Service Service Changed several backends

    to single backend (Istio Ingress Gateway)
  32. Good points with Fastly - Quick Rolled-out/back - Dev Experience

    - Support - Security
  33. Security - WAF - Same rule for different backend infra

    - +2 - Customer on-boarding process - Not white-boxed rules - Access Restriction with VCL - Restrictions are not affected by backend infra
  34. Making more portability

  35. The “God” Cluster - Create a single big cluster -

    Maintain it carefully and nicely
  36. The “God” Cluster - Pros - Resource Utilization - Less

    things to consider e.g. CI/CD Canary release ... - Less things to manage - Cons - Version updates - Testing new features - Cluster outage directly goes to service disruption
  37. Updates - Kubernetes (OSS) - Quortaly major updates - Security

    Patches - GKE (Managed) - OSS + GCP specific updates - Istio (OSS) - Frequent updates - Istio on GKE (Half-managed)
  38. Updates - Kubernetes (OSS) - Quortaly major updates - Security

    Patches - GKE (Managed) - OSS + GCP specific updates - Istio (OSS) - Frequent updates - Istio on GKE (Half-managed) So many chance to destroy the whole service!!
  39. 3rd Migration in Ubie - Migrating to multi-clusters - Like

    a East-West traffic - Migrating to OSS Istio - “Istio on GKE” has limitations… - Test alpha/experimental features in production safely
  40. 3rd Migration in Ubie Service Service Service Service Tenant 1

    Tenant 2 Round Robin Do not allow cross-cluster internal requests
  41. If a disruption happens in either of clusters... Service Service

    Service Service Tenant 1 Tenant 2
  42. Remove the whole cluster Service Service Service Service Tenant 1

    Tenant 2 The whole service keeps running with the rest of clusters!
  43. Traffic Management - External Requests - Internal Requests

  44. External Requests - Load Balance with Fastly - Origin to

    Istio Ingress Gateway Origin configuration
  45. External Requests Still working with the following problems... - Health

    Check - Kubernetes cluster level? - Each service level? - Session Stickiness - Definition of client... - Canary-based rollouts
  46. Not Multi Cluster Ingress with GKE? - https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress - Ingress

    = L7 LB, Istio use L4 LB - Still in beta - Small chance to use another cloud
  47. Internal Requests - Client-side load balancing with envoy in Istio

    - All the internal requests stays in the cluster
  48. Good points with Fastly - Quick Rolled-out/back - Dev Experience

    - Support - Security - Custom VCL
  49. Custom VCL - Implement domain specific features - e.g. sticky

    load balancing, WAF bypassing etc...
  50. Access Log - Fastly’s external access log - + envoy’s

    internal access log
  51. 4th Migration in Ubie - Frontend performance? maybe

  52. Takeaways

  53. Takeaways - With Fastly, it’s really easy to migrate services

    and improve architecture. - Even if architecture design is not good enough - In Ubie, external traffic management w/ Fastly + internal traffic management w/ Istio combination works well
  54. So many Good points with Fastly! - Quick Rolled-out/back -

    Dev Experience - Support - Security - Custom VCL
  55. Thank You