Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fastly Yamagoya Meetup: Leveraging Cloud Portability with Fastly

Fastly Yamagoya Meetup: Leveraging Cloud Portability with Fastly

sakajunquality

October 23, 2019
Tweet

More Decks by sakajunquality

Other Decks in Technology

Transcript

  1. Leveraging Cloud Portability
    with Fastly
    Jun Sakata / @sakajunquality
    Ubie, inc.
    October 23, 2019. Fastly Yamagoya Meetup

    View full-size slide

  2. Ubie, Inc.
    - Medical Startup
    - Founded in 2017
    - 30+ employee
    - 15+ engineers
    - Business
    - Clinical decision making support
    - Operation efficiency support

    View full-size slide

  3. Agenda
    How Fastly helps
    - Migrating services across vendors
    - Migrating services within a cloud
    - Making more portability in architecture

    View full-size slide

  4. - Kubernetes: Open-source container orchestration platform.
    - Istio: Open-source Service Mesh platform.
    - Envoy: Open-source service proxy. Used in Istio dataplane.
    - GKE: GCP’s managed version of GKE.
    - Istio on GKE: GKE’s addon to install Istio.
    Terminologies

    View full-size slide

  5. Migrating services across vendors

    View full-size slide

  6. Migrating services across vendors
    - OnPrem to cloud
    - One cloud to another
    - etc...

    View full-size slide

  7. 1st Migration in Ubie
    - Migrated from Heroku to GCP
    - Microservices w/ Kubernetes (GKE)
    - Data and ML services
    - Google’s Startup Support
    - etc...

    View full-size slide

  8. 1st Migration in Ubie
    - Two existing service
    - Frontend service
    - Old backend service
    - Older than the company!
    - One new service
    - New backend service
    - in Kotlin

    View full-size slide

  9. 1st Migration in Ubie
    Frontend
    Old
    Backend
    Frontend
    Old
    Backend
    New
    Backend

    View full-size slide

  10. Why Fastly?
    - Fast Configuration Activation
    - L7 Load Balancing
    - CDN Interconnect with GCP

    View full-size slide

  11. Fast Configuration Activation
    - DNS could take much time
    - Required a quick rollout and rollback
    - Not enough e2e test (at that time)
    - Personally not enough knowledge with applications (at that time)
    - Rolled-back once actually

    View full-size slide

  12. L7 Load Balancing
    - Outgoing traffic is sometimes limited in medical institutions
    - e.g. Domain name, IP address etc…
    - Meanwhile we want to use multiple services
    - Several backends are running and called from client-side

    View full-size slide

  13. L7 Load Balancing
    - With Fastly, we’re using L7 path based routing
    Service A
    Service B
    Service C
    /
    /log
    /static
    Client

    View full-size slide

  14. L7 Load Balancing
    - With Fastly, we’re using L7 path based routing
    - Can choose the best backend
    Service A
    Service B
    Service C
    /
    /log
    /static
    Client

    View full-size slide

  15. How we migrated
    Frontend
    Old
    Backend
    Frontend
    Old
    Backend
    New
    Backend

    View full-size slide

  16. How we migrated
    Frontend
    Old
    Backend
    Frontend
    Old
    Backend
    New
    Backend
    Changing the origin

    View full-size slide

  17. Good points with Fastly
    - Quick Rolled-out/back
    - Dev Experience
    - Support

    View full-size slide

  18. Dev Experience
    - Logging
    - Access log export to Google BigQuery
    - Most of logs are stored in BigQuery
    - Configuration via API
    - terraform

    View full-size slide

  19. Support Case
    - Satisfactional technical support
    - Both in English and Japanese

    View full-size slide

  20. Migrating services within a cloud

    View full-size slide

  21. https://twitter.com/kelseyhightower/status/935252923721793536

    View full-size slide

  22. Architecture Migration
    - Increase in # of users
    - Increase in # of services
    - Increase in # of developers
    - Changing softwares
    - Insufficient design
    - etc...

    View full-size slide

  23. 2nd Migration in Ubie
    - Changing Kubernetes (GKE) cluster
    - Installing Istio
    - Public to Private Cluster
    - Some new features
    - Changing logical deployment
    - namespaces

    View full-size slide

  24. Istio
    - Open-source Service Mesh platform
    - Originally from Google and etc…
    - Service Discovery / Traffic Control / Observability

    View full-size slide

  25. Why Istio?
    - Single Ingress-Gateway
    - More and more services are deployed
    - 2 services -> ~10 services
    - Client-side load balancing and its telemetry
    - Internal traffic has increased

    View full-size slide

  26. Why Istio? - Ingress Gateway
    Service A
    Service B
    Service C
    Service A
    Service B
    Service C
    Ingress
    Gateway
    ... ...
    Redundant origin
    config... Single origin config
    Separated Ingress
    LBs...

    View full-size slide

  27. Why Istio? - Internal traffic
    Service A Service B
    Service A Service B
    ILB
    It used to be using internal GCLB
    Need an additional configurations...

    View full-size slide

  28. Why Istio? - Internal traffic

    View full-size slide

  29. Logical Deployment
    - namespace = virtual separation of cluster
    - Frequently used with access control
    - namespace is separated into teams at first
    - => startup’s teams change often!!
    - Change to 1 namespace for 1 service

    View full-size slide

  30. 2nd Migrations
    namespace
    Services
    Services
    Service Service
    Separate
    namespaces
    Istio installed

    View full-size slide

  31. 2nd Migrations
    namespace
    Services
    Services
    Service Service
    Changed several
    backends to single
    backend (Istio Ingress
    Gateway)

    View full-size slide

  32. Good points with Fastly
    - Quick Rolled-out/back
    - Dev Experience
    - Support
    - Security

    View full-size slide

  33. Security
    - WAF
    - Same rule for different backend infra
    - +2
    - Customer on-boarding process
    - Not white-boxed rules
    - Access Restriction with VCL
    - Restrictions are not affected by backend infra

    View full-size slide

  34. Making more portability

    View full-size slide

  35. The “God” Cluster
    - Create a single big cluster
    - Maintain it carefully and nicely

    View full-size slide

  36. The “God” Cluster
    - Pros
    - Resource Utilization
    - Less things to consider e.g. CI/CD Canary release ...
    - Less things to manage
    - Cons
    - Version updates
    - Testing new features
    - Cluster outage directly goes to service disruption

    View full-size slide

  37. Updates
    - Kubernetes (OSS)
    - Quortaly major updates
    - Security Patches
    - GKE (Managed)
    - OSS + GCP specific updates
    - Istio (OSS)
    - Frequent updates
    - Istio on GKE (Half-managed)

    View full-size slide

  38. Updates
    - Kubernetes (OSS)
    - Quortaly major updates
    - Security Patches
    - GKE (Managed)
    - OSS + GCP specific updates
    - Istio (OSS)
    - Frequent updates
    - Istio on GKE (Half-managed)
    So many chance to destroy the whole service!!

    View full-size slide

  39. 3rd Migration in Ubie
    - Migrating to multi-clusters
    - Like a East-West traffic
    - Migrating to OSS Istio
    - “Istio on GKE” has limitations…
    - Test alpha/experimental features in production safely

    View full-size slide

  40. 3rd Migration in Ubie
    Service Service
    Service Service
    Tenant 1
    Tenant 2
    Round Robin
    Do not allow
    cross-cluster
    internal requests

    View full-size slide

  41. If a disruption happens in either of clusters...
    Service Service
    Service Service
    Tenant 1
    Tenant 2

    View full-size slide

  42. Remove the whole cluster
    Service Service
    Service Service
    Tenant 1
    Tenant 2
    The whole service
    keeps running with the
    rest of clusters!

    View full-size slide

  43. Traffic Management
    - External Requests
    - Internal Requests

    View full-size slide

  44. External Requests
    - Load Balance with Fastly
    - Origin to Istio Ingress Gateway
    Origin configuration

    View full-size slide

  45. External Requests
    Still working with the following problems...
    - Health Check
    - Kubernetes cluster level?
    - Each service level?
    - Session Stickiness
    - Definition of client...
    - Canary-based rollouts

    View full-size slide

  46. Not Multi Cluster Ingress with GKE?
    - https://cloud.google.com/kubernetes-engine/docs/how-to/multi-cluster-ingress
    - Ingress = L7 LB, Istio use L4 LB
    - Still in beta
    - Small chance to use another cloud

    View full-size slide

  47. Internal Requests
    - Client-side load balancing with envoy in Istio
    - All the internal requests stays in the cluster

    View full-size slide

  48. Good points with Fastly
    - Quick Rolled-out/back
    - Dev Experience
    - Support
    - Security
    - Custom VCL

    View full-size slide

  49. Custom VCL
    - Implement domain specific features
    - e.g. sticky load balancing, WAF bypassing etc...

    View full-size slide

  50. Access Log
    - Fastly’s external access log
    - + envoy’s internal access log

    View full-size slide

  51. 4th Migration in Ubie
    - Frontend performance? maybe

    View full-size slide

  52. Takeaways
    - With Fastly, it’s really easy to migrate services and improve
    architecture.
    - Even if architecture design is not good enough
    - In Ubie, external traffic management w/ Fastly + internal traffic
    management w/ Istio combination works well

    View full-size slide

  53. So many Good points with Fastly!
    - Quick Rolled-out/back
    - Dev Experience
    - Support
    - Security
    - Custom VCL

    View full-size slide