Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MTC2018 - Microservices Platform at Mercari

mercari
October 04, 2018

MTC2018 - Microservices Platform at Mercari

Speaker: 中島 大一(Taichi Nakashima)

Mercariではサービスと組織の拡大に合わせてMonolithアーキテクチャからMicroservicesアーキテクチャへの移行を進めています.Microservices Platform TeamはこのMicroservicesのためのPlatformの構築を行っています.開発者のDeveloper Productivityを最大限に高め各チームがオーナーシップを持ち独立して高速にサービス開発サイクルを回せるようなPlatformを目指しています.本セッションではMicroservices Platform Teamの取り組みを紹介します。

As Mercari continues to grow as a service and organization, we decided it was time to make the move from a monolith architecture to microservices. The Microservices Platform Team is the team responsible for building the platform for this. The goal is to create a platform that allows developers to maximize their productivity for a rapid service development cycle where teams are independent and have ownership over their projects. In this session, Nakashima will introduce the initiatives the Microservices Platform Team are taking to achieve this goal.

mercari

October 04, 2018
Tweet

More Decks by mercari

Other Decks in Technology

Transcript

  1. Microservices Platform at Mercari
    Taichi Nakashima
    Tech Lead (Microservices platform)

    View full-size slide

  2. Tech Lead
    Microservices platform
    Taichi Nakashima

    View full-size slide

  3. Microservices
    https://www.flickr.com/photos/ronduong/37605344086

    View full-size slide

  4. 1000
    Engineers Org.
    https://www.flickr.com/photos/owyzzz/3626718695

    View full-size slide

  5. Speed
    https://www.flickr.com/photos/szymipl/6093250294

    View full-size slide

  6. https://itrevolution.com/book/accelerate/
    ACCELERATE
    The Science of Lean Software
    and DevOps: Building and
    Scaling High Performing
    Technology Organizations

    View full-size slide

  7. https://itrevolution.com/book/accelerate/

    View full-size slide

  8. https://itrevolution.com/book/accelerate/

    View full-size slide

  9. https://itrevolution.com/book/accelerate/
    Organization Design

    View full-size slide

  10. Develop
    Monolith
    Test Deploy Operate

    View full-size slide

  11. Develop
    Monolith
    Test Deploy Operate
    Backend Team
    QA team SRE team

    View full-size slide

  12. Test Deploy Operate
    Backend Team
    QA team SRE team
    Develop
    Service B
    Develop
    Service A
    Develop
    Service C

    View full-size slide

  13. Test Deploy Operate
    Backend Team
    SRE team
    Develop
    Service B
    Develop
    Service A
    Develop
    Service C
    QA team

    View full-size slide

  14. Test Deploy Operate
    Backend Team
    SRE team
    Develop
    Service B
    Develop
    Service A
    Develop
    Service C
    QA team

    View full-size slide

  15. Test Deploy Operate
    Backend Team
    SRE team
    Develop
    Service B
    Develop
    Service A
    Develop
    Service C
    QA team

    View full-size slide

  16. Develop
    Service B
    Test Deploy Operate
    Service B Team
    Develop
    Service A
    Test Deploy Operate
    Service A Team
    Develop
    Service C
    Test Deploy Operate
    Service C Team

    View full-size slide

  17. https://medium.com/netflix-techblog/full-cycle-developers-at-netflix-a08c31f83249

    View full-size slide

  18. Develop
    Service B
    Test Deploy Operate
    Service Team B
    Develop
    Service A
    Test Deploy Operate
    Service Team A
    Develop
    Service C
    Test Deploy Operate
    Service Team C

    View full-size slide

  19. Develop
    Service B
    Test Deploy Operate
    Service Team B
    Develop
    Service A
    Test Deploy Operate
    Service Team A
    Develop
    Service C
    Test Deploy Operate
    Service Team C
    Developers need to act
    as SWE, SET and SRE

    View full-size slide

  20. Develop
    Service B
    Test Deploy Operate
    Service Team B
    Develop
    Service A
    Test Deploy Operate
    Service Team A
    Develop
    Service C
    Test Deploy Operate
    Service Team C
    Too much expected

    View full-size slide

  21. 73
    Developing
    19
    Production

    View full-size slide

  22. What We Achieved

    View full-size slide

  23. Pave the “Base” Road
    https://www.flickr.com/photos/martin_borjesson/7725354722

    View full-size slide

  24. https://www.flickr.com/photos/martin_borjesson/7725354722
    Architecture
    DevOps Culture

    View full-size slide

  25. API Gateway
    Communication
    Template
    Access manage

    View full-size slide

  26. MySQL Solr
    Monolith API

    View full-size slide

  27. API gateway
    MySQL Solr
    Kubernetes
    Monolith API

    View full-size slide

  28. API gateway
    Authority
    Service A
    MySQL Solr
    Kubernetes
    Monolith API

    View full-size slide

  29. API gateway
    Authority
    Service A
    MySQL Solr
    Kubernetes
    Monolith API
    Service B

    View full-size slide

  30. API gateway
    Authority
    Service A
    Spanner
    MySQL Solr
    Kubernetes
    Monolith API
    CloudSQL
    Service B

    View full-size slide

  31. API gateway
    Authority
    Service A
    Spanner
    MySQL Solr
    Kubernetes
    Monolith API
    CloudSQL
    Service B

    View full-size slide

  32. API Gateway
    Communication
    Template
    Access manage

    View full-size slide

  33. API gateway
    Authority
    Service A
    CloudSQL
    Spanner
    MySQL Solr
    Kubernetes
    Service B
    Monolith API

    View full-size slide

  34. Service Team PR generate
    platform-proto CircleCI

    View full-size slide

  35. API Gateway
    Communication
    Template
    Access manage

    View full-size slide

  36. Monitoring
    Error reporting
    Health checking
    Dockerfile
    CI setting

    View full-size slide

  37. API Gateway
    Communication
    Template
    Access manage

    View full-size slide

  38. API gateway
    Authority
    Service A
    CloudSQL
    Spanner
    MySQL Solr
    Kubernetes
    Service B
    Monolith API
    SRE team

    View full-size slide

  39. API gateway
    Authority
    Service A
    CloudSQL
    Spanner
    MySQL Solr
    Kubernetes
    Service B
    Monolith API
    Service Team B
    Service Team A

    View full-size slide

  40. GCP Project: Kubernetes
    Platform Team
    Kubernetes
    IAM

    View full-size slide

  41. GCP Project: Kubernetes
    Platform Team
    Kubernetes
    IAM
    Service A Team

    View full-size slide

  42. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    IAM
    RBAC
    Service A Team

    View full-size slide

  43. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    IAM
    RBAC
    Container A
    Container A
    Container A
    Service A Team

    View full-size slide

  44. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    IAM
    RBAC
    Container A
    Container A
    Container A
    GCP Project: Service A
    IAM
    Service A Team

    View full-size slide

  45. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    IAM
    RBAC
    Container A
    Container A
    Container A
    Spanner
    GCP Project: Service A
    IAM
    Service A Team

    View full-size slide

  46. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    IAM
    RBAC
    Container A
    Container A
    Container A
    Spanner
    GCP Project: Service A
    IAM
    Service B Team
    Service A Team

    View full-size slide

  47. GCP Project: Kubernetes
    Platform Team
    Spanner
    Namespace: Service A
    Kubernetes
    GCP Project: Service A
    Namespace: Service B
    GCP Project: Service B
    RBAC
    IAM
    RBAC
    IAM
    IAM
    Container A
    Container A
    Container A
    Service B Team
    Service A Team

    View full-size slide

  48. API Gateway
    Communication
    Template
    Access manage

    View full-size slide

  49. https://www.flickr.com/photos/martin_borjesson/7725354722
    Architecture
    DevOps Culture

    View full-size slide

  50. https://www.flickr.com/photos/marc_sud13/8453080595
    Provisioning
    Deploying

    View full-size slide

  51. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    GCP Project: Service A
    RBAC
    IAM
    IAM
    Service A Team
    Spanner

    View full-size slide

  52. Infrastructure as Code
    https://www.flickr.com/photos/trilanes/4714198782/

    View full-size slide

  53. Starter-kit
    as terraform module

    View full-size slide

  54. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    GCP Project: Service A
    Service A Team
    RBAC
    IAM
    IAM

    View full-size slide

  55. GCP Project: Kubernetes
    Platform Team
    Namespace: Service A
    Kubernetes
    GCP Project: Service A
    Service A Team
    RBAC
    IAM
    IAM
    Spanner

    View full-size slide

  56. PR Plan & Apply
    Platform Team
    Review
    microservices-terraform CircleCI
    Service A Team

    View full-size slide

  57. Service B Team
    Service A Team
    Service C Team
    PR
    PR
    PR
    Plan & Apply
    Platform Team
    Review
    microservices-terraform CircleCI

    View full-size slide

  58. PR Plan & Apply
    PR
    PR
    Platform Team
    Review
    microservices-terraform CircleCI
    Service B Team
    Service A Team
    Service C Team

    View full-size slide

  59. PR Plan & Apply
    PR
    PR
    Platform Team
    Review
    microservices-terraform CircleCI
    Service B Team
    Service A Team
    Service C Team
    CODEOWNERS

    View full-size slide

  60. Plan & Apply
    PR
    Platform Team
    Review
    microservices-terraform CircleCI
    Service B Team
    Service A Team
    Service C Team
    PR & Review
    PR & Review

    View full-size slide

  61. https://www.flickr.com/photos/marc_sud13/8453080595
    Provisioning
    Deploying

    View full-size slide

  62. GCP Project: Kubernetes
    Namespace: Service A
    Kubernetes
    Namespace: Service B
    Container A
    Container A
    Container A
    Container A
    Container A
    Container B
    Service A Team
    Service B Team

    View full-size slide

  63. 60+ pipelines

    View full-size slide

  64. Infrastructure as Code
    https://www.flickr.com/photos/trilanes/4714198782/

    View full-size slide

  65. Kubernetes v2 provider
    https://www.flickr.com/photos/trilanes/4714198782/

    View full-size slide

  66. GCP Project: Kubernetes
    Namespace: Service A
    Kubernetes
    Namespace: Service B
    Container A
    Container A
    Container A
    Container A
    Container A
    Container B
    Service Team A
    Service Team B
    PR
    PR
    Sync
    Platform Team
    Review
    microservices-kubernetes

    View full-size slide

  67. GCP Project: Kubernetes
    Namespace: Service A
    Kubernetes
    Namespace: Service B
    Container A
    Container A
    Container A
    Container A
    Container A
    Container B
    Service Team A
    Service Team B
    PR
    PR
    Sync
    Platform Team
    Review
    microservices-kubernetes
    Developers can learn
    k8s YAML, too

    View full-size slide

  68. What’s Next?

    View full-size slide

  69. 73
    Developing
    19
    Production

    View full-size slide

  70. 73
    Developing
    19
    Production
    Reliable Service
    Operation

    View full-size slide

  71. SLI/SLO
    Driven Operation
    https://www.flickr.com/photos/samarthshyamanur/8347584639

    View full-size slide

  72. Chaos Testing
    https://www.flickr.com/photos/foxgrrl/9671201784/

    View full-size slide

  73. Service Mesh
    https://www.flickr.com/photos/mjsonline/1971492464

    View full-size slide