In this course, we will explain a basic overview of Microservices and various tools combined with GitOps used inside Mercari which powers the developer platform.
Goals of this course ● Learn about Mercari microservice platform ● Learn about development tools at Mercari that support microservices. You can deploy your microservices from day 1 at Mercari
Non-Goals of this course ● We won’t cover what are microservices in details just the basics. ● We don’t cover how Mercari listing works ● We don’t cover how Mercari transaction works ● … We don’t cover domain-specific knowledges here
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities.
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities.
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities. Availability If one part is dead, everything is dead...
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities. If one part is dead, everything is dead... Availability
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities. If one part is dead, everything is dead... Availability
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities. If one part is dead, everything is dead... Availability
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities. If one part is dead, everything is dead... Complexity Difficult to add new features Difficult to try new technologies Difficult to onboard Availability
Listing Shipping Notification Review Purchase Login Timeline Search Search MySQL Mercari Monolith Solr Scalability Need to scale everything for one part of functionalities. If one part is dead, everything is dead... Complexity Difficult to add new features Difficult to try new technologies Difficult to onboard Velocity Too many overheads, no isolation in criticality Availability
Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work
Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work
Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work
Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work
Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work Flexible scaling Each service can scale independently and on demand
Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work Flexible scaling Each service can scale independently and on demand
Item Item Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work Flexible scaling Each service can scale independently and on demand
Item Item Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work Flexible scaling Each service can scale independently and on demand Simplicity Easy to add new features Easy to use well suited technologies (e.g., DB)
Item Item Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work Flexible scaling Each service can scale independently and on demand Simplicity Easy to add new features Easy to use well suited technologies (e.g., DB) Agility Independent deployment Independent decisions in a small context
Item Item Listing DB Listing team User DB User team Item DB Item team Shipping DB Shipping team Timeline DB Timeline team Resilient Even if one service is dead, others can continue to work Flexible scaling Each service can scale independently and on demand Simplicity Easy to add new features Easy to use well suited technologies (e.g., DB) Agility Independent deployment Independent decisions in a small context
More about microservices ● Microservices by Martin Fowler ● Building Microservices by Sam Newman (Book) ● Production-Ready Microservices by Susan Fowler (Book) ● Adopting Microservices at Netflix ● Microservices at Spotify
Roles of microservices developers ● Develop software for business benefits ● Write unit tests and integration tests ● Response to alerts and incidents ● Automate operations and reduce toils ● Improve your service reliability based on SLI/SLO ● You can and should go bold You need to act as SWE, SET, and SRE
Develop Service B Test Deploy Operate Service B Team Develop Service A Test Deploy Operate Service A Team Develop Service C Test Deploy Operate Service C Team
Technical stack at Mercari ● Go for application language ● gRPC for communication protocol ● Kubernetes for container orchestration ● Terraform for cloud service provisioning ● Spinnaker for continuous delivery ● Argo CD and Tekton for continuous delivery ● CUE for configuration language ● Github Actions for CI ● AWS and GCP for Cloud
Google Cloud Platform Services ● Provides Cloud Spanner and Cloud SQL for databases ● Provides Cloud Pub/Sub for message queuing ● Provides Google Cloud Storage (GCS) for blob stores ● Provides BigQuery for analytics data warehouses ● Provides … (*) * You can use any GCP service for your microservice component
Go at Mercari ● Why: ○ Safety by statically typed & compiled ○ Maintainability (+readability) by simplicity & explicitness ○ High performance and minimum memory footprints
More about Go ● Effective Go ● Go Code Review Comments ● Advanced Testing with Go ● High Performance Go Workshop ● So you want to expose Go on the Internet ● Go Walkthrough ● The Go Programming language (Book)
Language Choices ● Go is currently a main language for Mercari. But you can choose any programming language for your microservices under your responsibility ● You must know: ○ Lots of ecosystems and tools are designed for Go (at the moment) ○ It may be difficult to get supports from others if you get stuck ● Responsibilities: ○ Write everything by yourself ○ Hire engineers for your team
gRPC at Mercari ● Why: ○ Simple service definitions by an Interface Definition Language (IDL) ○ Works across languages and platforms ○ Faster serialization
Kubernetes at Mercari ● Kubernetes is: ○ Container orchestration system ○ Derived from Google internal system named Borg & Omega ● Why (*): ○ Extensibility (answer for why not PaaS) ○ Ecosystem (answer for why not ECS, Nomad, Mesos) * See more details about this in Microservices platform on Kubernetes at Mercari
Terraform at Mercari ● We use Terraform to set up cloud services: ○ Set up GCP services ○ Set up Pagerduty, Sentry, GitHub, Slack, and more ○ Don’t control your infrastructure by hand (UI, CLI, or API) ● We expect developers to write terraform ○ Gives them full control over their resources.
Spinnaker at Mercari ● We use Spinnaker for continuous delivery of microservices ● Developers have control over their CD strategy ○ They control the pipeline via GitOps ○ Allows rollback
Argo CD/Tekton at Mercari ● We’re in the middle of migrating from Spinnaker to a combination of Argo CD and other components ○ Currently in Beta ● Abstractions are provided through our GitOps repos
GCP Project: Kubernetes Platform Team Spanner Namespace: Service A Kubernetes GCP Project: Service A Namespace: Service B GCP Project: Service B RBAC IAM RBAC IAM IAM Container A Container A Container A Service B Team Service A Team
Datadog ● We use Datadog for monitoring microservices: ○ Send application metrics ○ Send distributed traces ○ Create dashboards for microservices ○ Integrate with Slack and Pagerduty for alerting
Starter Kit ● Terraform module to set up tenant resources for services ○ GCP project & IAM ○ Kubernetes namespace ○ GitHub team ○ Pagerduty team and schedules ○ Sentry project ○ …
Team Kit ● Terraform module to set up resources for teams ○ GCP IAM ○ Kubernetes RBAC ○ GitHub team ○ Pagerduty team and schedules ○ Slack channel and group ○ …
Kubernetes Kit ● CUE-based Kubernetes manifest abstraction ● Recommended over YAML-based configurations ● Allows backend engineers to focus on their business ○ Reduces the amount of configuration and required knowledge on Kubernetes and associated technologies ○ Provides sensible defaults and constraints
Key Takeaways ● Mercari Microservices platform has many tools and services ● We enable developers to take ownership and control ● We use GitOps to enforce policies and ease development https://engineering.mercari.com/
mercari
oekazuma
nttcom
wadayusuke
tamaiyutaro
daina0321
jordihofc
eliasnogueira
yururoi
odasho
muno92
soachr
koic
chriscoyier
bluesmoon
ufuk
bryan
mthomps
vanstee
zakiyama
lauravandoore
nonsquared
quramy
frogandcode
michaelherold