How to manage increasingly complex microservices

After several years of running microservices, I share what we have done and are doing as a team on how to manage when the entire microservice becomes large and complex.

Below is the presentation material from the event.
https://findy.connpass.com/event/283285/

Transcript

1. How to manage increasingly complex
   microservices
   Kazushi Hirai
   

   View Slide

2. Self-introduction
   l LINE Fukuoka 株式会社
   l Joined the company in 2016
   l 開発1室 Dev K team Manager
   l Server-side Engineer
   l 2 kids
   l Interests/Hobbies
   l Exercising / Marathon
   l Twitter: @hkazushi0627
   ฏҪ Ұ࢙
   ,";64)*)*3"*
   

   View Slide

3. LINE STORE
   Products
   LINE STORE Stickershop Themeshop Auto Suggest Recommendation OA
   

   View Slide

4. History
   2012
   Stickershop
   Release
   2013
   LINE STORE Release
   2014
   LINE Theme
   LINE Creators sticker
   2017
   New year campaign Start
   Recommendation OA messaging
   from LINE OA
   2018
   LINE Emoji
   2019
   LINE Sticker Premium
   2020
   Message sticker
   LINE Sticker Premium
   Deluxe Course
   2021
   LINE Sticker Premium
   Taiwan、Indonesia
   LINE Animation Emoji
   2011
   LINE Release
   2022
   LINE Sticker Premium
   LINE Music
   LINEMO Bundle
   l Server-side engineers︓Less than 10 ?
   l Numbers of engineer team︓2 (Product team)
   l Number of microservice components︓3 ?
   Microservices and team structure at the beginning of the service launch
   

   View Slide

5. History
   2023 Current microservices and team structure
   l Server-side engineers︓30 or more
   l Numbers of engineer team︓6 or more (Product Team、Feature Team、SRE)
   l Number of microservice components︓80 or more
   l Product and feature expansion and architecture improvement
   l Managed by Monorepo
   l Manage a multiple component code base in a single repository
   2023 Now
   2012
   Stickershop
   Release
   2013
   LINE STORE Release
   2014
   LINE Theme
   LINE Creators sticker
   2017
   New year campaign Start
   Recommendation OA messaging
   from LINE OA
   2018
   LINE Emoji
   2019
   LINE Sticker Premium
   2020
   Message sticker
   LINE Sticker Premium
   Deluxe Course
   2021
   LINE Sticker Premium
   Taiwan、Indonesia
   LINE Animation Emoji
   2011
   LINE Release
   2022
   LINE Sticker Premium
   LINE Music
   LINEMO Bundle
   

   View Slide

6. l Difficult to study microservices and code base
   l Too many components on the repository
   l Unclear which product or function each component is responsible for
   l Unclear who is in charge of each component
   l Unclear who is the right person to ask questions about a component or make a change and ask for source review
   Problems
   New members of the team
   Existing members of the team
   l Similar problems with new members because of the limited number of components for which they were responsible
   l In fact, no one knows the whole picture of microservices
   How do we solve these problems and manage the growing complexity of microservices?
   

   View Slide

7. l Logical grouping of components
   l Group = Component domain
   l Dealing with high-level picture is less complicated and lower the cost of studying to understand the whole picture
   Solution: Component Domain
   Introducing Component Domains
   High-Level architecture Image
   Store Domain: Component Image
   

   View Slide

8. l With GitHub, we can set the code owner
   l Hierarchy by domain makes it easy for code owners to set up
   l Code owner is automatically set to Reviewers when a pull request is created
   Component Domain – Code Base
   store/
   + store-server
   + store-cms
   + store-decaton
   shop/
   …
   subscription/
   …
   Repository Configuration
   /store/ @kazushi-hirai @xxx @yyy
   /shop/ @aaa @bbb
   /subscription/ @ccc @ddd
   …
   .github/CODEOWNERS GitHub: Pull Request
   How do we represent the domain of the codebase ?
   Question
   The top level of the repository is a domain unit, and the components included in the domain are placed under it
   How would we describe who is in charge of each domain and component ?
   Question
   

   View Slide

9. l By product, function or responsibility of component
   l Storage and cache layer should be closed per domain
   l Storage and cache failure affect only components in the domain
   l The development of a domain can be assigned in its entirely to a product/feature team
   l We can focus on the domain we are in charge of, but both internal and external dependencies
   are clearly established so component ownership is represented effectively.
   l This enables smooth communication during both daily work and also urgent situation
   Component Domain – How to define
   Responsible for the web application part of LINE STORE
   Access to storage and cache is only available for components in the store domain
   All other components are accessed via public APIs
   LINE STORE Dev is in charge of development
   In what units are the components grouped by domain?
   Question
   

   View Slide

10. l Refactoring the architecture
    l Ensure that the data is accessed from another domain via public API
    l Split database per domain
    Component Domain - Refactoring
    What if multiple domains are accessing the same storage?
    Question
    Access data via public API
    Split database per domain
    Components in xxx domain access storage in yyy domain
    

    View Slide

11. 発表の流れ
    l Introduce component domain to group microservices
    l Represent the domain on the code base, clarify who is in charge
    l Domains are divided by product, function or responsibility
    l Data is closed the domain to improve fault tolerance and development efficiency
    l In some cases, architectural refactoring is necessary to achieve the domain
    configuration
    Summary
    

    View Slide

12. THANK YOU!
    

    View Slide