Slide 1
Slide 1 text
Enable Resiliency and
Efficiency at LINE Invoice
Matt Chang, Steve Yu
2023.06
Slide 2
Slide 2 text
Matt Chang
FullStack Engineer Intern @ LINE Invoice
Slide 3
Slide 3 text
• Introduce LINE Invoice
• Share my intern experience
• Share some tech knowledge
• Mainly focus on Backend.
Introduction
Slide 4
Slide 4 text
LINE 發票管家
Slide 5
Slide 5 text
Backend: CRUD Boy
• CRUD (Create, Read, Update, Delete)
• Example: TODO List
• What's CRUD Boy?
Slide 6
Slide 6 text
Backend: CRUD on Distributed System
• CRUD on Distributed System
• 寫的 code 變成跑在多台 server 上
• 不同 server 間彼此需要緊密溝通
• 需要考慮的事情變多
• Race condition
• Unreliable connection
Slide 7
Slide 7 text
Resilliency and Efficiency
• Resilliency
• Infra:
• E.g. self-healing, rolling upgrade
• Backend:
• E.g. Retry, Timeout and Rate Limit
• Efficiency:
• Infra:
• Vertical Scaling
• Horizontal Scaling
• Backend:
• E.g. Time complexity, concurrent/multithread programming, System Design
Slide 8
Slide 8 text
Resilliency: Unreliable connection
• Be careful with Input & Output.
• Retry:
• What happened when requested service is temporarily unavailable?
• Timeout:
• What happened when requested service is unresponsive?
• Rate limit:
• In-bound: Restrict others from accessing service too frequently.
• Out-bound: Restrict request frequency to other service.
Slide 9
Slide 9 text
Efficiency: Concurrent Programming
Real world example: 7-11
Slide 10
Slide 10 text
Efficiency: System Design
• Usage of Cache and Why ?
• Memory access is faster than disk.
• No need to do complex computation again.
• Real world example: Human Brain
• In-memory cache Problem
• Cache stored in server S1 can't be accessed
by server S2.
• Try In-memory database
• E.g. Redis
Slide 11
Slide 11 text
Short Summary
• CRUD / CRUD in distributed System
• Resiilliency
• Retry / Timeout / Rate Limiting
• Efficiency
• Concurrent programming
• System Design: Cache
Slide 12
Slide 12 text
SteveYi Yu
Site Reliability Engineering Intern @ LINE
12
● Service maintenance (k8s)
● Deployment (containerization & YAML)
● Monitoring (Grafana)
Slide 13
Slide 13 text
簡單來說就是 CI/CD 一條龍
Alert & Monitor
13
CI/CD
Docker-compose / containerization & ArgoCD
● CI/CD
○ 自動化 build 成
Image
○ 上傳到內部的
Docker Registry
● ArgoCD
○ 以自動規模化
(Auto Scale) 的方
式部署
● Alert & Monitor
○ Grafana 上查看視
覺化面板
○ 異常情況發送通
知到 Slack
Slide 14
Slide 14 text
簡單來說就是 CI/CD 一條龍
14
Slide 15
Slide 15 text
Deploy to ArgoCD
15
● Build to Docker Image
● Upload to Harbor
(Internal Image Registry)
● Update k8s Deploy YAML
Slide 16
Slide 16 text
Deploy to ArgoCD
16
Slide 17
Slide 17 text
How LINE Invoice Work?
17
● Kubernetes
○ 根據流量自動擴展 Pod
○ rolling update 會將流量分散
，不影響用戶訪問
Slide 18
Slide 18 text
How LINE Invoice Work?
18
● Treafik
○ 負責處理用戶的請求
○ 將流量分散的分配給不
同的 Pod
Slide 19
Slide 19 text
THANK YOU