Enable Resiliency and Efficiency at LINE Invoice

活動：LINE Developers Meetup 開發者小聚 #18 - TECH FRESH 畢業展
活動頁面：https://linegroup.kktix.cc/events/20230627-tech-fresh-gratudation
講者：Matt Chang, Steve Yu

Transcript

1. Enable Resiliency and Efficiency at LINE Invoice Matt Chang, Steve Yu 2023.06
2. Matt Chang FullStack Engineer Intern @ LINE Invoice
3. • Introduce LINE Invoice • Share my intern experience • Share some tech knowledge • Mainly focus on Backend. Introduction
4. LINE 發票管家
5. Backend: CRUD Boy • CRUD (Create, Read, Update, Delete) • Example: TODO List • What's CRUD Boy?
6. Backend: CRUD on Distributed System • CRUD on Distributed System • 寫的 code 變成跑在多台 server 上 • 不同 server 間彼此需要緊密溝通 • 需要考慮的事情變多 • Race condition • Unreliable connection
7. Resilliency and Efficiency • Resilliency • Infra: • E.g. self-healing, rolling upgrade • Backend: • E.g. Retry, Timeout and Rate Limit • Efficiency: • Infra: • Vertical Scaling • Horizontal Scaling • Backend: • E.g. Time complexity, concurrent/multithread programming, System Design
8. Resilliency: Unreliable connection • Be careful with Input & Output. • Retry: • What happened when requested service is temporarily unavailable? • Timeout: • What happened when requested service is unresponsive? • Rate limit: • In-bound: Restrict others from accessing service too frequently. • Out-bound: Restrict request frequency to other service.
9. Efficiency: Concurrent Programming Real world example: 7-11
10. Efficiency: System Design • Usage of Cache and Why ? • Memory access is faster than disk. • No need to do complex computation again. • Real world example: Human Brain • In-memory cache Problem • Cache stored in server S1 can't be accessed by server S2. • Try In-memory database • E.g. Redis
11. Short Summary • CRUD / CRUD in distributed System • Resiilliency • Retry / Timeout / Rate Limiting • Efficiency • Concurrent programming • System Design: Cache
12. SteveYi Yu Site Reliability Engineering Intern @ LINE 12 • Service maintenance (k8s) • Deployment (containerization & YAML) • Monitoring (Grafana)
13. 簡單來說就是 CI/CD 一條龍 Alert & Monitor 13 CI/CD Docker-compose / containerization & ArgoCD • CI/CD ◦ 自動化 build 成 Image ◦ 上傳到內部的 Docker Registry • ArgoCD ◦ 以自動規模化 (Auto Scale) 的方 式部署 • Alert & Monitor ◦ Grafana 上查看視 覺化面板 ◦ 異常情況發送通 知到 Slack
14. 簡單來說就是 CI/CD 一條龍 14
15. Deploy to ArgoCD 15 • Build to Docker Image • Upload to Harbor (Internal Image Registry) • Update k8s Deploy YAML
16. Deploy to ArgoCD 16
17. How LINE Invoice Work? 17 • Kubernetes ◦ 根據流量自動擴展 Pod ◦ rolling update 會將流量分散 ，不影響用戶訪問
18. How LINE Invoice Work? 18 • Treafik ◦ 負責處理用戶的請求 ◦ 將流量分散的分配給不 同的 Pod
19. THANK YOU