In addition to setting your status as Safe or Affected, you can also enter additional information from either message template or typing yourself. Input your status
Safety Check CMS Architecture Safety Check CMS Server Safety Check Server Web CMS Safety Check DB Notification Service User REST Request getDisasterCases Server cache Operator A P I G a t e w a y Read & Write Read Local DB
Safety Check CMS Architecture Safety Check CMS Server Safety Check Server Web CMS Safety Check DB Notification Service User REST Request getDisasterCases Server cache Operator A P I G a t e w a y Read & Write Read Local DB CMS
CMS • Content Management System • Manage service meta data easily • Front-end: Web HTTP service with authentication • Back-end: REST API implemented by Armeria
Safety Check CMS Architecture Safety Check CMS Server Safety Check Server Web CMS Safety Check DB Notification Service User REST Request getDisasterCases Server cache Operator A P I G a t e w a y Read & Write Read Local DB
Notification Service • A light weighted event delivery system for LINE • A signal to send to the client side when changing contents in the CMS • iOS/Android side request the server content after receiving the signal • Delay time and client filter are provided Notification Service Safety Check Server 1. signal 2. request 3. response
Safety Check CMS Architecture Safety Check CMS Server Safety Check Server Web CMS Safety Check DB Notification Service User REST Request getDisasterCases Server cache Operator A P I G a t e w a y Read & Write Read Local DB
Safety Check CMS Architecture Safety Check CMS Server Safety Check Server Web CMS Safety Check DB Notification Service User REST Request getDisasterCases Server cache Operator A P I G a t e w a y Read & Write Read Local DB Server API
Server API • getDisasterCases is defined in THRIFT • Server gets user country and language via API request struct GetDisasterCasesResponse { 1: list disasters, 2: list messageTemplates, /** * Indicates the TTL (time to live) in milliseconds for the the response so that the clients can cache it and request for updated information when it becomes stale. */ 3: i64 ttl, }
Safety Check CMS Architecture Safety Check CMS Server Safety Check Server Web CMS Safety Check DB Notification Service User REST Request getDisasterCases Server cache Operator A P I G a t e w a y Read & Write Read Local DB
Server-side Cache • Server local memory cache is used to reduce DB request • Disaster data do not change frequently, and they are the same for all users in the country and language • Cache is updated asynchronously every fixed time Safety Check Server 1. request 2. response Server cache Safety Check DB Async update
Client-side Cache • iOS/Android keep their local DB for disaster cases in the previous request • Banner will keep showing for some users when notification is delayed or lost • Strategies of disaster API • Client will request API (getDisasterCases) when user access end page of safety check • Client will request API when TTL expires
Decaton • Stream task processing framework built on top of Kafka developed by LINE. • Design goals included enabling concurrent processing of records from a single partition. • https://github.com/line/decaton
Safety Check Update Architecture Safety Check Server A P I G a t e w a y Safety Check Decaton Record Format: {“sourceUser”, “targetUser”, ”disasterId”,”status”,”message”} Local Db updateSafetySatus Thrift User A User C User B friends Notification Service produce consume produce consume
Benefits of Mocking HTTP Services • Guarantee the service latency • Avoid Rate Limits • Avoid Modifying State • Test Edge Cases • Test Error Scenarios • Results are deterministic • Useful for integration tests and load tests
Spring Cloud Contract • Spring Cloud Contract is a tool that enables Consumer Driven Contract (CDC) development. • It consists of: • Spring Cloud Contract Verifier Plugin (Gradle and Maven) • Spring Cloud Contract Stub Runner (Mock Server)
Load Testing with Ayaperf • A LINE developed tool for distributed load testing based on Locust. • Allows you define your tests using Java. • Command tool to set up the secondary nodes (workers) that will generate the load for the server under test using Kubernetes. • Metrics in Grafana.
Load Test Architecture Locust Master Worker 1 Java Client Worker 2 Java Client Worker 3 Java Client Kubernetes Cluster Kubernetes Cluster Safety Check Server Deployment Pod 1 Pod 2 Pod N L o a d B a l a n c e r Ayaperf Spring Cloud Contract (Wiremock) Deployment Pod 1 Pod 2 Pod N Service
Load Test both APIs Given that a single server is able to handle about 1,600 RPS with about 30 % CPU then 14 servers were enough to support our initial estimation of 22,000 RPS.
Summary • Robust CMS allows easy configuration and fast distribution. • Cache strategies increase traffic tolerance and keep client up-to-date. • Event-Driven Architecture decouples your microservices. • Decaton allows to achive higher throughput with small number of partitions. • Mocking Server helps you control the test scenarios and the service latency. • Load testing allows to measure application throughput and resource utilization.
techverse_2022
1000ch
rigolon
.png){kind=icon}
3playmarketing
moriyuya
kochizufan
bengo4com
line_developers
masakamayama
miu_crescent
1ftseabass
mongolyy
miyake
dougneiner
bkeepers
quramy
phodgson
wjessup
frogandcode
sstephenson
panda_program
holman
akmur
chriscoyier
zakiwarfel