Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
추천 API 최적화하기
Search
RIDI
August 31, 2018
Technology
4
2k
추천 API 최적화하기
RIDI
August 31, 2018
Tweet
Share
More Decks by RIDI
See All by RIDI
원격 근무 팀 운영 경험 공유
ridi
0
2.2k
SVG Icon Design Guide
ridi
2
3.7k
빠르게 훑어보는 리디페이 백엔드 개발기
ridi
2
5k
Next.js는 정말 zero config였다.
ridi
0
2.1k
3일 걸릴 것 같던 구매목록 다운로드는 왜 3주가 걸렸을까?
ridi
0
420
원격으로 한 달 일해보니
ridi
0
1.4k
리디북스 앱에 S Pen Remote 연동하기
ridi
2
2.7k
UI 라이브러리 개발기
ridi
1
2.6k
테스트 환경 개선하기
ridi
8
3.4k
Other Decks in Technology
See All in Technology
日経電子版 for Android の技術的課題と取り組み(令和最新版)/android-20250423
nikkei_engineer_recruiting
1
500
3D生成AIのための画像生成
kosukeito
2
330
3月のAWSアップデートを5分間でざっくりと!
kubomasataka
0
130
Databricksで完全履修!オールインワンレイクハウスは実在した!
akuwano
0
110
Winning at PHP in Production in 2025
beberlei
1
200
SREからゼロイチプロダクト開発へ ー越境する打席の立ち方と期待への応え方ー / Product Engineering Night #8
itkq
2
1k
AIにおけるソフトウェアテスト_ver1.00
fumisuke
1
260
白金鉱業Meetup_Vol.18_AIエージェント時代のUI/UX設計
brainpadpr
1
230
彩の国で始めよう。おっさんエンジニアから共有したい、当たり前のことを当たり前にする技術
otsuki
0
160
Road to Go Gem #rubykaigi
sue445
0
990
更新系と状態
uhyo
8
2k
PostgreSQL Log File Mastery: Optimizing Database Performance Through Advanced Log Analysis
shiviyer007
PRO
1
140
Featured
See All Featured
Building Adaptive Systems
keathley
41
2.5k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Making Projects Easy
brettharned
116
6.1k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
2.9k
It's Worth the Effort
3n
184
28k
GraphQLとの向き合い方2022年版
quramy
46
14k
Building Applications with DynamoDB
mza
94
6.3k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
19
1.2k
VelocityConf: Rendering Performance Case Studies
addyosmani
329
24k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
31
1.1k
Bash Introduction
62gerente
611
210k
Transcript
୶ୌ API ୭ച ೞӝ 2018֙ 8ਘ 31ੌ ਬള © RIDI
Corp.
1. ୶ୌ API ࣗѐ • Book-to-book ୶ୌ • Book-to-user ୶ୌ
• API ҳઑ 2. ୶ୌ API ୭ച ೞӝ • উੋ ࢲࡂਸ ਤೠ ࣗ ӝ • ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ • API दр ӝ 2 ݾର ୶ୌ API ୭ച ೞӝ
3 ܻ٣࠘झ Book-to-book ୶ୌ ୶ୌ API ࣗѐ
4 بࢲр ਬࢎب द ୶ୌ API ࣗѐ 0.11651 0.07987
5 بࢲр ਬࢎب ҅ ୶ୌ API ࣗѐ 1. ҳݒݾ۾ •
ଲ: ా۸ Ӗॳӝ • അ୍: ా۸ Ӗॳӝ, যڌѱ Ѫੋо • ࢳ: যڌѱ Ѫੋо 2. بࢲӝળ ҳݒݾ۾ • ా۸ Ӗॳӝ: ଲ, അ୍ • যڌѱ Ѫੋо: അ୍, ࢳ 3. ਬࢎب (ా۸, যڌѱ) = |{അ୍}| / |{ଲ, അ୍, ࢳ}| ≈ 0.33 ?
6 ܻ٣࠘झ Book-to-user ୶ୌ ୶ୌ API ࣗѐ
بࢲр ਬࢎب৬ ਬ ۽ੌਸ ࢎਊ • അח ҳݒ৬ ୭Ӕ ઑഥೠ
بࢲٜ۽ ۽ੌ ࢤࢿ • ۽ੌҗ ਬࢎبо ֫ بࢲٜਸ ୶ୌ 7 ܻ٣࠘झ Book-to-user ਗܻ ୶ୌ API ࣗѐ
recommend-api data-api.ridibooks.com Classic Load Balancer Store Web Server Client Store
Web Elasticache (Redis) proxy-azure.data.ridi.io hbase.data.ridi.io HAProxy 192.168.0.200:33309 HAProxy 192.168.0.200:33319 HAProxy aurora-db.data.ridi.io aurora-db.ro.data.ridi.io Aurora HBase Cluster HTTP Request Thrift RPC MySQL CRUD Cache MySQL Write MySQL Read Plain Requests Load Balancing ridibooks.com HTTP Request 8 API ҳઑ ୶ୌ API ࣗѐ
1. উੋ ࢲࡂਸ ਤೠ ࣗ ӝ 2. ୶ୌ ઁ৻ ؘఠ߬झ
ܻ࠙ೞӝ 3. API दр ӝ 9 ୶ୌ API ୭ച ೞӝ ୶ୌ API ୭ച ೞӝ
উੋ ࢲࡂਸ ਤೠ ࣗ ӝ
11 ୶ୌ ؘఠ উੋ ࢲࡂਸ ਤೠ ࣗ ӝ • Bulk
write৬ real-time read • ড 150݅Ѥ book-to-book ୶ୌҗ ড 90݅Ѥ book-to-user ୶ୌਸ पदрਵ۽ ࢲࡂೞҊ ח ࣗী স۽٘ • Key-value • Key: b_id/u_idx • Value: بࢲ ୶ୌ ܻझ JSON String
After • Scala, Python • Multi-machine batch • ড 2दр
• MongoDBী স۽٘ Before • C++ • Single-machine batch • ড 10दр • MariaDBী স۽٘ 12 ۞झఠ بੑ /റ উੋ ࢲࡂਸ ਤೠ ࣗ ӝ
13 Bulk Loadী ஂডೠ MongoDB উੋ ࢲࡂਸ ਤೠ ࣗ ӝ
• ড 1.6রѐ rowܳ bulk loadೡ ٸ, load average ҃Ҋ ߊࢤ • Bulk loadܳ ೡ ٸ݅ ۽ࣁझ ࣻܳ ৈࢲ प೯ => ৈ ޙઁ ߊࢤ!
14 HBase بੑ উੋ ࢲࡂਸ ਤೠ ࣗ ӝ 1. Bulk
loadী 2. ೞن ীदझమীࢲ ࢎਊ оמ • औѱ بੑೡ ࣻ • ഛࢿ ڪযթ
15 ࠛউೠ HBase উੋ ࢲࡂਸ ਤೠ ࣗ ӝ •
दр оՔঀ ח അ࢚ਸ ߊѼ
16 উੋ ࢲࡂਸ ਤೠ ࣗ ӝ • HBase ࢲ࠺झܳ प೯ೞ
ঋח machineীࢲ݅ ܲ batch স प೯غب۾ ࢸ 1. ܲ Batch স ೱਸ ߉Ҋ ঋਸө? => ৈ ޙઁ ߊࢤ! ޙઁ ױ 1
17 উੋ ࢲࡂਸ ਤೠ ࣗ ӝ • VCNC৬ IBM о٘
١ਸ ଵҊೞৈ ౚ: 2. HBase ౚ ޙઁ ইקө? => ઑӘ աইӟ ೮݅, ৈ ޙઁ ߊࢤ! ޙઁ ױ 2 * http://engineering.vcnc.co.kr/2013/04/hbase-configuration/ * https://www.ibm.com/support/knowledgecenter/en/SSPT3X_4.2.0/ com.ibm.swg.im.infosphere.biginsights.analyze.doc/doc/bigsql_TuneHbase.html ড 20ѐ ۄఠܳ Ѩషೠ റ, 5ѐ ۄఠܳ ౚ. ) ৻ࠗ ਃਸ ܻೞח ؘ ࢎਊೡ Thread ࣻܳ ૐо
18 ޙઁ ױ 3 উੋ ࢲࡂਸ ਤೠ ࣗ ӝ 3.
HappyBase (Python-HBase connector) ޙઁо ইקө? • Locust۽ ۽٘ పझܳ ೮ਸ ٸ, गо হ • HappyBase৬ Django৬ ഐജࢿ ब => ೞ݅, Thrift RPC ࢲߡীࢲ response time ۽Ӓ ߂ ठ۽ ௪ܻ ࠙ࢳ ١ਸ ೡ ࣻ হ.
19 1. ElastiCache + Redis উੋ ࢲࡂਸ ਤೠ ࣗ ӝ
ҙܻഋ ੋ ݫݽܻ ؘఠ झషয ࢲ࠺झ • • Key-value ߑधী • ࡅܲ ࣘب • ױ • ࠺ਊ; DB ਊ݅ఀ ݫݽܻо ਃ • ӝ: 60GB (୭ࣗ cache.m4.4xlarge ਃ) => 1֙ ডী $3100. replica setਸ ҳࢿೞҊ, ਊ טযաݶ…?
20 2. DynamoDB উੋ ࢲࡂਸ ਤೠ ࣗ ӝ ৮߷ೞѱ ҙܻغח
NoSQL ؘఠ߬झ ࢲ࠺झ • • Key-value ߑधী • ױ • ࠺ਊ; ೠ ۨ٘ ӝо ਸ ٸ݅ ਬܻ • ӝ: 60GB (ೠ ۨ٘ ӝ: 100KB) • ੍ӝ/ॳӝ പࣻ: ୡ 40/100ഥ => ೠ ׳ী $5782.80
21 3. Elasticsearch উੋ ࢲࡂਸ ਤೠ ࣗ ӝ ࠙ഋ RESTful
Ѩ࢝ ߂ ࠙ࢳ ূ • • Ѩ࢝ APIীࢲ উਵ۽ ੜ ࢎਊೞҊ • ױ • Bulk loadী ೞ ঋ; load average ޙઁ ߊࢤ • Key-value ߑधী ೞ ঋ
द MongoDB۽… উੋ ࢲࡂਸ ਤೠ ࣗ ӝ • Үള: ܻ
࢚ടী MongoDBо ઁੌ ੜ ݏח Ѫ э. • HBaseࠁ উੋ ࢲࡂ • ࠺Ү ۴ೠ ࠺ਊ (ElastiCache + Redis, DynamoDB, …) • Bulk loadೡ ٸ ޙઁ… • Bulk loadೡ ٸ݃ ࢜ instance۽ Үೞ! 22
1. উੋ ࢲࡂਸ ਤೠ ࣗ ӝ 2. ୶ୌ ઁ৻ ؘఠ߬झ
ܻ࠙ೞӝ 3. API दр ӝ 23 ୶ୌ API ୭ച ೞӝ ୶ୌ API ୭ച ೞӝ
୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ
25 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ AI ୶ୌ ୶ୌ ઁ৻
26 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ DB ҳઑ (୶ୌ ઑഥद)
27 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ DB ҳઑ (୶ୌઁ৻ ١۾द) Single
Point of Failure!!
SPoF ೱ ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ • ؘఠࣃఠ ਗ
গ۽ ੋ೧ ؘఠ ݃झఠ DB • DB ࢲߡ ࠂҳदө ୶ୌઁ৻ ӝמ ࢎਊࠛо • DB ࢲߡ ݫݽܻ ૐࢸਸ ਤ೧ ࢲߡ ࠗ ਃ • ࠗೞח زউ ୶ୌઁ৻ ӝמ ࢎਊࠛо
SPoF ೱ ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ ୶ୌઁ৻ ప࠶݅ ࡅݶ...
ࢿѺ ܲ ప࠶ٜ ҕઓ 30 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ •
୶ୌઁ৻ ప࠶ • 24दр ղղ INSERT ߊࢤ • Ҋё ࢲ࠺झ৬ ೠ ҙ۲ • ా҅ ప࠶ • ࢜߷ ߓ प೯दী݅ INSERT ߊࢤ • ഥࢎ ղ ࢲ࠺झী݅ ҙ۲ => ߹ب ੋۄী فחѱ જѷ!
୶ୌઁ৻ ప࠶ਸ Aurora۽ ܻ࠙ೞӝ۽ Ѿ 31 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ
• ҙܻഋ ࢲ࠺झ ಞܻೣ • ഛࢿ, উࢿ ഛࠁ • 10ms ݅ ܻܻா࣌ ۑ
Aurora ܻ࠙ റ 32 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ • ӝઓ
DB ࢲߡח ઁٚ ࠗ оמ • ୶ୌઁ৻ ӝמী Րӣহ ݫݽܻ ૐࢸী ࢿҕ • DB উࢿ ૐо • ܲ ࢲ࠺झী ೱ߉ ঋ • ݃झఠо ইצ ठۨ࠳ীࢲ ੍ب۾ ߸҃ • যٚ ܻܻா࣌ ۑ ؋࠙ী оמ
թ ࣼઁ 33 ୶ୌ ઁ৻ ؘఠ߬झ ܻ࠙ೞӝ • Auroraীח JOINী
ਃೠ ప࠶ হ • (അ) যܻா࣌ ױ҅ীࢲ JOIN ࣻ೯ • ܻܻா࣌ ࢸ • APIܳ ࢎਊೠ ӝੋ সؘ
1. উੋ ࢲࡂਸ ਤೠ ࣗ ӝ 2. ୶ୌ ઁ৻ ؘఠ߬झ
ܻ࠙ೞӝ 3. API दр ӝ 34 ୶ୌ API ୭ച ೞӝ ୶ୌ API ୭ച ೞӝ
API दр ӝ
36 ޙઁ ੋध API दр ӝ
37 ޙઁ ഛੋ API दр ӝ 2׳ р 3ୡ ࢚
दрਸ о AI ୶ୌ API ਃ = ড 4݅Ѥ ( ਃ 0.1359%)
38 ޙઁ ױ 1 API दр ӝ Django ORM ࢤࢿೠ
ҳݒݾ۾ ઑഥ ௪ܻ: EXPLAIN SELECT `tb_user_book`.`pk_id`, `tb_user_book`.`u_id`,… => tb_user_book ઑഥо बػ.
39 ޙઁ ױ 2 API दр ӝ • Ѩૐ •
ؘఠ MariaDBо ಣࣗ ې (࠙ ୭ 600ߣ ઑഥ) ਸ Ѽ٣ח ഛੋ • Ѿҗ • ҳݒݾ۾ ઑഥীࢲ ठ۽ ௪ܻ ߊࢤ! • ؘఠ DBח நएо ѱ غҊ “ࢲ DBীח ۠ ޙઁо হ؍ Ѫ эؘ…?”
40 API दр ӝ ୶ୌ APIীࢲ ҳݒݾ۾ ઑഥܳ হগ!
ࢎਊ ۽ੌ - ҳݒ - ୭Ӕ ઑഥ 41 API दр
ӝ بࢲр ਬࢎب Batch API AI ୶ୌ ୶ୌ ܻझ ࢤࢿ җ
ࢎਊ ۽ੌ - ҳݒ - ୭Ӕ ઑഥ 42 API दр
ӝ بࢲр ਬࢎب Batch API AI ୶ୌ ୶ୌ ܻझ ࢤࢿ җ
43 ҳݒݾ۾ ӝ߈ ୶ୌ ܻझܳ ܻ ҅ೞ! API दр ӝ
بࢲр ਬࢎب Batch API ҳݒ ۽ੌ ҳݒӝ߈ ୶ୌ ઑഥ ۽ੌ ઑഥӝ߈ ୶ୌ AI ୶ୌ
44 ೧Ѿ଼ Ѩૐ API दр ӝ 3ୡ ࢚ दрਸ о
AI ୶ୌ API ਃ = ѐࢶ 1/8 ( ਃ 0.0176%) * MongoDBܳ ࢎਊೞݶ ࠗ࠙ب ѐࢶؼ Ѫਸ ࢚פ.
45 Ѿۿ • ו՛ • ݣ૽ ੜ ࢲࡂೞח Ѥ
য۰ ੌ • ݽܰҊ ࢎਊೞח Ѫҗ ঌҊ ࢎਊೞח Ѫ ܰ • খਵ۽ ೡ ੌ • ؘఠ ࢎ౭झ ࡳӝ • അ ࢚ടী উೞӝ ঋӝ ୶ୌ API ୭ച ೞӝ
46 THANK YOU хࢎפ