Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Aerospike で 大量のトランザクション処理をさばく
Search
ikosin
October 30, 2015
Programming
8
2.4k
Aerospike で 大量のトランザクション処理をさばく
https://atnd.org/events/71301
での発表資料です。
ikosin
October 30, 2015
Tweet
Share
Other Decks in Programming
See All in Programming
プログラミングどうやる? ~テスト駆動開発から学ぶ達人の型~
a_okui
0
190
Serena MCPのすすめ
wadakatu
4
880
CSC305 Lecture 02
javiergs
PRO
1
260
止められない医療アプリ、そっと Swift 6 へ
medley
1
110
エンジニアとして高みを目指す、 利益を生み出す設計の考え方 / design-for-profit
minodriven
23
12k
高度なUI/UXこそHotwireで作ろう Kaigi on Rails 2025
naofumi
4
3.2k
iOSアプリの信頼性を向上させる取り組み/ios-app-improve-reliability
shino8rayu9
0
140
XP, Testing and ninja testing ZOZ5
m_seki
2
260
プログラマのための作曲入門
cheebow
0
530
CSC509 Lecture 03
javiergs
PRO
0
320
After go func(): Goroutines Through a Beginner’s Eye
97vaibhav
0
230
そのpreloadは必要?見過ごされたpreloadが技術的負債として爆発した日
mugitti9
2
2.9k
Featured
See All Featured
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.5k
The Pragmatic Product Professional
lauravandoore
36
6.9k
Agile that works and the tools we love
rasmusluckow
331
21k
KATA
mclloyd
32
15k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
Faster Mobile Websites
deanohume
310
31k
Designing for Performance
lara
610
69k
Statistics for Hackers
jakevdp
799
220k
Six Lessons from altMBA
skipperchong
28
4k
Transcript
Aerospike Ͱ େྔͷτϥϯβΫγϣϯॲཧΛ ͘͞
ʘ͜Μʹͪʗʘ͜Μʹͪʗ • @62jun26 • Java Ͱ͢ • ࠷ۙΠϯϑϥدΓͷࣄ͕ଟ͍ • ΠϯλʔωοτͱΧʔϓ͕͖Ͱ͢
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
ΞυςΫྖҬʹ͓͚Δ τϥϯβΫγϣϯܥॲཧ • Ϣʔβใর߹ • ଐੑσʔλΛ༻͍ͨλʔήςΟϯάʹ༻͍Δ • ࠂใऔಘ • ΫϦΤΠςΟϒͷࠂใ
• ࠂ৴࣮ੵΈ্͛ • ༧ࢉཧͷͨΊͷ৴࣮
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
τϥϯβΫγϣϯܥॲཧ ʹඞཁͳཁ݅ • ϨΠςϯγ • ߴεϧʔϓοτ • ߴՄ༻ੑ
͋Δͱخ͍͠ • εέʔϥϏϦςΟ • ؆୯ͳूܭΛ࣮ݱ͢Δ UDF ·ͨΈࠐΈؔ • σʔλͷӬଓԽ =>
RDB Ͱͳ͘σʔλΛγϯϓϧʹͯ͠ KVS Λ ༻͢ΔྲྀΕ
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
͜Ε·ͰͷาΈ • Tokyo Tyrant & Tokyo Cabinet • Redis •
DynamoDB
Tokyo Tyrant & Tokyo Cabinet • ࠃ࢈ KVS • 2010
લޙ • memcache ޓϓϩτίϧͰσʔλӬଓԽՄೳ • ϚελɾεϨʔϒܕͷϨϓϦέʔγϣϯػೳ • ϚϧνϚελߏՄೳ • ·ͩ·ͩݱͰಈ͍ͯΔͱ͍͏ฉ͘
Redis • σʔλӬଓԽՄೳ • ༷ʑͳσʔλߏͷ··อଘՄೳʢจࣈྻ, List, Set, ιʔτࡁΈSet, Hashʣ •
ϚελɾεϨʔϒܕͷϨϓϦέʔγϣϯػೳ • Amazon ͷ ElastiCache ͰϑϧϚωʔδυͳαʔϏεͱͯ͠ར༻Մೳ • 20154݄ʹग़ͨ Redis 3.0 Ͱʹ Redis Cluster ͕ਖ਼ࣜϦϦʔεʂ
༨ஊ: Redis • Ϧιʔε͕๛Ͱ֤ΞϓϦέʔγϣϯαʔό ʹಉࠝͰ͖ΔͳΒ Redis ͕ྑ͍ͱࢥ͍·͢ • Write ϚελʢElastiCache
ʣɺRead ΞϓϦέʔγϣϯαʔόಉࠝͷεϨʔϒ͔ Β
DynamoDB • Amazon ͷϑϧϚωʔδυͳNoSQL σʔλϕʔε • ϨΠςϯγ • ߴεϧʔϓοτ •
ߴՄ༻ੑ • ߴ͍εέʔϥϏϦςΟ • ͓ۚΛ͑શͯͷղܾ͞ΕΔʂ
ʢ͓·͚ʣ ͦͷଞީิʹ͕͋ΔϓϩμΫτ • Memcached • MongoDB • Cassandra • HBase
• BigTable
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
Aerospike IUUQXXXBFSPTQJLFDPNBCPVUBFSPTQJLF
Aerospike ͷಛ • ૣ͍ʂʢϨΠςϯγʣ • ͍҆ʂʢߴεϧʔϓοτ => αʔόઅʣ • ࢫ͍ʂʢࣗಈΫϥελߏɾγϟʔσΟϯάʣ
• ҆৺ʂʢσʔλԽʣ ϕλͰ͢Ͷ
ϨΠςϯγɾߴεϧʔϓοτ • ҎલɺAerospike ͷதͷਓͷهࣄΛ༁͠·ͨ͠ • Aerospike on Amazon EC2 Ͱ
100ສTPS Λͨͬͨ 1.68 υϧ /࣌ Ͱ࣮ݱ͢Δํ๏ https://fa-works.com/blog/1-aerospike-server-x-1-amazon- ec2-instance-achieve-1-million-tps • C3.8xlarge 1Ͱ 100ສTPS ͷ Read ϦΫΤετΛॲཧ • 97% Ҏ্ͷϦΫΤετΛ 1ms Ҏʹฦ٫ʢຊՔಇதͷαʔόʣ
ࣗಈΫϥελߏɾγϟʔσΟ ϯά • ࠾༻ͷܾΊखʹͳͬͨ • ৽͍͠αʔόΛ༻ҙͯ͠ɺطଘϊʔυͷ IP Λ ࢦఆͯ͠ىಈ͢ΕউखʹΫϥελϦϯά •
ύʔςΟγϣϯͷ࠶ܭࢉΛߦͬͯϚΠάϨʔ τʢσʔλϦόϥϯγϯάʣͯ͘͠ΕΔ
σʔλԽ • ϨϓϦέʔγϣϯɾϑΝΫλ (replication factor) ͱ͍͏ ύϥϝʔλͰσʔλͷෳͷΛࢦఆͰ͖Δ • replication factor
= 2 ͷ߹ɺϝΠϯͰΞΫηε͢ΔϚ ελσʔλͱผʹɺଞͷαʔόʹ 1ͭϨϓϦΧσʔλ ͕ଘࡏ͢Δ • 1 Ϋϥογϡͯ͠ɺϨϓϦΧͷσʔλ͕ੜ͖͍ͯΔ ͨΊσʔλΛϩετ͠ͳ͍
্࣮Ͱҙ͢Δ • Limit • Setʢςʔϒϧʣͷ => 1024ݸҎ • BinʢΧϥϜʣ໊ =>
14จࣈҎ • Record => ʢσϑΥϧτʣ128KB ʢ࠷େ1MBʣ • http://www.aerospike.com/docs/guide/FAQ.html
্࣮Ͱҙ͢Δ̎ • ClientLibrary ͰͷγϦΞϥΠζ͕݁ߏඍົ • SecondaryIndex ͰͷෳΠϯσοΫεͷ And ݕࡧෆՄ •
SecondaryIndexͰɺIn ͕۟ͳ͍ • Range αϙʔτ͞Ε͍ͯΔ͕ɺจࣈྻΛҾֻ͚ͬ ʹ͍͘
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
མͱ݀̍͠:κϯϏϨίʔυ Delete & TTL ʹΑΔ Expire আͰͳ͍ʂʂ • ϝϞϦʹࡌͤͯ͋ΔΠϯσοΫεʢPKʣΛআ͢Δ͚ͩ ͰɺετϨʔδ͔Βଈ࠲ʹফ͑ΔΘ͚Ͱͳ͍
• SSD ͷಛੑ্ɺσʔλআෛՙ͕ߴ͍ • σϑϥάͷ͍ͭͰʹ্ॻ͖ʢ͍ͭফ͑Δ͔ෆ໌ʣ • ௨ৗΞΫηεग़དྷͳ͍ͷͰͳ͍ͣͩͬͨ...
མͱ݀̍͠:κϯϏϨίʔυ ͨͩ͠ɺAerospike Λ࠶ىಈͨ͠Β෮׆͢Δʂ • Aerospike ىಈ࣌ʹɺετϨʔδ͔Β PK ΛಡΜͰϝϞϦ ʹΠϯσοΫεΛߏங͢Δ •
আ͞ΕͨϨίʔυͱ͞Εͯͳ͍Ϩίʔυͷ۠ผ͕ͳ͍ʂ • আͨ͠ϨίʔυҰॹʹΠϯσοΫε͞ΕͯಡΈऔΓͰ ͖ΔΑ͏ʹͳΔʂʂʁ
མͱ݀̍͠:κϯϏϨίʔυ • Aerospike ࠶ىಈ࣌ʹɺετϨʔδΛۭʹ͢Δ cold-start-empty ΦϓγϣϯΛ༗ޮʹ͠Α͏ • σʔλ͕Խͯ͋͠Εɺଞͷϊʔυ͔Βσʔ λΛΒ͑Δ
མͱ݀̎͠: ར༻Ͱ͖Δσʔλ༰ྔ • લड़ͷσϑϥάͷͨΊɺ༻ҙͨ͠ετϨʔδɾ ϝϞϦΛશ͍͖ͯΕΔΘ͚Ͱͳ͍ • σϑΥϧτͰετϨʔδͷ 50%, ϝϞϦͷ 60%
͠ ͔ར༻Ͱ͖ͳ͍(high water mark) • 90% Λաͨ͠Βॻ͖ࠐΈ͕Ͱ͖ͳ͍(stop write)
མͱ݀̏͠: ϨίʔυαΠζʹؾΛ͚ͭΔ • વͳ͕Βɺ̍Ϩίʔυ͋ͨΓͷαΠζ͕େ ͖͚ΕωοτϫʔΫଳҬΛѹഭ͢Δ • ͋Ε͜Ε̍Ϩίʔυʹ٧ΊࠐΈա͗ͳ͍ • ৗʹσʔλߏΛݟ͢ϚΠϯυ͕ॏཁ
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ ࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
͓·͚ɿEC2 Ͱӡ༻͢Δ߹ͷ ύϑΥʔϚϯεʹ͍ͭͯ ओͳϘτϧωοΫ • ωοτϫʔΫ • Disk IO (SSD)
ωοτϫʔΫ • Amazon ͷΧλϩάεϖοΫ 10G Ҏ֎ Low ͱ͔ High ͱ͔ϘΧͨ͠
• ಉ͡ High ͰΠϯελϯελΠϓʹΑ͔ͬͯͳΓҟͳΔ • iperf Ͱܭଌͨ͠ํ͕͍͍ʢެ։ͯ͘͠Ε͍ͯΔਓ͍Δʣ • http://qiita.com/saitara/items/882a3e70639637e24a4f • ྫʣc4.2xlarge => 1.01 Gbits/sec • ྫʣc4.4xlarge => 4.27 Gbits/sec
Disk IO & εϧʔϓοτ • EBS ͗͢Δ • ඞཁͳ IO
ੑೳΛग़͢ͷ Provisioned IOPS Λར༻͢Δ ͜ͱͰ࣮ݱͰ͖Δ͕ɺߴ͍ʂ • ΠϯελϯεετϨʔδ Λ͏ • ཧతʹܨ͕͍ͬͯΔετϨʔδͳͷͰૣ͍ʂ • ͨͩ͠ɺΠϯελϯεͷ stop/start Ͱσʔλشൃ͢Δ