Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Aerospike で 大量のトランザクション処理をさばく
Search
ikosin
October 30, 2015
Programming
8
2.3k
Aerospike で 大量のトランザクション処理をさばく
https://atnd.org/events/71301
での発表資料です。
ikosin
October 30, 2015
Tweet
Share
Other Decks in Programming
See All in Programming
Honoとフロントエンドの 型安全性について
yodaka
7
1.3k
Multi Step Form, Decentralized Autonomous Organization
pumpkiinbell
1
750
クリーンアーキテクチャから見る依存の向きの大切さ
shimabox
2
470
『GO』アプリ データ基盤のログ収集システムコスト削減
mot_techtalk
0
130
コミュニティ駆動 AWS CDK ライブラリ「Open Constructs Library」 / community-cdk-library
gotok365
2
140
なぜイベント駆動が必要なのか - CQRS/ESで解く複雑系システムの課題 -
j5ik2o
12
4.1k
Serverless Rust: Your Low-Risk Entry Point to Rust in Production (and the benefits are huge)
lmammino
1
110
『品質』という言葉が嫌いな理由
korimu
0
160
Amazon S3 TablesとAmazon S3 Metadataを触ってみた / 20250201-jawsug-tochigi-s3tables-s3metadata
kasacchiful
0
170
Software Architecture
hschwentner
6
2.1k
楽しく向き合う例外対応
okutsu
0
150
さいきょうのレイヤードアーキテクチャについて考えてみた
yahiru
3
760
Featured
See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
100
18k
The Language of Interfaces
destraynor
156
24k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
7.1k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
7
630
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
27
1.9k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
A Philosophy of Restraint
colly
203
16k
Producing Creativity
orderedlist
PRO
344
39k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Transcript
Aerospike Ͱ େྔͷτϥϯβΫγϣϯॲཧΛ ͘͞
ʘ͜Μʹͪʗʘ͜Μʹͪʗ • @62jun26 • Java Ͱ͢ • ࠷ۙΠϯϑϥدΓͷࣄ͕ଟ͍ • ΠϯλʔωοτͱΧʔϓ͕͖Ͱ͢
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
ΞυςΫྖҬʹ͓͚Δ τϥϯβΫγϣϯܥॲཧ • Ϣʔβใর߹ • ଐੑσʔλΛ༻͍ͨλʔήςΟϯάʹ༻͍Δ • ࠂใऔಘ • ΫϦΤΠςΟϒͷࠂใ
• ࠂ৴࣮ੵΈ্͛ • ༧ࢉཧͷͨΊͷ৴࣮
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
τϥϯβΫγϣϯܥॲཧ ʹඞཁͳཁ݅ • ϨΠςϯγ • ߴεϧʔϓοτ • ߴՄ༻ੑ
͋Δͱخ͍͠ • εέʔϥϏϦςΟ • ؆୯ͳूܭΛ࣮ݱ͢Δ UDF ·ͨΈࠐΈؔ • σʔλͷӬଓԽ =>
RDB Ͱͳ͘σʔλΛγϯϓϧʹͯ͠ KVS Λ ༻͢ΔྲྀΕ
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
͜Ε·ͰͷาΈ • Tokyo Tyrant & Tokyo Cabinet • Redis •
DynamoDB
Tokyo Tyrant & Tokyo Cabinet • ࠃ࢈ KVS • 2010
લޙ • memcache ޓϓϩτίϧͰσʔλӬଓԽՄೳ • ϚελɾεϨʔϒܕͷϨϓϦέʔγϣϯػೳ • ϚϧνϚελߏՄೳ • ·ͩ·ͩݱͰಈ͍ͯΔͱ͍͏ฉ͘
Redis • σʔλӬଓԽՄೳ • ༷ʑͳσʔλߏͷ··อଘՄೳʢจࣈྻ, List, Set, ιʔτࡁΈSet, Hashʣ •
ϚελɾεϨʔϒܕͷϨϓϦέʔγϣϯػೳ • Amazon ͷ ElastiCache ͰϑϧϚωʔδυͳαʔϏεͱͯ͠ར༻Մೳ • 20154݄ʹग़ͨ Redis 3.0 Ͱʹ Redis Cluster ͕ਖ਼ࣜϦϦʔεʂ
༨ஊ: Redis • Ϧιʔε͕๛Ͱ֤ΞϓϦέʔγϣϯαʔό ʹಉࠝͰ͖ΔͳΒ Redis ͕ྑ͍ͱࢥ͍·͢ • Write ϚελʢElastiCache
ʣɺRead ΞϓϦέʔγϣϯαʔόಉࠝͷεϨʔϒ͔ Β
DynamoDB • Amazon ͷϑϧϚωʔδυͳNoSQL σʔλϕʔε • ϨΠςϯγ • ߴεϧʔϓοτ •
ߴՄ༻ੑ • ߴ͍εέʔϥϏϦςΟ • ͓ۚΛ͑શͯͷղܾ͞ΕΔʂ
ʢ͓·͚ʣ ͦͷଞީิʹ͕͋ΔϓϩμΫτ • Memcached • MongoDB • Cassandra • HBase
• BigTable
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
Aerospike IUUQXXXBFSPTQJLFDPNBCPVUBFSPTQJLF
Aerospike ͷಛ • ૣ͍ʂʢϨΠςϯγʣ • ͍҆ʂʢߴεϧʔϓοτ => αʔόઅʣ • ࢫ͍ʂʢࣗಈΫϥελߏɾγϟʔσΟϯάʣ
• ҆৺ʂʢσʔλԽʣ ϕλͰ͢Ͷ
ϨΠςϯγɾߴεϧʔϓοτ • ҎલɺAerospike ͷதͷਓͷهࣄΛ༁͠·ͨ͠ • Aerospike on Amazon EC2 Ͱ
100ສTPS Λͨͬͨ 1.68 υϧ /࣌ Ͱ࣮ݱ͢Δํ๏ https://fa-works.com/blog/1-aerospike-server-x-1-amazon- ec2-instance-achieve-1-million-tps • C3.8xlarge 1Ͱ 100ສTPS ͷ Read ϦΫΤετΛॲཧ • 97% Ҏ্ͷϦΫΤετΛ 1ms Ҏʹฦ٫ʢຊՔಇதͷαʔόʣ
ࣗಈΫϥελߏɾγϟʔσΟ ϯά • ࠾༻ͷܾΊखʹͳͬͨ • ৽͍͠αʔόΛ༻ҙͯ͠ɺطଘϊʔυͷ IP Λ ࢦఆͯ͠ىಈ͢ΕউखʹΫϥελϦϯά •
ύʔςΟγϣϯͷ࠶ܭࢉΛߦͬͯϚΠάϨʔ τʢσʔλϦόϥϯγϯάʣͯ͘͠ΕΔ
σʔλԽ • ϨϓϦέʔγϣϯɾϑΝΫλ (replication factor) ͱ͍͏ ύϥϝʔλͰσʔλͷෳͷΛࢦఆͰ͖Δ • replication factor
= 2 ͷ߹ɺϝΠϯͰΞΫηε͢ΔϚ ελσʔλͱผʹɺଞͷαʔόʹ 1ͭϨϓϦΧσʔλ ͕ଘࡏ͢Δ • 1 Ϋϥογϡͯ͠ɺϨϓϦΧͷσʔλ͕ੜ͖͍ͯΔ ͨΊσʔλΛϩετ͠ͳ͍
্࣮Ͱҙ͢Δ • Limit • Setʢςʔϒϧʣͷ => 1024ݸҎ • BinʢΧϥϜʣ໊ =>
14จࣈҎ • Record => ʢσϑΥϧτʣ128KB ʢ࠷େ1MBʣ • http://www.aerospike.com/docs/guide/FAQ.html
্࣮Ͱҙ͢Δ̎ • ClientLibrary ͰͷγϦΞϥΠζ͕݁ߏඍົ • SecondaryIndex ͰͷෳΠϯσοΫεͷ And ݕࡧෆՄ •
SecondaryIndexͰɺIn ͕۟ͳ͍ • Range αϙʔτ͞Ε͍ͯΔ͕ɺจࣈྻΛҾֻ͚ͬ ʹ͍͘
࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
མͱ݀̍͠:κϯϏϨίʔυ Delete & TTL ʹΑΔ Expire আͰͳ͍ʂʂ • ϝϞϦʹࡌͤͯ͋ΔΠϯσοΫεʢPKʣΛআ͢Δ͚ͩ ͰɺετϨʔδ͔Βଈ࠲ʹফ͑ΔΘ͚Ͱͳ͍
• SSD ͷಛੑ্ɺσʔλআෛՙ͕ߴ͍ • σϑϥάͷ͍ͭͰʹ্ॻ͖ʢ͍ͭফ͑Δ͔ෆ໌ʣ • ௨ৗΞΫηεग़དྷͳ͍ͷͰͳ͍ͣͩͬͨ...
མͱ݀̍͠:κϯϏϨίʔυ ͨͩ͠ɺAerospike Λ࠶ىಈͨ͠Β෮׆͢Δʂ • Aerospike ىಈ࣌ʹɺετϨʔδ͔Β PK ΛಡΜͰϝϞϦ ʹΠϯσοΫεΛߏங͢Δ •
আ͞ΕͨϨίʔυͱ͞Εͯͳ͍Ϩίʔυͷ۠ผ͕ͳ͍ʂ • আͨ͠ϨίʔυҰॹʹΠϯσοΫε͞ΕͯಡΈऔΓͰ ͖ΔΑ͏ʹͳΔʂʂʁ
མͱ݀̍͠:κϯϏϨίʔυ • Aerospike ࠶ىಈ࣌ʹɺετϨʔδΛۭʹ͢Δ cold-start-empty ΦϓγϣϯΛ༗ޮʹ͠Α͏ • σʔλ͕Խͯ͋͠Εɺଞͷϊʔυ͔Βσʔ λΛΒ͑Δ
མͱ݀̎͠: ར༻Ͱ͖Δσʔλ༰ྔ • લड़ͷσϑϥάͷͨΊɺ༻ҙͨ͠ετϨʔδɾ ϝϞϦΛશ͍͖ͯΕΔΘ͚Ͱͳ͍ • σϑΥϧτͰετϨʔδͷ 50%, ϝϞϦͷ 60%
͠ ͔ར༻Ͱ͖ͳ͍(high water mark) • 90% Λաͨ͠Βॻ͖ࠐΈ͕Ͱ͖ͳ͍(stop write)
མͱ݀̏͠: ϨίʔυαΠζʹؾΛ͚ͭΔ • વͳ͕Βɺ̍Ϩίʔυ͋ͨΓͷαΠζ͕େ ͖͚ΕωοτϫʔΫଳҬΛѹഭ͢Δ • ͋Ε͜Ε̍Ϩίʔυʹ٧ΊࠐΈա͗ͳ͍ • ৗʹσʔλߏΛݟ͢ϚΠϯυ͕ॏཁ
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ ࣍ •ΞυςΫྖҬʹ͓͚ΔτϥϯβΫγϣϯܥॲཧ •τϥϯβΫγϣϯܥॲཧʹඞཁͳཁ݅ •͜Ε·ͰͷาΈ •Aerospike ͷಛ •ӡ༻ͯ͠Έͯ……
͓·͚ɿEC2 Ͱӡ༻͢Δ߹ͷ ύϑΥʔϚϯεʹ͍ͭͯ ओͳϘτϧωοΫ • ωοτϫʔΫ • Disk IO (SSD)
ωοτϫʔΫ • Amazon ͷΧλϩάεϖοΫ 10G Ҏ֎ Low ͱ͔ High ͱ͔ϘΧͨ͠
• ಉ͡ High ͰΠϯελϯελΠϓʹΑ͔ͬͯͳΓҟͳΔ • iperf Ͱܭଌͨ͠ํ͕͍͍ʢެ։ͯ͘͠Ε͍ͯΔਓ͍Δʣ • http://qiita.com/saitara/items/882a3e70639637e24a4f • ྫʣc4.2xlarge => 1.01 Gbits/sec • ྫʣc4.4xlarge => 4.27 Gbits/sec
Disk IO & εϧʔϓοτ • EBS ͗͢Δ • ඞཁͳ IO
ੑೳΛग़͢ͷ Provisioned IOPS Λར༻͢Δ ͜ͱͰ࣮ݱͰ͖Δ͕ɺߴ͍ʂ • ΠϯελϯεετϨʔδ Λ͏ • ཧతʹܨ͕͍ͬͯΔετϨʔδͳͷͰૣ͍ʂ • ͨͩ͠ɺΠϯελϯεͷ stop/start Ͱσʔλشൃ͢Δ