Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
TimeFuzeアーキテクチャ構想 / The concept of TimeFuze Arc...
Search
Yuuki Tsubouchi (yuuk1)
December 23, 2017
Research
1
9.6k
TimeFuzeアーキテクチャ構想 / The concept of TimeFuze Architecture
第1回ウェブシステムアーキテクチャ(WSA)研究会@京都
Yuuki Tsubouchi (yuuk1)
December 23, 2017
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
98
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
11k
Cloudless Computingの論文紹介
yuukit
2
390
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
1.4k
エンジニアのためのSRE論文への招待 / Introduction to SRE Papers for Engineers
yuukit
2
11k
博士課程での研究まとめ 2023年1月版 / Summary of my research in the PhD course
yuukit
1
230
AI時代に向けたクラウドにおける信頼性エンジニアリングの未来構想 / DICOMO2022 6A-1
yuukit
7
2.8k
AIOps研究録―SREのための システム障害の自動原因診断 / SRE NEXT 2022
yuukit
10
12k
Interactive AIOps
yuukit
0
2k
Other Decks in Research
See All in Research
KDD論文読み会2024: False Positive in A/B Tests
ryotoitoi
0
140
論文紹介/Expectations over Unspoken Alternatives Predict Pragmatic Inferences
chemical_tree
1
240
初めての研究発表を成功させよう! スライド作成の基本
ayaco0
11
4.6k
The Fellowship of Trust in AI
tomzimmermann
0
110
LLM based AI Agents Overview -What, Why, How-
masatoto
2
530
[第62回NLPコロキウム]「なりきり」を促すHCI設計:対話型接客ロボットの遠隔操作者へのリアルタイム変換音声フィードバックの適用
nami_ogawa
0
290
自動運転・AIシステムの問題を賢く探す・賢く直す / Smart Search & Repair Techniques for Automated Driving Systems and AI Systems
ishikawafyu
0
220
Isotropy, Clusters, and Classifiers
hpprc
3
580
3次元点群の分類における評価指標について
kentaitakura
0
270
Weekly AI Agents News!
masatoto
22
21k
いしかわ暮らしセミナー~移住にまつわるお金の話~
matyuda
0
130
Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve
eumesy
PRO
6
1.1k
Featured
See All Featured
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
27
1.9k
Documentation Writing (for coders)
carmenintech
65
4.4k
Scaling GitHub
holman
458
140k
A Tale of Four Properties
chriscoyier
156
22k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
31
2.6k
Build The Right Thing And Hit Your Dates
maggiecrowley
32
2.3k
The Cost Of JavaScript in 2023
addyosmani
44
5.9k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
504
140k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
327
21k
Producing Creativity
orderedlist
PRO
341
39k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
33k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
26
4.1k
Transcript
TimeFuzeΞʔΩςΫνϟߏ ͯͳ id:y_uuki / @y_uuk1 2017/12/23 ୈ1ճΣϒγεςϜΞʔΩςΫνϟ(WSA)ݚڀձ@ژ ॲཧͱσʔλͱλΠϚʔΛҰମԽͨ͠σʔλύΠϓϥΠϯ
id:y_uuki / @y_uuk1 Yuuki TSUBOUCHI https://yuuk.io/ ͯͳͷΣϒΦϖϨʔγϣϯΤϯδχΞ WSAݚڀձ ཧࣄ?෭ࠪ?
TimeFuze?
࣌ݶ৴ Time Fuze
ىരஔ+Րༀ+λΠϚʔ
ॲཧ+σʔλ+λΠϚʔ
TimeFuze ΞʔΩςΫνϟ
͘͡ 1. എܠͱత 2. TimeFuzeΞʔΩςΫνϟ 3. ߟ 4. Ή͢ͼ
1. എܠͱత
WebαʔϏεͷσʔλॲཧཁٻͷෳࡶԽ • ϦΫΤετʹରͯ͠HTMLͷϨεϙϯεΛฦ٫͢ΔҎ֎ ͷϫʔΫϩʔυͷଟ༷Խ͕ਐΜͰ͍Δ • ੑೳಛੑͷҟͳΔσʔλετΞͷซ༻ɺػցֶशʹΑ Δ࣌ؒपظͰͷେنͳσʔλॲཧΛٻΊΒΕ͍ͯΔ • εέʔϧ͍ͤ͢͞σʔλύΠϓϥΠϯॲཧ͕ඞཁ
ҰൠͷσʔλύΠϓϥΠϯ DataSource Data Destination όονॲཧ λΠϜεέδϡʔϥʔ ىಈ
ҰൠͷσʔλύΠϓϥΠϯͷ՝ • σʔλྔ૿େʹ͋ΘͤͨεέʔϦϯάӡ༻ͷ͠͞ • όονॲཧ్தͷΤϥʔॲཧͷ͠͞ • λΠϜεέδϡʔϥͦͷͷͷӡ༻ͷࡶ͞ • ࣮ߦසઃఆͷॊೈੑͷ͞
σʔλ λΠϚʔ σʔλͱॲཧͱλΠϚʔͷҰମԽϞσϧͷఏҊ ॲཧ DataSource σʔλ λΠϚʔ ॲཧ DataDestination
ҰൠͷσʔλύΠϓϥΠϯͷ՝ͷղܾ • σʔλྔ૿େʹ͋ΘͤͨεέʔϦϯάӡ༻ͷ͠͞ • ࠷ॳ͔ΒฒߦॲཧΛલఏͱͨ͠ΞʔΩςΫνϟͰղܾ • όονॲཧ్தͷΤϥʔॲཧͷ͠͞ • όονॲཧͰͳ͘Ϩίʔυ୯ҐͰδϣϒఆٛͰղܾ •
λΠϜεέδϡʔϥͦͷͷͷӡ༻ͷࡶ͞ • σʔλετΞͷTTLʹΑΓεέδϡʔϥϗετ͕ෆཁ • ࣮ߦසઃఆͷॊೈੑͷ͞ • λΠϚʔઃఆͷཻΛϨίʔυ୯ҐͰઃఆ͢Δ͜ͱͰղܾ
http://blog.yuuk.io/entry/the-rebuild-of-tsdb-on-cloud
֊ܕσʔλετΞߏͷ࣌ܥྻσʔλϕʔε ΠϯϝϞϦ DB ΦϯσΟεΫ DB ΦϒδΣΫτ ετϨʔδ ৽ ݹ ߴ
҆
֊ܕσʔλετΞͷ͍͠ͱ͜Ζͱղܾ • σʔλετΞؒͷσʔλҠಈύΠϓϥΠϯͷεέʔϥϏ ϦςΟ • ಛఆͷϨίʔυͷΈɺσʔλετΞҠಈλΠϛϯάΛ੍ ޚ͢Δ • ͜ΕΒΛTimeFuzeΞʔΩςΫνϟʹΑΓղܾ •
DynamoDBͷTTLͱLambda TriggerʹΑΔ࣮
ͦͷଞͷTimeFuzeͷద༻ΞϓϦέʔγϣϯྫ • ࣌ܥྻσʔλҟৗݕͷֶशϞσϧͷఆظߋ৽ • Ϛϧνςφϯτڥʹ͓͚ΔେنͳSSL/TLSূ໌ॻߋ ৽
2. TimeFuzeΞʔΩςΫνϟ
1. insert DataSource record Data TTL DataDestination Trigger Other API/DataStore
record 2. expire 3. insert or update
TimeFuzeͷϝϦοτ • ฒߦॲཧΛલఏͱͨ͠ΞʔΩςΫνϟͰ͋Δ͜ͱ • Τϥʔ͔Βͷճ෮ॲཧ͕༰қͰ͋Δ͜ͱ • λΠϜεέδϡʔϥͱόονॲཧͷͨΊͷϗετ͘͠ Ϋϥελͷߏஙɾӡ༻͕ෆཁͰ͋Δ͜ͱ • Ϩίʔυ୯ҐλΠϚʔʹΑΓɺδϣϒͷ࣮ߦසΛϨίʔ
υ୯ҐͰௐՄೳͰ͋Δ͜ͱ
AWSʹΑΔ࣮
DynamoDB TTL + Lambda Trigger • DynamoDB: ϑϧϚωʔδυܕͷNoSQLσʔλϕʔε αʔϏεɻTTLΛαϙʔτ͍ͯ͠Δɻ •
Lambda: ΠϕϯτൃՐΛܖػʹFunctionΛݺͼग़ͤΔ • DynamoDB্ͷϨίʔυʹର͢ΔTTLظݶΕΠϕϯτ ΛܖػʹɺLambda FunctionΛτϦΨʔͱ࣮ͯ͠ߦ
1. insert DynamoDB record Data TTL DynamoDB,S3,… Lambda Other API/DataStore
record 2. expire 3. insert or update
ΦϯϓϨϛεڥͰͷ࣮
Redis Keyspace௨ + ϝοηʔδΩϡʔ • RedisͷKeyspace௨ػೳʹΑΓTTLΠϕϯτΛߪಡͰ͖ Δ • ͨͩ͠ɺ҆શͳτϦΨʔॲཧͷͨΊʹɺߪಡͨ͠Πϕϯ τΛϝοηʔδΩϡʔʹΑΓӬଓԽ͢Δඞཁ͕͋Δ
• ݴޠΘͣར༻͍͢͠ϝοηʔδΩϡʔͱͯ͠ Fireworq͕͋Δ
TimeFuzeͷ ۩ମతͳΞϓϦέʔγϣϯ ͷద༻
࣌ܥྻσʔλͷҟৗݕ • ऩूͨ͠ϝτϦοΫܥྻʹରͯ͠ɺֶशϞσϧΛߏங • ৽نϝτϦοΫͷରԠͷͨΊϞσϧΛఆظతʹߋ৽͢Δ • ଞͷσʔλετΞʹҠಈ͢ΔΘ͚Ͱͳ͘ɺDataSource ͱDataDestinationಉҰͷσʔλετΞΛར༻ • ֶ֤शϞσϧͷϨίʔυʹTTLΛઃఆ͠ɺTriggerͰֶश
ॲཧޙɺϞσϧΛߋ৽͢Δ
1. insert DataSource record Metric TTL Trigger ࣌ܥྻ σʔλϕʔε 2.
expire 3. update ֶशॲཧ
େنSSL/TLSূ໌ॻཧ • ϒϩάαʔϏεͳͲͰେྔͷSSL/TLSূ໌ॻΛཧ͠ɺ ϲ݄͝ͱʹߋ৽͢Δέʔε • ূ໌ॻΛσʔλϕʔεʹ֨ೲ͠ಈతऔಘ͢ΔΞʔΩςΫ νϟΛલఏͱ͢Δ • ֤υϝΠϯʹର͢Δূ໌ॻϨίʔυʹTTLΛઃఆ͠ɺ Trigger͕ূ໌ॻΛ࠶औಘ͠ɺDataDestination্ॻ͖
1. insert DataSource record ূ໌ॻ TTL Trigger Let’s Encrypt 2.
expire 3. update ূ໌ॻ ߋ৽ॲཧ
3. ߟ
TimeFuzeͷσϝϦοτ • ΞʔΩςΫνϟϨϕϧ • DataSource্ͷෳͷϨίʔυΛϚʔδ͢ΔॲཧΛͮ͠Β͍ • ࣮Ϩϕϧ • Ұ࣌తʹϨίʔυ͕আ͞ΕΔͨΊɺσʔλͷҰ؏ੑͷ ͕ඞཁ
• TTLظݶ͔Βআॲཧ։࢝·ͰͷσΟϨΠ͕࣌ؒϕετΤ ϑΥʔτͳͨΊɺִ࣌ؒؒ
TimeFuzeͷσϝϦοτղܾҊ • TTLΛར༻ͤͣɺTime to EventػೳΛσʔλετΞʹ ΈࠐΉ • Time to Eventػೳઃఆͨ࣌͠ࠁΛܦաͨ͠Πϕϯτ
ͷΈΛൃߦ͠ɺϨίʔυআ͠ͳ͍
4. Ή͢ͼ
Ή͢ͼ • εέʔϧ͍ͤ͢͞σʔλύΠϓϥΠϯΞʔΩςΫνϟ͕ WebαʔϏε։ൃʹٻΊΒΕ͍ͯΔ • ୯ҰͷλΠϜεέδϡʔϥʔʹΑΔσʔλύΠϓϥΠϯͷ ՝ΛɺσʔλͱॲཧͱλΠϚʔͷҰମԽʹΑΓղܾ͢Δ ख๏ΛఏҊͨ͠ • ࣮ͷʹΑΓɺݱஈ֊Ͱ࣮ӡ༻͢ΔͨΊʹΞϓϦέʔ
γϣϯͷ͕ඞཁͱͳΔ
ࠓޙͷऔΓΈ • طଘͷఆظσʔλύΠϓϥΠϯͷ՝ΛαʔϕΠ͠ཧ • طଘσʔλετΞ࣮ͷTTLσΟϨΠ࣌ؒͷௐࠪ • طଘͷσʔλετΞͷTime to EventͷΈࠐΈ
ࣗੑͷ͋ΔσʔλॲཧͷՄೳੑ • VM => ίϯςφ, ίϯςφ => εϨου • σʔλϕʔε
=> ςʔϒϧ, ςʔϒϧ => σʔλϨίʔυ
SRE/ΦϖϨʔγϣϯΤϯδχΞͷઓ • TimeFuzeͷΑ͏ͳσʔλॲཧΞʔΩςΫνϟΛࣄʹ͍ͯ͠ͳ͍ • ۙͰ͍ͬͯΔ͜ͱɺΠϯϑϥετϥΫνϟࣗମͷมߋΛ࠷ େԽ͢Δࣄ • ج൫ͷந͕͕͋ΔʹͭΕɺSREͷࣄ͕ΑΓࣄۀʹۙ͘ͳ͍ͬͯ͘ • ࣄۀʹ͍ۙ΄ͲɺநԽ͕͘͠ͳΓج൫Ͱٵऩͮ͠Β͘ͳΔ
• ෳͷσʔλετΞΛ͏·͘׆༻ͨ͠ΓɺෳͷϫʔΫϩʔυΛ͏· ͍͍ͯ͘͘͞ΞʔΩςΫνϟʹؔ৺͕͋Δ