Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Akka streamsを活用したログ集計に優しいデータフローの構築
Search
Shuhei Kimura
December 05, 2018
Technology
2
5.4k
Akka streamsを活用したログ集計に優しいデータフローの構築
Shuhei Kimura
December 05, 2018
Tweet
Share
More Decks by Shuhei Kimura
See All by Shuhei Kimura
pmconf2017
kimurashuhei
0
3.6k
MDH-LT-Thursday-20170126
kimurashuhei
0
1.6k
AWS Summit Tokyo 2015
kimurashuhei
0
100
Other Decks in Technology
See All in Technology
25分で解説する「最小権限の原則」を実現するための AWS「ポリシー」大全 / 20250625-aws-summit-aws-policy
opelab
9
1.1k
Understanding_Thread_Tuning_for_Inference_Servers_of_Deep_Models.pdf
lycorptech_jp
PRO
0
110
CI/CD/IaC 久々に0から環境を作ったらこうなりました
kaz29
1
160
Azure AI Foundryでマルチエージェントワークフロー
seosoft
0
180
実践! AIエージェント導入記
1mono2prod
0
160
250627 関西Ruby会議08 前夜祭 RejectKaigi「DJ on Ruby Ver.0.1」
msykd
PRO
2
240
rubygem開発で鍛える設計力
joker1007
2
190
Welcome to the LLM Club
koic
0
160
TechLION vol.41~MySQLユーザ会のほうから来ました / techlion41_mysql
sakaik
0
180
監視のこれまでとこれから/sakura monitoring seminar 2025
fujiwara3
11
3.8k
GitHub Copilot の概要
tomokusaba
1
130
Snowflake Summit 2025 データエンジニアリング関連新機能紹介 / Snowflake Summit 2025 What's New about Data Engineering
tiltmax3
0
300
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Unsuck your backbone
ammeep
671
58k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
Producing Creativity
orderedlist
PRO
346
40k
Making the Leap to Tech Lead
cromwellryan
134
9.3k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Music & Morning Musume
bryan
46
6.6k
Statistics for Hackers
jakevdp
799
220k
Rails Girls Zürich Keynote
gr2m
94
14k
Gamification - CAS2011
davidbonilla
81
5.3k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
161
15k
Transcript
BLLBTUSFBNΛ׆༻ͨ͠ϩάूܭʹ༏͍͠ σʔλϑϩʔͷߏஙʹ͍ͭͯ !#POpSF#BDLFOE
1SPpMF ໊લ ଜऺฏ4IVIFJ,JNVSB IUUQTXXXGBDFCPPLDPNTIVIFJLJNVSB ॴଐ גࣜձࣾαΠόʔΤʔδΣϯτ
ΞυςΫຊ෦"E5FDI4UVEJP%ZOBMZTU 1SPEVDU.BOBHFS4PGUXBSF&OHJOFFS ܦྺ ৽ଔೖࣾ 5SBDLJOH 1$"ET 4NBSU 1IPOF"ET %ZOBNJD 3FUBSHFUJOH "ET (MPCBM &YQBOTJPO
%ZOBMZTUͱ͍͏ࠂ৴αʔϏεͷنײ d,SFRTFD NTGPSFBDI &$*OTUBODFT 'PS#VZJOH*NQSFTTJPOT 5# MPHpMFTEBZ ѻ͏ͷ·͊·͊େมɻɻ ϚϧνϦʔδϣϯͰ
࣮ࡍ͜ͷഒͷྔ
5PEBZ`T5PQJDdϩάσʔλΛ៉ྷʹ4ʹӡͿd d,SFRTFD 5# QFSEBZ ‑‑‑‑‑
"LLB4USFBN͕ొͨ͠എܠ طଘͷ-JCSBSZ͕όά͍ͬͯͨ ,JOFTJT$POOFDUPS-JCSBSZͦͦ044ʹλμΓ͍ͯ͠Δ͜ͱ͕ ΑΖ͘͠ͳ͍͕ɺ ͍ͩͿϝϯς͞Ε͍ͯͳ͍ͷͰɺ CVHpY͢Δͷɻɻ ͍·ͩͱ͍Ζ͍ΖͳιϦϡʔγϣϯ͕͋Δʁ ,JOFTJT'JSFIPTFʁΩϨΠʹ4ʹΦϒδΣΫτΛஔ͢Δʹɺ ͕ඞཁʢ-BNCEB'VODUJPOڬΉͷͪΐͬͱɻʣ
-BNCEB'VODUJPOʁ,JOFTJT%BUB4USFBN͔Βͷ1PMMJOH*OUFSWBMΛ ࢦఆͰ͖ͳ͍ͷͰ4ͷΦϒδΣΫτ͕େʹ ʢ&5-Λ"84(MVFʹͤͯΑ͍͕ɻ8PSL'MPX&OHJOFొɻɻʣ ,$-XJUI"LLB4USFBN͕γϯϓϧͰΑ͍͔ʁ
"LLB4USFBNTͷߏཁૉ 4PVSDFϝοηʔδΛు͖ग़͢ίϯϙʔωϯτ ࢝ 'MPXϝοηʔδΛड͚औͬͯϝοηʔδΛૹΔίϯϙʔωϯτ 4JOLϝοηʔδΛड͚औΔίϯϙʔωϯτ ऴ ࢀߟURL: http://chariotsolutions.com/wp-content/uploads/blog/2016/08/akkastreamstopology.png
͜ΕΒͷίϯϙʔωϯτΛΈ߹Θͤͯ 3VOOBCMF(SBQIΛߏங
৴ϩάΛ4ʹӡͿͨΊͷ(SBQIߏங Kinesis Source Extract Record Flow Upload Log Flow Sink
˞ຊ424NFTTBHJOH'MPXɺ #SPBE$BTUͳͲར༻͍ͯ͠·ׂ͕͢Ѫ
ͲͷΜ͕ूܭʹ༏͍͠ͷ͔ʁ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ TCVDLFUYYYYYYYYZZZZH[ YYYYYY[[[[[H[ ʜ TCVDLFUYYYYZZH[ YY[[H[ ʜ 6QMPBEͨ࣌ؒ͠Ͱͳ͘ɺ ϩά͕ग़ྗ͞Εͨ࣌ؒϕʔεͰ 4ͷ࣌ؒ۠Γͷύʔςʔγϣϯ Ͱ֨ೲ 4QBSL$MVTUFS4ͷࢦఆ࣌ؒͷύεͷϩάΛ ରʹूܭॲཧΛΒͤΔ ूܭɺIPVSMZ EBJMZ ͕ओ
ͲͷΜ͕ूܭʹ༏͍͠ͷ͔ʁ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ TCVDLFUYYYYYYYYZZZZH[ YYYYYY[[[[[H[ ʜ TCVDLFUYYYYZZH[ YY[[H[ ʜ Before After BufferType Memory Local Disc S3 Object Size 5mbલޙ 15mbલޙ ूܭ࣌ؒ 30min 15min
͓·͚ɿ&$4ͷDPNQVUF3FTPVSDFʹ༏͍͠࡞Γʹ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ 'MPXͷத ,JOFTJT3FDPSE͔ΒɺඞཁͳใΛൈ͖ग़ͯ͠ɺΩϨΠʹ͢Δ ύʔςΟγϣϯ͝ͱʹ-PDBMͷετϨʔδʹϩάΛόοϑΝϦϯά ϝϞϦফඅ੍ʢ(#(#ʣ Ұఆ༰ྔҰఆ࣌ؒܦա͢Δͱ4ʹ1VU0CKFDU ‑‑‑‑‑
͓ฉ͖͍͖ͨͩ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ