Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Akka streamsを活用したログ集計に優しいデータフローの構築
Search
Shuhei Kimura
December 05, 2018
Technology
2
4.9k
Akka streamsを活用したログ集計に優しいデータフローの構築
Shuhei Kimura
December 05, 2018
Tweet
Share
More Decks by Shuhei Kimura
See All by Shuhei Kimura
pmconf2017
kimurashuhei
0
3.4k
MDH-LT-Thursday-20170126
kimurashuhei
0
1.5k
AWS Summit Tokyo 2015
kimurashuhei
0
74
Other Decks in Technology
See All in Technology
Python と Snowflake はズッ友だょ!~ Snowflake の Python 関連機能をふりかえる ~
__allllllllez__
1
110
AOAI をきっかけに 社内の Azure 管理を見直した話
recruitengineers
PRO
1
260
マルチアカウント環境への発見的統制の導入
ch1aki
1
1.3k
MapLibreとAmazon Location Service
dayjournal
1
150
開発生産性向上サービスを作るFindyが自分たちで開発生産性を爆上げした組織づくりの歩み / Findy's path to boosting its own development productivity 2024-04-17
ma3tk
3
630
Cracking the KubeCon CfP
inductor
2
230
JSON攻略法.pdf
miyakemito
8
4.9k
開発生産性大幅アップ!Postman VS Code拡張機能
nagix
2
370
MySQL の SQL クエリチューニングの要所を掴む勉強会
andpad
2
6.1k
ここが嬉しいABAC ここが辛いよABAC #再解説+補足編
masahirokawahara
1
270
コードを書く隙間を見つけて生きていく技術/Findy 思考の現在地
fujiwara3
27
5.9k
コンパウンドスタートアップのためのスケーラブルでセキュアなInfrastructure as Codeパイプラインを考える / Scalable and Secure Infrastructure as Code Pipeline for a Compound Startup
yuyatakeyama
4
4.7k
Featured
See All Featured
A Tale of Four Properties
chriscoyier
151
22k
Raft: Consensus for Rubyists
vanstee
132
6.3k
Rails Girls Zürich Keynote
gr2m
91
13k
The Mythical Team-Month
searls
216
42k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
Adopting Sorbet at Scale
ufuk
68
8.6k
Agile that works and the tools we love
rasmusluckow
325
20k
Optimising Largest Contentful Paint
csswizardry
8
2.4k
Code Review Best Practice
trishagee
55
15k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
78
42k
[RailsConf 2023] Rails as a piece of cake
palkan
23
3.9k
The Art of Programming - Codeland 2020
erikaheidi
42
12k
Transcript
BLLBTUSFBNΛ׆༻ͨ͠ϩάूܭʹ༏͍͠ σʔλϑϩʔͷߏஙʹ͍ͭͯ !#POpSF#BDLFOE
1SPpMF ໊લ ଜऺฏ4IVIFJ,JNVSB IUUQTXXXGBDFCPPLDPNTIVIFJLJNVSB ॴଐ גࣜձࣾαΠόʔΤʔδΣϯτ
ΞυςΫຊ෦"E5FDI4UVEJP%ZOBMZTU 1SPEVDU.BOBHFS4PGUXBSF&OHJOFFS ܦྺ ৽ଔೖࣾ 5SBDLJOH 1$"ET 4NBSU 1IPOF"ET %ZOBNJD 3FUBSHFUJOH "ET (MPCBM &YQBOTJPO
%ZOBMZTUͱ͍͏ࠂ৴αʔϏεͷنײ d,SFRTFD NTGPSFBDI &$*OTUBODFT 'PS#VZJOH*NQSFTTJPOT 5# MPHpMFTEBZ ѻ͏ͷ·͊·͊େมɻɻ ϚϧνϦʔδϣϯͰ
࣮ࡍ͜ͷഒͷྔ
5PEBZ`T5PQJDdϩάσʔλΛ៉ྷʹ4ʹӡͿd d,SFRTFD 5# QFSEBZ ‑‑‑‑‑
"LLB4USFBN͕ొͨ͠എܠ طଘͷ-JCSBSZ͕όά͍ͬͯͨ ,JOFTJT$POOFDUPS-JCSBSZͦͦ044ʹλμΓ͍ͯ͠Δ͜ͱ͕ ΑΖ͘͠ͳ͍͕ɺ ͍ͩͿϝϯς͞Ε͍ͯͳ͍ͷͰɺ CVHpY͢Δͷɻɻ ͍·ͩͱ͍Ζ͍ΖͳιϦϡʔγϣϯ͕͋Δʁ ,JOFTJT'JSFIPTFʁΩϨΠʹ4ʹΦϒδΣΫτΛஔ͢Δʹɺ ͕ඞཁʢ-BNCEB'VODUJPOڬΉͷͪΐͬͱɻʣ
-BNCEB'VODUJPOʁ,JOFTJT%BUB4USFBN͔Βͷ1PMMJOH*OUFSWBMΛ ࢦఆͰ͖ͳ͍ͷͰ4ͷΦϒδΣΫτ͕େʹ ʢ&5-Λ"84(MVFʹͤͯΑ͍͕ɻ8PSL'MPX&OHJOFొɻɻʣ ,$-XJUI"LLB4USFBN͕γϯϓϧͰΑ͍͔ʁ
"LLB4USFBNTͷߏཁૉ 4PVSDFϝοηʔδΛు͖ग़͢ίϯϙʔωϯτ ࢝ 'MPXϝοηʔδΛड͚औͬͯϝοηʔδΛૹΔίϯϙʔωϯτ 4JOLϝοηʔδΛड͚औΔίϯϙʔωϯτ ऴ ࢀߟURL: http://chariotsolutions.com/wp-content/uploads/blog/2016/08/akkastreamstopology.png
͜ΕΒͷίϯϙʔωϯτΛΈ߹Θͤͯ 3VOOBCMF(SBQIΛߏங
৴ϩάΛ4ʹӡͿͨΊͷ(SBQIߏங Kinesis Source Extract Record Flow Upload Log Flow Sink
˞ຊ424NFTTBHJOH'MPXɺ #SPBE$BTUͳͲར༻͍ͯ͠·ׂ͕͢Ѫ
ͲͷΜ͕ूܭʹ༏͍͠ͷ͔ʁ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ TCVDLFUYYYYYYYYZZZZH[ YYYYYY[[[[[H[ ʜ TCVDLFUYYYYZZH[ YY[[H[ ʜ 6QMPBEͨ࣌ؒ͠Ͱͳ͘ɺ ϩά͕ग़ྗ͞Εͨ࣌ؒϕʔεͰ 4ͷ࣌ؒ۠Γͷύʔςʔγϣϯ Ͱ֨ೲ 4QBSL$MVTUFS4ͷࢦఆ࣌ؒͷύεͷϩάΛ ରʹूܭॲཧΛΒͤΔ ूܭɺIPVSMZ EBJMZ ͕ओ
ͲͷΜ͕ूܭʹ༏͍͠ͷ͔ʁ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ TCVDLFUYYYYYYYYZZZZH[ YYYYYY[[[[[H[ ʜ TCVDLFUYYYYZZH[ YY[[H[ ʜ Before After BufferType Memory Local Disc S3 Object Size 5mbલޙ 15mbલޙ ूܭ࣌ؒ 30min 15min
͓·͚ɿ&$4ͷDPNQVUF3FTPVSDFʹ༏͍͠࡞Γʹ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ 'MPXͷத ,JOFTJT3FDPSE͔ΒɺඞཁͳใΛൈ͖ग़ͯ͠ɺΩϨΠʹ͢Δ ύʔςΟγϣϯ͝ͱʹ-PDBMͷετϨʔδʹϩάΛόοϑΝϦϯά ϝϞϦফඅ੍ʢ(#(#ʣ Ұఆ༰ྔҰఆ࣌ؒܦա͢Δͱ4ʹ1VU0CKFDU ‑‑‑‑‑
͓ฉ͖͍͖ͨͩ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ