Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Akka streamsを活用したログ集計に優しいデータフローの構築
Search
Shuhei Kimura
December 05, 2018
Technology
2
5.6k
Akka streamsを活用したログ集計に優しいデータフローの構築
Shuhei Kimura
December 05, 2018
Tweet
Share
More Decks by Shuhei Kimura
See All by Shuhei Kimura
pmconf2017
kimurashuhei
0
3.6k
MDH-LT-Thursday-20170126
kimurashuhei
0
1.6k
AWS Summit Tokyo 2015
kimurashuhei
0
110
Other Decks in Technology
See All in Technology
2つのフロントエンドと状態管理
mixi_engineers
PRO
3
160
KotlinConf 2025_イベントレポート
sony
1
140
メルカリIBISの紹介
0gm
0
420
Snowflake Intelligenceにはこうやって立ち向かう!クラシルが考えるAI Readyなデータ基盤と活用のためのDataOps
gappy50
0
280
エンジニアが主導できる組織づくり ー 製品と事業を進化させる体制へのシフト
ueokande
1
110
いま注目のAIエージェントを作ってみよう
supermarimobros
0
360
Aurora DSQLはサーバーレスアーキテクチャの常識を変えるのか
iwatatomoya
1
1.2k
roppongirb_20250911
igaiga
1
250
企業の生成AIガバナンスにおけるエージェントとセキュリティ
lycorptech_jp
PRO
3
200
20250913_JAWS_sysad_kobe
takuyay0ne
2
250
要件定義・デザインフェーズでもAIを活用して、コミュニケーションの密度を高める
kazukihayase
0
120
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
sh_fk2
3
460
Featured
See All Featured
Designing for humans not robots
tammielis
253
25k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
The World Runs on Bad Software
bkeepers
PRO
70
11k
How STYLIGHT went responsive
nonsquared
100
5.8k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
4k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
3k
Context Engineering - Making Every Token Count
addyosmani
3
62
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.7k
The Invisible Side of Design
smashingmag
301
51k
Transcript
BLLBTUSFBNΛ׆༻ͨ͠ϩάूܭʹ༏͍͠ σʔλϑϩʔͷߏஙʹ͍ͭͯ !#POpSF#BDLFOE
1SPpMF ໊લ ଜऺฏ4IVIFJ,JNVSB IUUQTXXXGBDFCPPLDPNTIVIFJLJNVSB ॴଐ גࣜձࣾαΠόʔΤʔδΣϯτ
ΞυςΫຊ෦"E5FDI4UVEJP%ZOBMZTU 1SPEVDU.BOBHFS4PGUXBSF&OHJOFFS ܦྺ ৽ଔೖࣾ 5SBDLJOH 1$"ET 4NBSU 1IPOF"ET %ZOBNJD 3FUBSHFUJOH "ET (MPCBM &YQBOTJPO
%ZOBMZTUͱ͍͏ࠂ৴αʔϏεͷنײ d,SFRTFD NTGPSFBDI &$*OTUBODFT 'PS#VZJOH*NQSFTTJPOT 5# MPHpMFTEBZ ѻ͏ͷ·͊·͊େมɻɻ ϚϧνϦʔδϣϯͰ
࣮ࡍ͜ͷഒͷྔ
5PEBZ`T5PQJDdϩάσʔλΛ៉ྷʹ4ʹӡͿd d,SFRTFD 5# QFSEBZ ‑‑‑‑‑
"LLB4USFBN͕ొͨ͠എܠ طଘͷ-JCSBSZ͕όά͍ͬͯͨ ,JOFTJT$POOFDUPS-JCSBSZͦͦ044ʹλμΓ͍ͯ͠Δ͜ͱ͕ ΑΖ͘͠ͳ͍͕ɺ ͍ͩͿϝϯς͞Ε͍ͯͳ͍ͷͰɺ CVHpY͢Δͷɻɻ ͍·ͩͱ͍Ζ͍ΖͳιϦϡʔγϣϯ͕͋Δʁ ,JOFTJT'JSFIPTFʁΩϨΠʹ4ʹΦϒδΣΫτΛஔ͢Δʹɺ ͕ඞཁʢ-BNCEB'VODUJPOڬΉͷͪΐͬͱɻʣ
-BNCEB'VODUJPOʁ,JOFTJT%BUB4USFBN͔Βͷ1PMMJOH*OUFSWBMΛ ࢦఆͰ͖ͳ͍ͷͰ4ͷΦϒδΣΫτ͕େʹ ʢ&5-Λ"84(MVFʹͤͯΑ͍͕ɻ8PSL'MPX&OHJOFొɻɻʣ ,$-XJUI"LLB4USFBN͕γϯϓϧͰΑ͍͔ʁ
"LLB4USFBNTͷߏཁૉ 4PVSDFϝοηʔδΛు͖ग़͢ίϯϙʔωϯτ ࢝ 'MPXϝοηʔδΛड͚औͬͯϝοηʔδΛૹΔίϯϙʔωϯτ 4JOLϝοηʔδΛड͚औΔίϯϙʔωϯτ ऴ ࢀߟURL: http://chariotsolutions.com/wp-content/uploads/blog/2016/08/akkastreamstopology.png
͜ΕΒͷίϯϙʔωϯτΛΈ߹Θͤͯ 3VOOBCMF(SBQIΛߏங
৴ϩάΛ4ʹӡͿͨΊͷ(SBQIߏங Kinesis Source Extract Record Flow Upload Log Flow Sink
˞ຊ424NFTTBHJOH'MPXɺ #SPBE$BTUͳͲར༻͍ͯ͠·ׂ͕͢Ѫ
ͲͷΜ͕ूܭʹ༏͍͠ͷ͔ʁ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ TCVDLFUYYYYYYYYZZZZH[ YYYYYY[[[[[H[ ʜ TCVDLFUYYYYZZH[ YY[[H[ ʜ 6QMPBEͨ࣌ؒ͠Ͱͳ͘ɺ ϩά͕ग़ྗ͞Εͨ࣌ؒϕʔεͰ 4ͷ࣌ؒ۠Γͷύʔςʔγϣϯ Ͱ֨ೲ 4QBSL$MVTUFS4ͷࢦఆ࣌ؒͷύεͷϩάΛ ରʹूܭॲཧΛΒͤΔ ूܭɺIPVSMZ EBJMZ ͕ओ
ͲͷΜ͕ूܭʹ༏͍͠ͷ͔ʁ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ TCVDLFUYYYYYYYYZZZZH[ YYYYYY[[[[[H[ ʜ TCVDLFUYYYYZZH[ YY[[H[ ʜ Before After BufferType Memory Local Disc S3 Object Size 5mbલޙ 15mbલޙ ूܭ࣌ؒ 30min 15min
͓·͚ɿ&$4ͷDPNQVUF3FTPVSDFʹ༏͍͠࡞Γʹ Kinesis Source Extract Record Flow Upload Log Flow Sink
‑‑‑‑‑ 'MPXͷத ,JOFTJT3FDPSE͔ΒɺඞཁͳใΛൈ͖ग़ͯ͠ɺΩϨΠʹ͢Δ ύʔςΟγϣϯ͝ͱʹ-PDBMͷετϨʔδʹϩάΛόοϑΝϦϯά ϝϞϦফඅ੍ʢ(#(#ʣ Ұఆ༰ྔҰఆ࣌ؒܦա͢Δͱ4ʹ1VU0CKFDU ‑‑‑‑‑
͓ฉ͖͍͖ͨͩ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ