Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Applibotデータ分析基盤の刷新 / Applibot
Search
CyberAgent
PRO
February 22, 2019
Technology
2
730
Applibotデータ分析基盤の刷新 / Applibot
サイバーエージェントの技術者(エンジニア・クリエイター)向けカンファレンス『CA BASE CAMP 2019』
Applibotデータ分析基盤の刷新
小川 詩織 伊藤 皓程
CyberAgent
PRO
February 22, 2019
Tweet
Share
More Decks by CyberAgent
See All by CyberAgent
2025年度 生成AI 実践編
cyberagentdevelopers
PRO
3
200
LLMを用いたメタデータベースレコメンド検証
cyberagentdevelopers
PRO
5
1.5k
CodeAgentとMCPで実現するデータ分析エージェント
cyberagentdevelopers
PRO
1
240
SQL Agentによるタップルのデータ利活用促進
cyberagentdevelopers
PRO
1
420
NAB Show 2025 動画技術関連レポート / NAB Show 2025 Report
cyberagentdevelopers
PRO
1
390
【2025年度新卒技術研修】100分で学ぶ サイバーエージェントのデータベース 活用事例とMySQLパフォーマンス調査
cyberagentdevelopers
PRO
7
10k
【CA.ai #1】未来を切り拓くAIエージェントの可能性
cyberagentdevelopers
PRO
3
160
【CA.ai #1】MCP世界への招待:AIエンジニアが創る次世代エージェント連携の世界
cyberagentdevelopers
PRO
2
160
【CA.ai #1】ABEMA のコンテンツ制作を最適化! 生成 AI × クラウド映像編集システム
cyberagentdevelopers
PRO
0
130
Other Decks in Technology
See All in Technology
使いたいMCPサーバーはWeb APIをラップして自分で作る #QiitaBash
bengo4com
0
1.6k
Glacierだからってコストあきらめてない? / JAWS Meet Glacier Cost
taishin
1
150
AI時代の開発生産性を加速させるアーキテクチャ設計
plaidtech
PRO
3
120
AI専用のリンターを作る #yumemi_patch
bengo4com
5
4.1k
KubeCon + CloudNativeCon Japan 2025 に行ってきた! & containerd の新機能紹介
honahuku
0
120
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
shift_evolve
PRO
2
6.7k
マーケットプレイス版Oracle WebCenter Content For OCI
oracle4engineer
PRO
3
960
マネジメントって難しい、けどおもしろい / Management is tough, but fun! #em_findy
ar_tama
7
900
Delta airlines Customer®️ USA Contact Numbers: Complete 2025 Support Guide
deltahelp
0
330
「クラウドコスト絶対削減」を支える技術—FinOpsを超えた徹底的なクラウドコスト削減の実践論
delta_tech
4
140
第4回Snowflake 金融ユーザー会 Snowflake summit recap
tamaoki
1
240
KubeCon + CloudNativeCon Japan 2025 Recap by CA
ponkio_o
PRO
0
290
Featured
See All Featured
Designing Experiences People Love
moore
142
24k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.7k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
810
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Gamification - CAS2011
davidbonilla
81
5.3k
Code Review Best Practice
trishagee
69
18k
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
Transcript
Applibot /
1.')! 2. (" 3.(&*+ 4.& 5.& %
6.$#
48 MF / Shiori Ogawa • - $.".# •
- (6H: .+.( - ED)+J? etc. … • - 2015.04 gloops - Ameba/GREE/FP Web(&,31>5 (K<!-) - /L0 - 2017.08 - 2017.08 2@!-,!-*.%.31&,, - 2018.03; =I(A.R.T. G7 /L=IB9 - 2018.10; &,'!AC'"
# &" / Kotei Ito • -
• - • - 2015.04 QualiArts - () - - 2017 AJA - AJA SSP - 2019 - A.R.T.%$!
6fNc") D.I.V.E. • (Data/Infrastructure/Value/Elevation) 6fNc6f<]IBb8 ! & •
GV2M • Q.35g`Nc EIJ7 Nci;/4KY@C1M- RPEI8X;NcDOh2?T ;W',#SA ^9 ;Nc[^= 1 [d?T Q.115g`Nc HA F: #+(-% a_Ue Zj*-$L> g`',#\38X0
!" #B • ## &;$ 96#B08% /; • !$ 7.(5
/;2<*A:4 • $ @,)>=+'3-1? " #
&R*"@) 6;AJ N:5Q /8 -(=F 1S E< 0CN: OD>B M@
$3H5 #! OD*1S 6;?+ OD% 7, ' ) KPI>B L;G9 2P.I>B 4K OD>B
… • K%"W2`_ • ;ZPO)74J6V9X51 • < CAF<
51 • S)/4LBT"( • ' Y2IQ="( • #&0:,+ :U\3L> • AF*]4LN @ DG • .R/?E • M^!$8-[H etc. ………
…
*3( /+ • :HF7 !!#'-2 $, ". %
"0)4 &1 & 68<2 5 ;�#-.3… *% 1'!($G $"/)2 … @A,+E?92 … =C2 D4B>
4I @5_1 • 01-BE 9> ,.BE
&#BE !,; -4FG= !,7A < ,./2:6= • !,,.2:H /8 -? '0+ -C3J $1D (1% *)"!
3F =7_2 =OJ8 !-/2:UU9;>DF5: D:@ IRB ,'0*)."4 ?@#!0<
$-C8+#+1&(%B56 ."ADE &.Q>G ,3(9# 2Q>G .0*1K @ G 2:<AMP? @ HC 4E/";E/")$6LN&.G %!'(7 +3-3(7C
9b!L\TN • + &- ", $ (!#.
*% .'CD9b P<7C+)1Q8.'Y; WVOJ#>! &(*#[TN=@:MB ! .'C #d` 46FE^>"aX%,cS#Z • )'&- $-/ ($0# G?5 .'Y3" Q8 !_H 2AI45!…U]KLR !!
Applibot8316 • #%)2, !$DWH(Redshift)/' (7# 5(/') •
Redshift !20 5(Redshift !/20 • RedshiftBI"*4& - TableauGeckoBoard .+
*7 #<)Va • 1 '(# % $& UZA1/%*7::.HM
"GB#XI # >R • )* ! !#8,DB \D$/'6=)1;+3b ?! *7C'5=2O&'07%1SF#&(-9= &'07YT 8,.0/.\D API<) ^J;+3# • % +$% "! &'07YT<)8,'5=2[]_K .0/.'=4;E WN!P! #QL O^J<) `@#
/,?D'28- • DB /(6/,&UDBdumpAf$ P<C5*/,Update %@:=aI !idJ% /, 1:M>c%$>RNbE
Z OKApplibot • DB History DB/,\History.17&?D$"[ /, 1:M>c%$ • History?D/,&8)h ?DBB%VHAf $ ;F _#DWH SeT $ ETLYG!^]X`gT $ +0639)%6'7,(4RNJQ LWApplibot
DB
+ History DB ()
.+L?&04, • DWH( ) ;X\.+.+%:#"5JVL?@ZY.+N?()-2 PrestoHive, DremelUE%1) RedshiftBigQuery 7P
CIApplibot Applibot<G[ • DB DB3'.+History.+%L? 65JV.+O "L?RK T • (+DWH) .+%" QJV=$!>W FS/*FS3' /*8$9I.+%H" "MB DAApplibot
=32 B7 logloglogloglog logloglogloglog loglogloglogolo gloglog logloglogloglog logloglogloglog loglogloglogolo gloglog
>@1 ' # ( * , * )$ +" 40!- %* F.O.X. etc… :E +96&) DAGJ<H. F5/I;?1:E(BI) HadoopC:E=8/I:E
' E0B4 logloglogloglog logloglogloglog loglogloglogolo gloglog logloglogloglog logloglogloglog loglogloglogolo
gloglog >A. # $ & ( & %! ' 2,) "& F.O.X. etc… /C5H=:*L 1F-8?9IK M ' 7 G 7G H3+J;@.7G(BI) HadoopD7G <6+J7G
20180424 AWS Black Belt Online Seminar AWS
https://www.slideshare.net/AmazonWebServicesJapan/20180424-aws-black-belt-online-seminar-aws?ref=https://aws.amazon.com/jp/blogs/news/webinar- bb-datalake-2018/
$'.(-6RW • Redshift]g y:H (^Jc85%…) onMf \! aZ(&8 `l>K@ /,on
• aZ(&8 Y pEaZ _" Redshift#ftOX2%3% SortKeyNqH! (7+,Ue! ACG;v ik= • 1+0/,#wrm SF+0859)*4+Flydata#^ … <DETLdIQLz! • P[hVbsj?T/,#B!aZuxH
OEIK • -)S^?PEIK RedshiftFlydata 7@1600HMg -)%C_ &(.g#… • a@/+*UDY dfWD
$ XD8J a@/+*-)%>$# UD0)4\L!$dfB • Redshift-)%V>$#bFFlydata`Ti [RUD%9"h_,236 =]'1(%N" c;A%G# eZ 15:-)QKPI<$ !…
!$.%,9T\ (}t) • RedshiftajT\ • .6=k[)$7esyY@F dS`P')0 | •
Redshift RpO /+AIJ | • n/+HS_eOHg i • w]ZDBB]ZTDQXz • %;)+YhGINB%"< ^NBlM urKv • Ex$*0$Vacuumpb35#7>),:1>& AW • ?C/+fm~7-,V • 2)0/+ {qoUL)0<8>&(4)Flydata c
jX\` () • mX\`RedshiftFlydata1600[d • L9/:/ EJ)4>!$+*tc vL!k.b F
4>Op"ga)++ • NU L#G^ i,F'c' "]H ' • B0 "Sw ;@;@lz uV- • 83.f!se%QT _+(( • R;54"rW { xW- yWG^ • nEJh}KBI6A"?C7G^,F *!, • Redshift!83.sP-,&!YFlydata"q }o rW.I+"7=A D O~ 1<2.h+"M T.Z, |
Redshifti & • D1=8*F23F2kS&<E$Gdj63fRW -,D027'Xq#027G-,D QJ $ Z TZ 63#fR$
• Lrp\ZDB"#o[ • Ua] DWHcn b $ 63'N%^ %lHYo[ • ?27>C-5)2"B1/2OeImP:+;+ ] g$ !(4A9F._M$@F5hVK %Ua`b
• E6O? • )* &T4.G> • -V8AHSWX0I •
2,KPIG>C81F • % F.O.X.(dump# (QDBOK)N# /30I • %HSM<!&P@ • G>,'+"$+B; O? • DGG>9= 7U JLYR:K5
None
ETL
Lambda ""(#event ./16 : *9 =>- !
3;!70 $,)!</'% → 8& 5+42 API
^LpX 7$8,%5tH@ `W ?wPZbsxyCd E:KPI`W\8D_ ! 4+,F.O.X.n1,BGCd +(4bskU#-5qY
! `W:69039+[T# !pX RV O u!eizrSfK → M200QF/> → 10Q+α/> M1/20'.2 → → `W)&7#aPdkU! → ADI; lo<" j → S3]APIqc →=NJ/8hm →=NJ/8hm →/8hmgv+*8
• %#')F@ • )!)"0A/L< • >9%4J2 • RedshiftD! I,
• Athena ()&$6K ( :*%%) • =-%6ECG2 • >91.Glue B3? NM2 (5;H) • 8%0 4J2+7Redshift Spectrum …
>M(6.6-O;G • JV QD LHT5'IY8
LH 5' QD&%2." NC :X)12ETB 9 • KPI SAJ7KPI?!R5' 8=P KPI05*#&/<,4KPI&$2KF(+/ W • BI BI-3 U@QD…
… mail:
[email protected]
Facebook: https://www.facebook.com/100008453594381 Chatwork
mail:
[email protected]
Facebook: https://www.facebook.com/mk.mdrm.cmtr Chatwork …m(_ _)m