Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Introduction to "Spark Connect" and Implementat...
Search
LINEヤフーTech (LY Corporation Tech)
PRO
June 30, 2025
Technology
0
89
Introduction to "Spark Connect" and Implementation of Managed Spark as a Service
「Spark Connect」プロトコルについて紹介し、Spark Connectを活用した社内管理型Spark as a Serviceの実装アーキテクチャおよび使用例を共有します。
LINEヤフーTech (LY Corporation Tech)
PRO
June 30, 2025
Tweet
Share
More Decks by LINEヤフーTech (LY Corporation Tech)
See All by LINEヤフーTech (LY Corporation Tech)
Yahoo!広告ビジネス基盤におけるバックエンド開発
lycorptech_jp
PRO
1
280
Yahoo!ニュースにおけるソフトウェア開発
lycorptech_jp
PRO
0
380
マルチモーダル基盤モデルに基づく動画と音の解析技術
lycorptech_jp
PRO
9
740
AI駆動開発 with MixLeap Study【大阪支部 #3】
lycorptech_jp
PRO
0
360
Yahoo!しごとカタログ 新しい境地を創るエンジニア募集!
lycorptech_jp
PRO
3
430
データグループにおけるフロントエンド開発
lycorptech_jp
PRO
2
880
Yahoo!知恵袋におけるフロントエンド開発
lycorptech_jp
PRO
0
1.1k
"LINE Planet" and AI: Conversations with AI
lycorptech_jp
PRO
0
83
Seamless inventory management with AI
lycorptech_jp
PRO
0
41
Other Decks in Technology
See All in Technology
Webアクセシビリティ入門
recruitengineers
PRO
1
370
[CV勉強会@関東 CVPR2025 読み会] MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos (Li+, CVPR2025)
abemii
0
200
Goでマークダウンの独自記法を実装する
lag129
0
220
Go で言うところのアレは TypeScript で言うとコレ / Kyoto.なんか #7
susisu
7
1.9k
Browser
recruitengineers
PRO
4
520
MySQL HeatWave:サービス概要のご紹介
oracle4engineer
PRO
4
1.7k
会社にデータエンジニアがいることでできるようになること
10xinc
9
1.6k
あなたの知らない OneDrive
murachiakira
0
240
どこで動かすか、誰が動かすか 〜 kintoneのインフラ基盤刷新と運用体制のシフト 〜
ueokande
0
190
ZOZOTOWNフロントエンドにおけるディレクトリの分割戦略
zozotech
PRO
18
5.5k
アジャイルテストで高品質のスプリントレビューを
takesection
0
120
あとはAIに任せて人間は自由に生きる
kentaro
3
1.1k
Featured
See All Featured
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
900
Build The Right Thing And Hit Your Dates
maggiecrowley
37
2.8k
Unsuck your backbone
ammeep
671
58k
Music & Morning Musume
bryan
46
6.7k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
46
7.6k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
The Invisible Side of Design
smashingmag
301
51k
Six Lessons from altMBA
skipperchong
28
4k
Bash Introduction
62gerente
614
210k
How GitHub (no longer) Works
holman
315
140k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
1k
A Tale of Four Properties
chriscoyier
160
23k
Transcript
*OUSPEVDUJPOUP4QBSL$POOFDUBOE *NQMFNFOUBUJPOPG.BOBHFE4QBSLBTB 4FSWJDF %BUB1MBUGPSN&OHJOFFSJOH .BSL-FF 이승 李承
"QBDIF4QBSL$POOFDU "QBDIF4QBSLBTB4FSWJDF %&.04QBSLBTB4FSWJDF %&.04QBSL$MJFOU 3FDBQ
"HFOEB
4QBSL$POOFDU
"QBDIF4QBSLä JTBNVMUJMBOHVBHFFOHJOFGPSFYFDVUJOHEBUBFOHJOFFSJOH EBUB TDJFODF BOENBDIJOFMFBSOJOHPOTJOHMFOPEFNBDIJOFTPSDMVTUFST ,FZ'FBUVSFT #BUDITUSFBNJOHEBUB 42-BOBMZUJDT
%BUBTDJFODFBUTDBMF .BDIJOFMFBSOJOH "QBDIF4QBSL IUUQTTQBSLBQBDIFPSH
4QBSL$MVTUFS IUUQTTQBSLBQBDIFPSHEPDTMBUFTUDMVTUFSPWFSWJFXIUNM
$IBMMFOHFTPGUSBEJUJPOBM4QBSL Single Point of Failure - Limited Fault Tolerance -
Resource Bottlenecks Network Dependency - Network partitions or instability disrupts execution - Clients Need Direct Access - Scheduling Restrictions Difficult Multi-Tenancy - Cluster endpoint must be protected 5JHIUDPVQMJOHPG$MJFOUBOE%SJWFS
3FNPUFDPOOFDUJWJUZUP4QBSLDMVTUFST CZ%BUB'SBNF "1*BOEVOSFTPMWFEMPHJDBMQMBOT WJBH31$ XJUI"QBDIF"SSPXFODPEFESPXCBUDIFT $BOCFFNCFEEFEFWFSZXIFSF BQQMJDBUJPOTFSWFST
*%&T OPUFCPPLTBOEQSPHSBNNJOHMBOHVBHFT "CPVU4QBSL$POOFDU %FDPVQMFMBZFSCFUXFFO$MJFOUBOE%SJWFS
4QBSL$POOFDUBSDIJUFDUVSF IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNM
4QBSL$POOFDUBSDIJUFDUVSF IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNM
4QBSL$POOFDU8PSLGMPX IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNM
4QBSLWT4QBSL$POOFDU TQBSL TQBSLDPOOFDU DMJFOUMPDBUJPO JODMVTUFS PVUDMVTUFS DMJFOUSFRVJSFNFOU GVMMTQBSL DMVTUFSBDDFTTBOE +7.CBTFE
UIJODMJFOUNJOJNBMH31$ DMJFOUXJUI%BUB'SBNF -JCSBSZ OFUXPSLUPQPMPHZ NVTUIBWFEJSFDUOFUXPSL BDDFTTUPUIFDMVTUFS SFNPUFBDDFTTWJBH31$ TDBMBCJMJUZ ESJWFS DMJFOUIBOEMFTBMM TDIFEVMJOHBOEDPPSEJOBUJPO ESJWFSSVOTPODMVTUFSDMJFOUT BSFTUBUFMFTT GBVMUUPMFSBODF DMJFOUDSBTIDBVTFTKPCMPTU DMJFOUDBOEJTDPOOFDUBOE SFDPOOFDU
4QBSLWT4QBSL$POOFDU DPOU TQBSL TQBSLDPOOFDU MBOHVBHF MJNJUFEUPTQBSLTVQQPSUFE MBOHVBHFT+7. 1Z4QBSL 3 BOZ
FY KBWBTDSJQU HP D SVTU DMPVE OBUJWF UJHIUDPVQMJOHBOEESJWFS SFRVJSFNFOUTMJNJUGMFYJCJMJUZ EFTJHOFEGPSDMPVEOBUJWF DPOUBJOFSJ[FEBOETFSWFSMFTT TFDVSJUZ FYQPTJOHDMVTUFSFOEQPJOUTDBO SBJTFTFDVSJUZDPODFSOT DBOTFDVSFH31$ DIBOOFMXJUI DFOUSBMJ[FEDPOUSPM WFSTJPOJOH DMJFOUBOETFSWFSNVTUCFUJHIUMZ WFSTJPONBUDIFE DMJFOUBOETFSWFSDBOFWPMWF JOEFQFOEFOUMZ
1Z4QBSL%BUB'SBNF 'VODUJPOTBOE$PMVNO 4DBMB%BUBTFU GVODUJPOT $PMVNO $BUBMPHBOE,FZ7BMVF(SPVQFE%BUBTFU 6TFS%FGJOFE'VODUJPOT 4USFBNJOH"1*T%BUB4USFBN3FBEFS %BUB4USFBN8SJUFS 4USFBNJOH2VFSZ
BOE 4USFBNJOH2VFSZ-JTUFOFS 4VQQPSUFE"1*T IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMXIBUJTTVQQPSUFE
1SJWBUF'JFMETPG%BUB'SBNF $PMVNO 4QBSL4FTTJPO FUD &YFDVUJPO"1*T3%% &OWJSPONFOU.BOJQVMBUJPO"1*T4QBSL$POUFYU 4QBSL$POGJHVSBUJPO
6OTVQQPSUFE"1*T IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMIPXTQBSLDPOOFDUDMJFOUBQQMJDBUJPOTEJGGFSGSPNDMBTTJD TQBSLBQQMJDBUJPO
&YUFOTJCJMJUZ 4QBSLBTB4FSWJDFJO$MPVE&OWJSPONFOU .VMUJMBOHVBHF$MJFOUT3VTU (P +BWBTDSJQU FUD
*OUFSBDUJWJUZ /PUFCPPLT+VQZUFS 74$PEF *OUFHSBUFE%FWFMPQNFOU&OWJSPONFOU .JOJNBMJUZ 8FC6*PS#*5PPMT .PCJMFPS.JDSPTFSWJDFT .$1TFSWFST 6TF$BTFT
&YUFOTJCJMJUZ 4QBSLBTB4FSWJDFJO$MPVE&OWJSPONFOU .VMUJMBOHVBHF$MJFOUT3VTU (P +BWBTDSJQU FUD
*OUFSBDUJWJUZ /PUFCPPLT+VQZUFS 74$PEF *OUFHSBUFE%FWFMPQNFOU&OWJSPONFOU .JOJNBMJUZ 8FC6*PS#*5PPMT .PCJMFPS.JDSPTFSWJDFT .$1TFSWFST 6TF$BTFT
QPXFSFECZ4QBSL$POOFDU 4QBSLBTB4FSWJDF
*OUFHSBUFXJUIJOIPVTFQMBUGPSNT "DDFTT$POUSPM $VTUPNJ[FTQBSLDMJFOUQFSVTFSTFTTJPO QZUIPO3TDBMBMJCSBSJFT DMJFOUESJWFSTQFDT$16
.FNPSZ (16 FUD $VTUPNJ[F4QBSLJUTFMG JOIPVTFQBUDIFT GJOFUVOFTQBSLQSPQFSUJFT SFNPUFTIVGGMFTFSWJDFT 3FRVJSFNFOUT
4QBSL$POOFDUWT"MUFSOBUJWFT TQBSLDPOOFDU MJWZ LZVVCJ BCTUSBDUJPO MPX NFEJVN IJHI QSPUPDPM H31$
3&45 +%#$0%#$ MBOHVBHF BOZ QZUIPO TDBMB KBWB 3 42- TQBSLWFSTJPO Yr Y TUBOEBMPOF Yr Y JOUFHSBUFE
4QBSL.VMUJUFOBODZ TQBSLDPOOFDU MJWZ LZVVCJ CVJMUJO ❌ /P ⚠ 1BSUJBM ✅
:FT 4FTTJPO *TPMBUJPO ✅ :FT ✅ :FT ✅ :FT 4FTTJPO .BOBHFNFOU ❌ /P ⚠ #BTJD ✅ 'VMM 6TFS "VUIFOUJDBUJPO ⚠ -JNJUFE ⚠ 1MVHHBCMF ✅ 4USPOH 3FTPVSDF 2VPUB ❌ /P ❌ /P ✅ :FT
4QBSL&YUFOTJCJMJUZ TQBSLDPOOFDU MJWZ LZVVCJ -BOHVBHF ✅ "OZ ⚠ +7. 1ZUIPO
3 42- ❌ 42- $VTUPN4QBSL ✅ :FT ⚠ -JNJUFE ❌ /P $VTUPN 4FTTJPO ✅ :FT ⚠ -JNJUFE ❌ /P $VTUPN6TFS "VUIFOUJDBUJPO ✅ :FT ⚠ -JNJUFE ❌ /P $VTUPN 3FTPVSDF2VPUB ✅ :FT ⚠ -JNJUFE ❌ /P
4FSWF4QBSL%SJWFSJOTUBODFXJUI4QBSL$POOFDU"EEPO 4VQQPSU4QBSL$SFEFOUJBMTLFSCFSPT LFZUBC 4VQQPSUSFTPVSDFRVPUBT 4VQQPSUSPMFNBOBHFNFOU
4VQQPSUBVEJUMPHHJOH )PSJ[POUBMMZ4DBMBCMF 3FRVJSFNFOU3VOUJNF
4FSWF4QBSL%SJWFSJOTUBODFXJUI4QBSL$POOFDU"EEPO 4VQQPSU4QBSL$SFEFOUJBMTLFSCFSPT LFZUBC 4VQQPSUSFTPVSDFRVPUBT 4VQQPSUSPMFNBOBHFNFOU
4VQQPSUBVEJUMPHHJOH )PSJ[POUBMMZ4DBMBCMF è,VCFSOFUFTXJUINVMUJQMFOBNFTQBDFT &OWPZ"%4 $%4 &%4 3FRVJSFNFOU3VOUJNF
8IJMF4QBSL$POOFDUEPFTOPUIBWFCVJMUJOBVUIFOUJDBUJPO JUJTEFTJHOFEUP XPSLTFBNMFTTMZXJUIZPVSFYJTUJOHBVUIFOUJDBUJPOJOGSBTUSVDUVSF*UTH31$ )551JOUFSGBDFBMMPXTGPSUIFVTFPGBVUIFOUJDBUJOHQSPYJFT XIJDINBLFTJU QPTTJCMFUPTFDVSF4QBSL$POOFDUXJUIPVUIBWJOHUPJNQMFNFOUBVUIFOUJDBUJPO MPHJDJO4QBSLEJSFDUMZ 3FRVJSFNFOU"VUI IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMXIBUJTTVQQPSUFE
8IJMF4QBSL$POOFDUEPFTOPUIBWFCVJMUJOBVUIFOUJDBUJPO JUJTEFTJHOFEUP XPSLTFBNMFTTMZXJUIZPVSFYJTUJOHBVUIFOUJDBUJPOJOGSBTUSVDUVSF*UTH31$ )551JOUFSGBDFBMMPXTGPSUIFVTFPGBVUIFOUJDBUJOHQSPYJFT XIJDINBLFTJU QPTTJCMFUPTFDVSF4QBSL$POOFDUXJUIPVUIBWJOHUPJNQMFNFOUBVUIFOUJDBUJPO MPHJDJO4QBSLEJSFDUMZ è&OWPZY%4 &YU"VUI[ GJMUFS
3FRVJSFNFOU"VUI IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMXIBUJTTVQQPSUFE
&OWPZY%4
4FMGNBOBHF4QBSL%SJWFSJOTUBODFT 4FMGNBOBHF$SFEFOUJBMUPBDDFTT4QBSL%SJWFSJOTUBODFT 3FRVJSFNFOU0QFSBUJPO
4FMGNBOBHF4QBSL%SJWFSJOTUBODFT 4FMGNBOBHF$SFEFOUJBMUPBDDFTT4QBSL%SJWFSJOTUBODFT è#BTJD8FCTJUF 3FRVJSFNFOU0QFSBUJPO
.BOBHFNFOU$POTPMF
)JCBOB"SDIJUFDUVSF
)JCBOB"SDIJUFDUVSF
)JCBOB"SDIJUFDUVSF
%&.04QBSLBTB4FSWJDF
)JCBOB
%&.04QBSL$MJFOU
+VQZUFS
*%&
.$1
3FDBQ
4QBSL$POOFDUJT BH31$CBTFEDMJFOUTFSWFSQSPUPDPMUIBUFOBCMFTSFNPUF JOUFSBDUJWFBDDFTTUP"QBDIF4QBSLVTJOHUIFOBUJWF%BUB'SBNF BOE42- "1*T 4QBSL$POOFDU
4QBSL$POOFDUJT BH31$CBTFEDMJFOUTFSWFSQSPUPDPMUIBUFOBCMFTSFNPUF JOUFSBDUJWFBDDFTTUP"QBDIF4QBSLVTJOHUIFOBUJWF%BUB'SBNF BOE42- "1*T 4QBSL$POOFDUDBO CFVTFEXJUIFYUFSOBM
KPCUSBDLFS KPCTDIFEVMFSBOE VTFSTFTTJPONBOBHFNFOU 4QBSL$POOFDU