Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Introduction to "Spark Connect" and Implementat...
Search
LINEヤフーTech (LY Corporation Tech)
PRO
June 30, 2025
Technology
0
67
Introduction to "Spark Connect" and Implementation of Managed Spark as a Service
「Spark Connect」プロトコルについて紹介し、Spark Connectを活用した社内管理型Spark as a Serviceの実装アーキテクチャおよび使用例を共有します。
LINEヤフーTech (LY Corporation Tech)
PRO
June 30, 2025
Tweet
Share
More Decks by LINEヤフーTech (LY Corporation Tech)
See All by LINEヤフーTech (LY Corporation Tech)
マルチモーダル基盤モデルに基づく動画と音の解析技術
lycorptech_jp
PRO
2
260
AI駆動開発 with MixLeap Study【大阪支部 #3】
lycorptech_jp
PRO
0
270
Yahoo!しごとカタログ 新しい境地を創るエンジニア募集!
lycorptech_jp
PRO
3
320
データグループにおけるフロントエンド開発
lycorptech_jp
PRO
2
280
Yahoo!知恵袋におけるフロントエンド開発
lycorptech_jp
PRO
0
270
"LINE Planet" and AI: Conversations with AI
lycorptech_jp
PRO
0
58
Seamless inventory management with AI
lycorptech_jp
PRO
0
34
AI Frontiers Revealed: Transforming LINE Shopping TW with LLM-Driven Product Attribute Extraction
lycorptech_jp
PRO
0
56
LINEヤフーの音声AIがもたらす未来:ASR/TTSと対話技術の新たな可能性 / LY Corporation's Speech AI Vision: Towards Realtime Spoken Dialogue through Advanced ASR and TTS
lycorptech_jp
PRO
0
63
Other Decks in Technology
See All in Technology
DatabricksのOLTPデータベース『Lakebase』に詳しくなろう!
inoutk
0
160
モバイルゲームの開発を支える基盤の歩み ~再現性のある開発ラインを量産する秘訣~
qualiarts
0
610
LLM開発を支えるエヌビディアの生成AIエコシステム
acceleratedmu3n
0
330
大規模組織にAIエージェントを迅速に導入するためのセキュリティの勘所 / AI agents for large-scale organizations
i35_267
6
330
データエンジニアがクラシルでやりたいことの現在地
gappy50
3
660
Railsの限界を超えろ!「家族アルバム みてね」の画像・動画の大規模アップロードを支えるアーキテクチャの変遷
ojima_h
4
520
今日からあなたもGeminiを好きになる
subaruhello
1
660
生成AIを活用した野球データ分析 - メジャーリーグ編 / Baseball Analytics for Gen AI
shinyorke
PRO
1
230
隙間時間で爆速開発! Claude Code × Vibe Coding で作るマニュアル自動生成サービス
akitomonam
2
200
大規模イベントを支える ABEMA の アーキテクチャ 変遷 2025
nagapad
5
510
ObsidianをLLM時代のナレッジベースに! クリッピング→Markdown→CLI連携の実践
srvhat09
7
9.8k
Recoil脱却の現状と挑戦
kirik
3
460
Featured
See All Featured
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Building Adaptive Systems
keathley
43
2.7k
Making the Leap to Tech Lead
cromwellryan
134
9.4k
RailsConf 2023
tenderlove
30
1.2k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
390
Into the Great Unknown - MozCon
thekraken
40
1.9k
A Tale of Four Properties
chriscoyier
160
23k
How to Ace a Technical Interview
jacobian
278
23k
Designing Experiences People Love
moore
142
24k
KATA
mclloyd
30
14k
Transcript
*OUSPEVDUJPOUP4QBSL$POOFDUBOE *NQMFNFOUBUJPOPG.BOBHFE4QBSLBTB 4FSWJDF %BUB1MBUGPSN&OHJOFFSJOH .BSL-FF 이승 李承
"QBDIF4QBSL$POOFDU "QBDIF4QBSLBTB4FSWJDF %&.04QBSLBTB4FSWJDF %&.04QBSL$MJFOU 3FDBQ
"HFOEB
4QBSL$POOFDU
"QBDIF4QBSLä JTBNVMUJMBOHVBHFFOHJOFGPSFYFDVUJOHEBUBFOHJOFFSJOH EBUB TDJFODF BOENBDIJOFMFBSOJOHPOTJOHMFOPEFNBDIJOFTPSDMVTUFST ,FZ'FBUVSFT #BUDITUSFBNJOHEBUB 42-BOBMZUJDT
%BUBTDJFODFBUTDBMF .BDIJOFMFBSOJOH "QBDIF4QBSL IUUQTTQBSLBQBDIFPSH
4QBSL$MVTUFS IUUQTTQBSLBQBDIFPSHEPDTMBUFTUDMVTUFSPWFSWJFXIUNM
$IBMMFOHFTPGUSBEJUJPOBM4QBSL Single Point of Failure - Limited Fault Tolerance -
Resource Bottlenecks Network Dependency - Network partitions or instability disrupts execution - Clients Need Direct Access - Scheduling Restrictions Difficult Multi-Tenancy - Cluster endpoint must be protected 5JHIUDPVQMJOHPG$MJFOUBOE%SJWFS
3FNPUFDPOOFDUJWJUZUP4QBSLDMVTUFST CZ%BUB'SBNF "1*BOEVOSFTPMWFEMPHJDBMQMBOT WJBH31$ XJUI"QBDIF"SSPXFODPEFESPXCBUDIFT $BOCFFNCFEEFEFWFSZXIFSF BQQMJDBUJPOTFSWFST
*%&T OPUFCPPLTBOEQSPHSBNNJOHMBOHVBHFT "CPVU4QBSL$POOFDU %FDPVQMFMBZFSCFUXFFO$MJFOUBOE%SJWFS
4QBSL$POOFDUBSDIJUFDUVSF IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNM
4QBSL$POOFDUBSDIJUFDUVSF IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNM
4QBSL$POOFDU8PSLGMPX IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNM
4QBSLWT4QBSL$POOFDU TQBSL TQBSLDPOOFDU DMJFOUMPDBUJPO JODMVTUFS PVUDMVTUFS DMJFOUSFRVJSFNFOU GVMMTQBSL DMVTUFSBDDFTTBOE +7.CBTFE
UIJODMJFOUNJOJNBMH31$ DMJFOUXJUI%BUB'SBNF -JCSBSZ OFUXPSLUPQPMPHZ NVTUIBWFEJSFDUOFUXPSL BDDFTTUPUIFDMVTUFS SFNPUFBDDFTTWJBH31$ TDBMBCJMJUZ ESJWFS DMJFOUIBOEMFTBMM TDIFEVMJOHBOEDPPSEJOBUJPO ESJWFSSVOTPODMVTUFSDMJFOUT BSFTUBUFMFTT GBVMUUPMFSBODF DMJFOUDSBTIDBVTFTKPCMPTU DMJFOUDBOEJTDPOOFDUBOE SFDPOOFDU
4QBSLWT4QBSL$POOFDU DPOU TQBSL TQBSLDPOOFDU MBOHVBHF MJNJUFEUPTQBSLTVQQPSUFE MBOHVBHFT+7. 1Z4QBSL 3 BOZ
FY KBWBTDSJQU HP D SVTU DMPVE OBUJWF UJHIUDPVQMJOHBOEESJWFS SFRVJSFNFOUTMJNJUGMFYJCJMJUZ EFTJHOFEGPSDMPVEOBUJWF DPOUBJOFSJ[FEBOETFSWFSMFTT TFDVSJUZ FYQPTJOHDMVTUFSFOEQPJOUTDBO SBJTFTFDVSJUZDPODFSOT DBOTFDVSFH31$ DIBOOFMXJUI DFOUSBMJ[FEDPOUSPM WFSTJPOJOH DMJFOUBOETFSWFSNVTUCFUJHIUMZ WFSTJPONBUDIFE DMJFOUBOETFSWFSDBOFWPMWF JOEFQFOEFOUMZ
1Z4QBSL%BUB'SBNF 'VODUJPOTBOE$PMVNO 4DBMB%BUBTFU GVODUJPOT $PMVNO $BUBMPHBOE,FZ7BMVF(SPVQFE%BUBTFU 6TFS%FGJOFE'VODUJPOT 4USFBNJOH"1*T%BUB4USFBN3FBEFS %BUB4USFBN8SJUFS 4USFBNJOH2VFSZ
BOE 4USFBNJOH2VFSZ-JTUFOFS 4VQQPSUFE"1*T IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMXIBUJTTVQQPSUFE
1SJWBUF'JFMETPG%BUB'SBNF $PMVNO 4QBSL4FTTJPO FUD &YFDVUJPO"1*T3%% &OWJSPONFOU.BOJQVMBUJPO"1*T4QBSL$POUFYU 4QBSL$POGJHVSBUJPO
6OTVQQPSUFE"1*T IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMIPXTQBSLDPOOFDUDMJFOUBQQMJDBUJPOTEJGGFSGSPNDMBTTJD TQBSLBQQMJDBUJPO
&YUFOTJCJMJUZ 4QBSLBTB4FSWJDFJO$MPVE&OWJSPONFOU .VMUJMBOHVBHF$MJFOUT3VTU (P +BWBTDSJQU FUD
*OUFSBDUJWJUZ /PUFCPPLT+VQZUFS 74$PEF *OUFHSBUFE%FWFMPQNFOU&OWJSPONFOU .JOJNBMJUZ 8FC6*PS#*5PPMT .PCJMFPS.JDSPTFSWJDFT .$1TFSWFST 6TF$BTFT
&YUFOTJCJMJUZ 4QBSLBTB4FSWJDFJO$MPVE&OWJSPONFOU .VMUJMBOHVBHF$MJFOUT3VTU (P +BWBTDSJQU FUD
*OUFSBDUJWJUZ /PUFCPPLT+VQZUFS 74$PEF *OUFHSBUFE%FWFMPQNFOU&OWJSPONFOU .JOJNBMJUZ 8FC6*PS#*5PPMT .PCJMFPS.JDSPTFSWJDFT .$1TFSWFST 6TF$BTFT
QPXFSFECZ4QBSL$POOFDU 4QBSLBTB4FSWJDF
*OUFHSBUFXJUIJOIPVTFQMBUGPSNT "DDFTT$POUSPM $VTUPNJ[FTQBSLDMJFOUQFSVTFSTFTTJPO QZUIPO3TDBMBMJCSBSJFT DMJFOUESJWFSTQFDT$16
.FNPSZ (16 FUD $VTUPNJ[F4QBSLJUTFMG JOIPVTFQBUDIFT GJOFUVOFTQBSLQSPQFSUJFT SFNPUFTIVGGMFTFSWJDFT 3FRVJSFNFOUT
4QBSL$POOFDUWT"MUFSOBUJWFT TQBSLDPOOFDU MJWZ LZVVCJ BCTUSBDUJPO MPX NFEJVN IJHI QSPUPDPM H31$
3&45 +%#$0%#$ MBOHVBHF BOZ QZUIPO TDBMB KBWB 3 42- TQBSLWFSTJPO Yr Y TUBOEBMPOF Yr Y JOUFHSBUFE
4QBSL.VMUJUFOBODZ TQBSLDPOOFDU MJWZ LZVVCJ CVJMUJO ❌ /P ⚠ 1BSUJBM ✅
:FT 4FTTJPO *TPMBUJPO ✅ :FT ✅ :FT ✅ :FT 4FTTJPO .BOBHFNFOU ❌ /P ⚠ #BTJD ✅ 'VMM 6TFS "VUIFOUJDBUJPO ⚠ -JNJUFE ⚠ 1MVHHBCMF ✅ 4USPOH 3FTPVSDF 2VPUB ❌ /P ❌ /P ✅ :FT
4QBSL&YUFOTJCJMJUZ TQBSLDPOOFDU MJWZ LZVVCJ -BOHVBHF ✅ "OZ ⚠ +7. 1ZUIPO
3 42- ❌ 42- $VTUPN4QBSL ✅ :FT ⚠ -JNJUFE ❌ /P $VTUPN 4FTTJPO ✅ :FT ⚠ -JNJUFE ❌ /P $VTUPN6TFS "VUIFOUJDBUJPO ✅ :FT ⚠ -JNJUFE ❌ /P $VTUPN 3FTPVSDF2VPUB ✅ :FT ⚠ -JNJUFE ❌ /P
4FSWF4QBSL%SJWFSJOTUBODFXJUI4QBSL$POOFDU"EEPO 4VQQPSU4QBSL$SFEFOUJBMTLFSCFSPT LFZUBC 4VQQPSUSFTPVSDFRVPUBT 4VQQPSUSPMFNBOBHFNFOU
4VQQPSUBVEJUMPHHJOH )PSJ[POUBMMZ4DBMBCMF 3FRVJSFNFOU3VOUJNF
4FSWF4QBSL%SJWFSJOTUBODFXJUI4QBSL$POOFDU"EEPO 4VQQPSU4QBSL$SFEFOUJBMTLFSCFSPT LFZUBC 4VQQPSUSFTPVSDFRVPUBT 4VQQPSUSPMFNBOBHFNFOU
4VQQPSUBVEJUMPHHJOH )PSJ[POUBMMZ4DBMBCMF è,VCFSOFUFTXJUINVMUJQMFOBNFTQBDFT &OWPZ"%4 $%4 &%4 3FRVJSFNFOU3VOUJNF
8IJMF4QBSL$POOFDUEPFTOPUIBWFCVJMUJOBVUIFOUJDBUJPO JUJTEFTJHOFEUP XPSLTFBNMFTTMZXJUIZPVSFYJTUJOHBVUIFOUJDBUJPOJOGSBTUSVDUVSF*UTH31$ )551JOUFSGBDFBMMPXTGPSUIFVTFPGBVUIFOUJDBUJOHQSPYJFT XIJDINBLFTJU QPTTJCMFUPTFDVSF4QBSL$POOFDUXJUIPVUIBWJOHUPJNQMFNFOUBVUIFOUJDBUJPO MPHJDJO4QBSLEJSFDUMZ 3FRVJSFNFOU"VUI IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMXIBUJTTVQQPSUFE
8IJMF4QBSL$POOFDUEPFTOPUIBWFCVJMUJOBVUIFOUJDBUJPO JUJTEFTJHOFEUP XPSLTFBNMFTTMZXJUIZPVSFYJTUJOHBVUIFOUJDBUJPOJOGSBTUSVDUVSF*UTH31$ )551JOUFSGBDFBMMPXTGPSUIFVTFPGBVUIFOUJDBUJOHQSPYJFT XIJDINBLFTJU QPTTJCMFUPTFDVSF4QBSL$POOFDUXJUIPVUIBWJOHUPJNQMFNFOUBVUIFOUJDBUJPO MPHJDJO4QBSLEJSFDUMZ è&OWPZY%4 &YU"VUI[ GJMUFS
3FRVJSFNFOU"VUI IUUQTTQBSLBQBDIFPSHEPDTMBUFTUTQBSLDPOOFDUPWFSWJFXIUNMXIBUJTTVQQPSUFE
&OWPZY%4
4FMGNBOBHF4QBSL%SJWFSJOTUBODFT 4FMGNBOBHF$SFEFOUJBMUPBDDFTT4QBSL%SJWFSJOTUBODFT 3FRVJSFNFOU0QFSBUJPO
4FMGNBOBHF4QBSL%SJWFSJOTUBODFT 4FMGNBOBHF$SFEFOUJBMUPBDDFTT4QBSL%SJWFSJOTUBODFT è#BTJD8FCTJUF 3FRVJSFNFOU0QFSBUJPO
.BOBHFNFOU$POTPMF
)JCBOB"SDIJUFDUVSF
)JCBOB"SDIJUFDUVSF
)JCBOB"SDIJUFDUVSF
%&.04QBSLBTB4FSWJDF
)JCBOB
%&.04QBSL$MJFOU
+VQZUFS
*%&
.$1
3FDBQ
4QBSL$POOFDUJT BH31$CBTFEDMJFOUTFSWFSQSPUPDPMUIBUFOBCMFTSFNPUF JOUFSBDUJWFBDDFTTUP"QBDIF4QBSLVTJOHUIFOBUJWF%BUB'SBNF BOE42- "1*T 4QBSL$POOFDU
4QBSL$POOFDUJT BH31$CBTFEDMJFOUTFSWFSQSPUPDPMUIBUFOBCMFTSFNPUF JOUFSBDUJWFBDDFTTUP"QBDIF4QBSLVTJOHUIFOBUJWF%BUB'SBNF BOE42- "1*T 4QBSL$POOFDUDBO CFVTFEXJUIFYUFSOBM
KPCUSBDLFS KPCTDIFEVMFSBOE VTFSTFTTJPONBOBHFNFOU 4QBSL$POOFDU