Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Realtime Data Platform: from Lakehouse to Strea...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for LINEヤフーTech (LY Corporation Tech) LINEヤフーTech (LY Corporation Tech) PRO
June 30, 2025
Technology
230
0

Realtime Data Platform: from Lakehouse to Streamhouse

Avatar for LINEヤフーTech (LY Corporation Tech)

LINEヤフーTech (LY Corporation Tech) PRO

June 30, 2025

More Decks by LINEヤフーTech (LY Corporation Tech)

See All by LINEヤフーTech (LY Corporation Tech)
検索技術知識0のエンジニアが広告検索システムを内製化して運用するまで
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
300
No More Log Digging 〜AIを一次調査係にする アラートレビュー改善〜
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
110
実践！既存 Project への AI-Driven Development 適用〜一ヶ月で Project 唯一のフロントエンドエンジニアを作り出せ〜
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
450
AIが実装を自走する時代の認知負債との戦い
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
3
1.2k
AIと1000本ノックしてたどり着いた、最速のプロダクト開発 ～toC向けAIエージェントUXを、動く選択肢とAIキャパシティで設計する～ 
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
180
プライバシー保護の理論と実践
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
320
Yahoo!ショッピングでのAI-DLCの取り組みについて
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
2
790
Business ID Integration at LY Corporation : Phased Migration for a B2B authentication platform with Tens of Millions of Users
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
78
LINE Messaging: From Active-Standby to Active-Active Multi-DC Architecture
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
120

Other Decks in Technology

See All in Technology
AIにブラウザを触らせて、E2EテストをPlaywrightで書く
Avatar for koji_kawamura koji_kawamura
0
180
老害フォレンジッカーはAI羊の夢を見るか？
Avatar for tadmaddad tadmaddad
0
350
LanceDB入門
Avatar for Tomoko Uchida mocobeta
9
630
AI・HPC開発を支えるGPU環境の新しい選択肢 液冷GPUシステム「AquSys」の取り組み
Avatar for GPU UNITE gpuunite_official
0
130
MIXIで活躍できるエンジニアを 若手社員目線で考えてみる
Avatar for MIXI ENGINEERS mixi_engineers
PRO
0
100
属人化を叩き割れ！「制作加速」と「安定化」の矛盾を打破する：8年目プロダクトが辿り着いた「ミスが起こり得ない」アセット制作フローの全貌
Avatar for gree_tech gree_tech
PRO
0
120
システム思考で問題に対処する
Avatar for yussak yussak
0
360
生成 AI の基礎 〜 サンプル実装で学ぶ基本原理
Avatar for Etsuji Nakai enakai00
7
4.5k
FPGAが実現する遠方宇宙の高空間分解能天体撮影 -大型地上望遠鏡の視力を補正する「補償光学」とは？-
Avatar for 高橋光明 / Mitsuaki Takahashi komei_mt
0
570
Kiro入門｜仕様駆動開発で変わるAI時代の開発スタイル
Avatar for 工藤皓夢 cmkudo
0
230
Android skills に学ぶ
Avatar for kokiko kokiko
0
130
Digitization部 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
2
7.7k

Featured

See All Featured
Embracing the Ebb and Flow
Avatar for Simon Collison colly
88
5.1k
The SEO Collaboration Effect
Avatar for Kristina Bergwall kristinabergwall1
1
520
Practical Orchestrator
Avatar for Shlomi Noach shlominoach
191
12k
Building Experiences: Design Systems, User Experience, and Full Site Editing
Avatar for Michelle Schulp Hunt marktimemedia
0
580
My Coaching Mixtape
Avatar for mlcsv mlcsv
0
250
Google's AI Overviews - The New Search
Avatar for Barry Adams badams
0
1.1k
Building Adaptive Systems
Avatar for Chris Keathley keathley
44
3.2k
What's in a price? How to price your products and services
Avatar for Michael Herold michaelherold
247
13k
Building the Perfect Custom Keyboard
Avatar for Naoto Takai takai
2
840
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
270
14k
Building a A Zero-Code AI SEO Workflow
Avatar for Ian Lurie portentint
PRO
0
680
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k

Transcript

  1. 3FBMUJNF%BUB1MBUGPSN GSPN-BLFIPVTFUP4USFBNIPVTF -:$PSQPSBUJPO 9JBP;IJZBO

  2. 🎯 0QFOSFTFBSDI PO3FBMUJNF%BUB1MBUGPSN &YQMPSFUIFMBUFTUUFDIOPMPHJFTBOEBSDIJUFDUVSFTMJLF4USFBNIPVTF 🔒 /PU SFQSFTFOUBUJWFPGJOIPVTFQSPKFDUT BU-:$PSQPSBUJPO 4DPQF PGUIF4FTTJPO

    8IBUXFXJMMDPWFSUPEBZ

  3. ঻ ࢤ඙ 9JBP;IJZBO γϠ΢ ζ΢ΠΤϯ 🏠 தࠃॏܚ $IPOHRJOH $IJOB ","l%.BHJD$JUZz

    🎓 ߳ߓதจେֶ 5IF$IJOFTF6OJWFSTJUZPG)POH,POH .BUI*OGPSNBUJPO&OHJOFFSJOH 👨💻 4PGUXBSF&OHJOFFS!-:$PSQPSBUJPO %BUB&OHJOFFSJOH%BUB1JQFMJOF&OHJOFFSJOH*OHFTUJPO1SPDFTTJOH ✨ *NQSPWJOHCJHEBUB TZTUFNTXJUI3VTU 0QFO%BUB%SJWFODPNNVOJUZ 4USFBNIPVTF "*OBUJWF 3VTU BOENPSF "CPVU.F !9JBP;IJZBO

  4. 💻 4PGUXBSFJTFBUJOHUIFXPSME &WFSZCVTJOFTTCFDBNFBTPGUXBSFDPNQBOZ 🤖 "*JTFBUJOHTPGUXBSF 🚀 3FBMUJNFJTFBUJOHFWFSZUIJOHFMTF -JWFEBTICPBSET 3FBMUJNFBMFSUT 0OUIFGMZQFSTPOBMJ[BUJPO

    *OTUBOUGFFECBDLGSPN"*BHFOUT 🐢 .PTU%BUB1MBUGPSNTà /PUGVMMZTVQQPSUJOHSFBMUJNFTDFOBSJPT 🐇 3FBMUJNF%BUB1MBUGPSNà 4USFBNIPVTF 8IZ3FBMUJNF %BUB1MBUGPSN "MBSHFTIJGUIBQQFOJOHBSPVOEVT 🌏 ˠ💻 ˠ🤖 ˠ🚀

  5. %BUB8BSFIPVTFBOE%BUB-BLF 5IFFBSMZEBZTPG%BUB1MBUGPSNT 4USVDUVSFE%BUB 4FNJTUSVDUVSFE%BUB 6OTUSVDUVSFE%BUB &5- &5- %BUB8BSFIPVTF #*3FQPSUT %BUB-BLF

    )%'44 2VFSZ&OHJOF 4QBSL .- %BUB8BSFIPVTF %BUB-BLF %BUB5ZQF 4USVDUVSFE ✅ 4USVDUVSFE TFNJTUSVDUVSFE VOTUSVDUVSFE 4DBMBCJMJUZ %JGGJDVMUBOEFYQFOTJWF ✅ &BTZBUBMPXDPTU 5SBOTBDUJPO ✅ 4VQQPSUFE /PUTVQQPSUFE 2VFSZ1FSGPSNBODF ✅ )JHI -PX 🤔 $BOXFDPNCJOF UIFBEWBOUBHFT PGCPUITZTUFNTJOUPPOF

  6. -BLFIPVTF %BUB-BLF %BUB8BSFIPVTF 4USVDUVSFE%BUB 4FNJTUSVDUVSFE%BUB 6OTUSVDUVSFE%BUB &5- -BLFIPVTF5BCMF'PSNBU *DFCFSH #*3FQPSUT

    %BUB-BLF )%'44 2VFSZ&OHJOF 4QBSL .- -BLFIPVTF ,FZ1PJOU *OEFYEBUBGJMFTXJUITUBUTUPTQFFEVQRVFSJFT 5BCMF'PSNBUT "QBDIF*DFCFSH %FMUB-BLF "QBDIF)VEJ 1SPT ✅ 5SBOTBDUJPO IJHIRVFSZQFSGPSNBODF $POT 🚫 )JHIMBUFODZ NJOVUFT MPXUISPVHIQVUPGTUSFBNJOHXSJUF 🤔 )PXUPTVQQPSUMPXMBUFODZTUSFBNJOHQSPDFTTJOH

  7. -BNCEB "SDIJUFDUVSF #BUDI1JQFMJOF 4USFBNJOH1JQFMJOF %BUB4PVSDFT #BUDI1JQFMJOF 4QBSL -BLFIPVTF4UPSBHF *DFCFSH 2VFSZ&OHJOF

    %BUB6TFST 4USFBNJOH1JQFMJOF 'MJOL 4USFBNJOH4UPSBHF ,BGLB -BNCEB"SDIJUFDUVSF ,FZ1PJOU %VBMMBZFSEFTJHO CBUDI TUSFBNJOH 6TBHF .FSHFSFTVMUTPGCPUIMBZFSTBURVFSZTJEF 1SPT ✅ -PXMBUFODZTUSFBNJOHQSPDFTTJOH TVCTFDPOE $POT 🚫 $PNQMFYJUZ IJHIDPTU EFMBZJOIJTUPSJDBMEBUB NJOVUFT 🤔 *TJUGFBTJCMFUPVOJGZ UIFTFUXPEBUBQJQFMJOFT

  8. ,BQQB "SDIJUFDUVSF 6OJGZ#BUDIBOE4USFBNJOH1JQFMJOFT ˠ ,BGLB4PVSDFT 4USFBNJOH1JQFMJOF 'MJOL 4USFBNJOH4UPSBHF ,BGLB 2VFSZ&OHJOF

    %BUB6TFST ,BQQB"SDIJUFDUVSF ,FZ1PJOU "MMEBUBBTTUSFBNT IJTUPSJDBMEBUBCZSFQMBZJOH 1SPT ✅ 4JNQMJDJUZ MPXMBUFODZ TVCTFDPOE $POT 🚫 &YQFOTJWFSFQMBZJOH JOFGGJDJFOURVFSJFT 🤔 $BOXFJNQSPWF,BQQB "SDIJUFDUVSFXJUI-BLFIPVTF 4UPSBHF

  9. 6OJGJFE/FBS3FBMUJNF1JQFMJOF ,BGLBˠ1BJNPO %BUB4PVSDFT 6OJGJFE1JQFMJOF 'MJOL 2VFSZ&OHJOF %BUB6TFST 🤔 )PXEPFT1BJNPO TVQQPSUCPUI4USFBNJOH

    BOE-BLFIPVTF GFBUVSFT -BLFIPVTF4UPSBHF *DFCFSH NJOVUFT 4USFBNJOH4UPSBHF ,BGLB TVCTFDPOE 4USFBNJOHGJSTU-BLFIPVTF4UPSBHF 1BJNPO TVCNJOVUF

  10. "-JUUMF.PSFPO1BJNPO 4USFBNJOHGJSTU-BLFIPVTF5BCMF'PSNBU 🤔 $BOXFGVSUIFSJNQSPWF6OJGJFE1JQFMJOFUPBDIJFWFTVCTFDPOEMBUFODZ IUUQTQBJNPOBQBDIFPSHEPDTNBTUFSDPODFQUTCBTJDDPODFQUT *OEFYEBUBGJMFTXJUITUBUT TJNJMBSUP*DFCFSH 8SJUFUPNVMUJQMFCVDLFUT JOQBSBMMFM 3FBEDIBOHFMPHPG1,UBCMFT

    4USFBNJOHSFBE"QQFOEUBCMFT

  11. 6OJGJFE3FBMUJNF1JQFMJOF 1BJNPO 'MVTT %BUB4PVSDFT 6OJGJFE1JQFMJOF -BLFIPVTF4UPSBHF 1BJNPO TVCNJOVUF 2VFSZ&OHJOF %BUB6TFST

    4USFBNJOH4UPSBHF 'MVTT TVCTFDPOE ,BGLB 'MVTT %BUB'PSNBU #JOBSZ SPXCBTFE ✅ "QBDIF"SSPX DPMVNOCBTFE 4DIFNB &YUFSOBMSFHJTUSZ ✅ *OUFSOBMTDIFNB 1SPKFDUJPO1VTIEPXO /PUTVQQPSUFE ✅ 4VQQPSUFE 2VFSZ*OUFSGBDF $POTVNFS3&45 ✅ "SSPX'MJHIU IJHIQFSGPSNBODF 📝 -FU`TTVNNBSJ[F XIBUXFIBWFEJTDVTTFETPGBS

  12. 4USFBNIPVTF 6OJGJFE3FBMUJNF1JQFMJOF %BUB8BSFIPVTF %BUB-BLF -BNCEB -BLFIPVTF ,BQQB 1BJNPO 'MVTT

  13. ✨ 8FIBWFTFFOMBUFTU4USFBNIPVTF "SDIJUFDUVSFXJUI1BJNPOBOE'MVTT 🤔 "SFUIFSFBOZPUIFSQPUFOUJBMJNQSPWFNFOUTBT3FBMUJNF%BUB1MBUGPSN )PXUPPSDIFTUSBUJPOPGEBUBQJQFMJOFT )PXUPJNQSPWFJU 1PUFOUJBM*NQSPWFNFOUT

  14. %ZOBNJD %BUB'MPX 4UBUJD%BUB1JQFMJOFTˠ-JWFMZ$PNQPTBCMF%BUB'MPXT %BUB4PVSDFT *OHFTU 0%4 $MFBO %8% "HHSFHBUF %84

    $VTUPNJ[F "%4 %BUB'MPX :".- 3FVTBCMF%BUB"DUJPOT 42- 'MPX"DUJPO3FHJTUSZ 'MPX.BUFSJBMJ[FS 'MPX5SJHHFS

  15. "*OBUJWF $POUSPM1MBOF "VUPNBUFEˠ"VUPOPNPVT %BUB"DUJPO %BUB'MPX 'MPX"DUJPO3FHJTUSZ 'MPX5SJHHFS 'MPX1MBOOFS "DUJPO$PNQPTFS "HFOUJD"*

    6TFS1SPNQU 0SDIFTUSBUPS 'MPX.BUFSJBMJ[FS

  16. "*OBUJWF4USFBNIPVTF /FYUHFOFSBUJPO3FBMUJNF%BUB1MBUGPSN "*OBUJWF$POUSPM1MBOF %ZOBNJD%BUB'MPX 6OJGJFE3FBMUJNF1JQFMJOF

  17. ✨ 4USFBNIPVTF "SDIJUFDUVSFJTTUJMMRVJUFOFX 🧭 -FU`TTFFIPXJUDBOCSJOHSFBMWBMVF UPPVSCVTJOFTT 🚀 &YQMPSF CVJME BOETIBSF

    4USFBNIPVTFJNQMFNFOUBUJPOTUPHFUIFS 5IF3PBE"IFBE

  18.  -BNCEB"SDIJUFDUVSF IUUQOBUIBONBS[DPNCMPHIPXUPCFBUUIFDBQUIFPSFNIUNM  ,BQQB"SDIJUFDUVSF IUUQTXXXPSFJMMZDPNSBEBSRVFTUJPOJOHUIFMBNCEBBSDIJUFDUVSF  -BLFIPVTF IUUQTXXXEBUBCSJDLTDPNCMPHXIBUJTBEBUBMBLFIPVTFIUNM 

    4USFBNIPVTF IUUQTXXXWFSWFSJDBDPNCMPHTUSFBNIPVTFVOWFJMFE  "*/BUJWF IUUQTXXXTQMVOLDPNFO@VTCMPHMFBSOBJOBUJWFIUNM 3FMBUFE-JOLT

  19. '*/