Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
【CVPR 2024】VTimeLLM: Empower LLM to Grasp Video...
Search
mori yuichiro
July 09, 2024
60
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
【CVPR 2024】VTimeLLM: Empower LLM to Grasp Video Moments 論文読み会発表資料
mori yuichiro
July 09, 2024
More Decks by mori yuichiro
See All by mori yuichiro
【EMNLP 2023】Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs 論文読み会発表資料
zaemon1251hesty
0
120
Featured
See All Featured
Organizational Design Perspectives: An Ontology of Organizational Design Elements
kimpetersen
PRO
1
720
How STYLIGHT went responsive
nonsquared
100
6.2k
Designing Powerful Visuals for Engaging Learning
tmiket
1
410
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
10
1.2k
The Language of Interfaces
destraynor
162
27k
The browser strikes back
jonoalderson
0
1.2k
Building a A Zero-Code AI SEO Workflow
portentint
PRO
0
570
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
techseoconnect
PRO
0
200
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.7k
What’s in a name? Adding method to the madness
productmarketing
PRO
24
4.1k
The untapped power of vector embeddings
frankvandijk
2
1.8k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
55k
Transcript
75JNF--.&NQPXFS--.UP (SBTQ7JEFP.PNFOUT #JO)VBOH 9JO8BOH )POH$IFO ;JIBO4POH 8FOXV;IV 5TJOHIVB6OJWFSTJUZ $713 :VJDIJSP.PSJ
1BQFS3FBEJOH 1
4VNNBSZ • -BSHFMBOHVBHFNPEFMT --.T IBWFCFFOBEBQUFEUPIBOEMFWJEFP EBUBCVUTUSVHHMFXJUIQSFDJTFFWFOUUJNJOH • 75JNF--. BOFX7JEFP--. BEESFTTFTUIJTCZVTJOHBUISFF
TUBHFUSBJOJOHTUSBUFHZUPJNQSPWFUFNQPSBMVOEFSTUBOEJOH • &YQFSJNFOUTTIPX75JNF--.FYDFMTJOGJOFHSBJOFEWJEFP DPNQSFIFOTJPOBOEPVUQFSGPSNTFYJTUJOH7JEFP--.TJOWBSJPVT UBTLT 2
#BDLHSPVOE • 7JEFP--. • VUJMJ[JOH-BSHF-BOHVBHF.PEFMUPVOEFSTUBOEWJEFP • BOBHFOUGPSJOUFSBDUJOHXJUIIVNBOCBTFEPOBWJEFP • FH
7JEFP--B." <;IBOH &./-1> 7JEFP$IBU <-J BS9JW> 3 7JEFP--B."
1SPCMFN • &YJTUJOH7JEFP--.TGPDVTPOHFOFSBUJPOPGHFOFSJDWJEFP DBQUJPOT UIVTGBJMUPDBQUVSFUIFSFMBUJPOTIJQTCFUXFFOTQFDJGJD CPVOEFEFWFOUT 4 7JEFP$IBU 7JEFP--B."DBOOPU
MPDBMJ[FUIFFWFOU UJNJOH 7JEFP--.TBSF GBSGSPNQSFDJTFMZ VOEFSTUBOEJOH WJEFP
3FTFBSDI0CKFDUJWF 5 5IFZCVJME75JNF--.5IF'JSTUCPVOEBSZBXBSF7JEFP--. 5BDLMJOHUXPDIBMMFOHFTGPS7JEFP--.TUPDBQUVSFFWFOUCPVOEBSJFTJO WJEFP 4DBSDJUZPGMBSHFTDBMFCPVOEBSZBXBSFWJEFPEBUBTFUT Ø 5IFZDSFBUFEBUBTFUTJODPSQPSBUJOHFYJTUJOHUFNQPSBMSFMBUFE 7JEFP6OEFSTUBOEJOHUBTLT
%FTJHOFGGFDUJWFUFNQPSBMSFMBUFEWJEFPUBTLTGPSUSBJOJOH7JEFP--. Ø 5IFZQSPQPTF#PVOEBSZBXBSF5SBJOJOH4USBUFHZ
1SPQPTFE.FUIPE • 1SFMJNJOBSZ5XPUFNQPSBMSFMBUFE7JEFP6OEFSTUBOEJOHUBTLT 6 5FNQPSBM7JEFP (SPVOEJOH 57( %FOTF7JEFP $BQUJPOJOH %7$
• UFNQPSBMMPDBMJ[BUJPODPSSFTQPOEJOHWJEFP TFHNFOUTGPSHJWFOUFYUVBMJOQVUT • UFNQPSBMMPDBMJ[BUJPOBOEDBQUJPOJOH GPSBMMFWFOUT 5FNQPSBMMZ(SPVOEJOH/BUVSBM4FOUFODFJO7JEFP <+JOHZVBO &./-1> %FOTF$BQUJPOJOHJOFWFOUT <,SJTIOB *$$7>
1SPQPTFE.FUIPE • #PVOEBSZBXBSF5SBJOJOH4USBUFHZ DPOTJTUT5SBJOJOH4UBHFT 7
1SPQPTFE.FUIPE • 4UBHF'FBUVSF"MJHONFOU • "JNTUPUSBJOUIFWJTVBMBEBQUFS UPBMJHOWJEFP GFBUVSFTXJUI--.`TTFNBOUJDTQBDF • %BUBTFU-$4, JNBHFUFYUQBJSVTFEGPS
QSFUSBJOJOH--B7"<-JV /FVS*14`> • /PUF • 4JODFDPOUFNQPSBSZMBSHFTDBMFWJEFPUFYUEBUBTFUTDPOUBJO TVCTUBOUJBMUFYUVBMOPJTF UIFZVTFUIFJNBHFUFYUEBUBTFU 8
1SPQPTFE.FUIPE • 4UBHF#PVOEBSZ1FSDFQUJPO • "JNTUPGBDJMJUBUJOHUIFVOEFSTUBOEJOHPG WBSJPVTFWFOUTPDDVSSJOHXJUIJOUIFWJEFP • %BUBTFUTJOHMFUVSOBOENVMUJUVSO RVFTUJPOBOTXFSJOH 2"
• GSPN*OUFSO7JE.'-5<8BOH *$-3> CPVOEBSZBXBSFEBUBTFU • *O*OUFSO7JE.'-5 1Z4DFOF%FUFDU BVUPNBUJDBMMZTQMJUVOUSJNNFEWJEFPJOUPNVMUJQMF DMJQT 9
1SPQPTFE.FUIPE • 4UBHF*OTUSVDUJPO5VOJOH • "JNTUPBMJHOXJUIIVNBOJOUFOU • %JBMPHVFNBEFGSPNIJHIRVBMJUZUFNQPSBM SFMBUFEEBUBTFUTFOBCMF75JNF--.NPSF QSFDJTFFWFOUMPDBMJ[BUJPOBOEEFTDSJQUJPO •
%BUBTFU--.HFOFSBUFEWJEFPCBTFE EJBMPHVFTGSPN"DUJWJUZ/FU$BQUJPO <,SJTIOB *$$7> %7$ BOE%J%F.P <)FOESJDLT *$$7> 75( 10 Trained on Stage 2
1SPQPTFE.FUIPE • 1SPNQUTPGUSBJOJOH4UBHF 11 WJEFPGFBUVSF FNCFEEJOHMJTU XJMMCFJOTFSUFEIFSF TZTUFN QSPNQU
1SPQPTFE.FUIPE • 1SPNQUTPGUSBJOJOH4UBHF 12 WJEFPGFBUVSF FNCFEEJOHMJTU XJMMCFJOTFSUFEIFSF TZTUFN QSPNQU Q1
A1 Q2 A2 Q3 A3 Q4 A4 #PUI2BOE"BSF HFOFSBUFECZ--.
&YQFSJNFOUT • *NQMFNFOUBUJPO • --.7JDVOBW # # <;IFOH /FVS*14> •
-PTTGVODUJPOPSJHJOBMBVUPSFHSFTTJWFUSBJOJOHPCKFDUJWFTPGUIF--. DSPTT FOUSPQZ • *OTFDPOEBOEUIJSETUBHFT UIFZPOMZUSBJO-P3"QBSBNFUFST XJUIGSFF[JOH QSFUSBJOFEQBSBNFUFST • 5SBJOJOH359(16 • #BTFMJOFT • &YJTUJOH7JEFP--.T 7JEFP$IBU#<-J BS9JW> 7JEFP--B." #<;IBOH &./-1> 7JEFP$IBU(15 #<.BB[ "$-> 13
&YQFSJNFOUT • &WBMVBUFPO5XP Ћ UBTLT • 57( • %BUBTFU"DUJWJUZ/FU$BQUJPOTBOE$IBSBEFT45" •
.FUSJDNFBO*06 N*06 SFDBMM!N 3!N • %7$ • %BUBTFU"DUJWJUZ/FU$BQUJPOT • .FUSJD MPDBMJ[JOH U*P6 DBQUJPOJOH $*%&Sɾ.&5&03 DPNCJOBUJPO 40%"@D • 7JEFP%JBMPHVF • %BUBTFU7JEFP$IBU(15 CFODINBSL • .FUSJD(15BTTJHOTTDPSFT d UPBHFOFSBUFEUFYUXJUIBTQFDUT 14 7JEFP%JBMPHVF&WBMVBUJPO
&YQFSJNFOUT 75( %7$ 7JEFP%JBMPHVF 15 3FTVMUTTIPXT75JNF--.PVUQFSGPSNTCBTFMJOFTBDSPTT WBSJPVTWJEFPVOEFSTUBOEJOHUBTLT
&YQFSJNFOUT "CMBUJPO4UVEZ *NBHF5FYUQBJSJTNPSFTVJUBCMFSBUIFSUIBO7JEFP5FYUQBJSGPS4UBHF 7JTVBM"EBQUFSTIPVMECFGSFF[FBGUFS4UBHF -P3" QBSBNFUFSUSBJOFEPO4UBHFTIPVMECFBEEFEPO4UBHF 16
&YQFSJNFOUT 2VBMJUBUJWFSFTVMUTPOWJEFPEJBMPHVF • 75JNF--. JMMVTUSBUFTHPPEEJBMPHVFQFSGPSNBODF 17
$PODMVTJPO • 5IFZJOUSPEVDF75JNF--. B7JEFP--.DBQBCMFPGDPNQSFIFOEJOH NVMUJQMFFWFOUTXJUIJOBWJEFPBOEQSPWJEJOHQSFDJTFUFNQPSBM CPVOEBSJFT • 5IFZQSPQPTFBUISFFTUBHFUFNQPSBMBXBSFUSBJOJOHGSBNFXPSLXJUI UFNQPSBMSFMBUFE2VFTUJPO"OTXFSJOH MFWFSBHJOHNVMUJFWFOUEBUBTFUTGPS
5FNQPSBM7JEFP(SPVOEJOHBOE%FOTF7JEFP$BQUJPOJOH • &YUFOTJWFFYQFSJNFOUTEFNPOTUSBUFUIBU75JNF--.PVUQFSGPSNT FYJTUJOH7JEFP--.TTJHOJGJDBOUMZBDSPTTWBSJPVTWJEFPVOEFSTUBOEJOH UBTLT 18
5IBOLZPV 19
.ZDPODFSOT • $PNQBSJTPOXJUI%7$ 57(POMZNPEFMT • 1FSGPSNBODFPOPUIFSEPNBJO%7$EBUBTFUT • &WBMVBUJPOWBMJEJUZGPSQFSGPSNBODFPO7JEFP%JBMPHVF 20
zaemon1251hesty
kimpetersen
nonsquared
tmiket
addyosmani
destraynor
jonoalderson
portentint
techseoconnect
thoeni
productmarketing
frankvandijk
