Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
【CVPR 2024】VTimeLLM: Empower LLM to Grasp Video...
Search
mori yuichiro
July 09, 2024
65
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
【CVPR 2024】VTimeLLM: Empower LLM to Grasp Video Moments 論文読み会発表資料
mori yuichiro
July 09, 2024
More Decks by mori yuichiro
See All by mori yuichiro
【EMNLP 2023】Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs 論文読み会発表資料
zaemon1251hesty
0
120
Featured
See All Featured
The Pragmatic Product Professional
lauravandoore
37
7.4k
Color Theory Basics | Prateek | Gurzu
gurzu
0
400
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Agile Leadership in an Agile Organization
kimpetersen
PRO
0
200
Paper Plane
katiecoart
PRO
2
52k
Embracing the Ebb and Flow
colly
88
5.1k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
2k
AI Search: Where Are We & What Can We Do About It?
aleyda
0
7.7k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
10
1.3k
Building a Modern Day E-commerce SEO Strategy
aleyda
45
9.1k
Discover your Explorer Soul
emna__ayadi
2
1.2k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
10k
Transcript
75JNF--.&NQPXFS--.UP (SBTQ7JEFP.PNFOUT #JO)VBOH 9JO8BOH )POH$IFO ;JIBO4POH 8FOXV;IV 5TJOHIVB6OJWFSTJUZ $713 :VJDIJSP.PSJ
1BQFS3FBEJOH 1
4VNNBSZ • -BSHFMBOHVBHFNPEFMT --.T IBWFCFFOBEBQUFEUPIBOEMFWJEFP EBUBCVUTUSVHHMFXJUIQSFDJTFFWFOUUJNJOH • 75JNF--. BOFX7JEFP--. BEESFTTFTUIJTCZVTJOHBUISFF
TUBHFUSBJOJOHTUSBUFHZUPJNQSPWFUFNQPSBMVOEFSTUBOEJOH • &YQFSJNFOUTTIPX75JNF--.FYDFMTJOGJOFHSBJOFEWJEFP DPNQSFIFOTJPOBOEPVUQFSGPSNTFYJTUJOH7JEFP--.TJOWBSJPVT UBTLT 2
#BDLHSPVOE • 7JEFP--. • VUJMJ[JOH-BSHF-BOHVBHF.PEFMUPVOEFSTUBOEWJEFP • BOBHFOUGPSJOUFSBDUJOHXJUIIVNBOCBTFEPOBWJEFP • FH
7JEFP--B." <;IBOH &./-1> 7JEFP$IBU <-J BS9JW> 3 7JEFP--B."
1SPCMFN • &YJTUJOH7JEFP--.TGPDVTPOHFOFSBUJPOPGHFOFSJDWJEFP DBQUJPOT UIVTGBJMUPDBQUVSFUIFSFMBUJPOTIJQTCFUXFFOTQFDJGJD CPVOEFEFWFOUT 4 7JEFP$IBU 7JEFP--B."DBOOPU
MPDBMJ[FUIFFWFOU UJNJOH 7JEFP--.TBSF GBSGSPNQSFDJTFMZ VOEFSTUBOEJOH WJEFP
3FTFBSDI0CKFDUJWF 5 5IFZCVJME75JNF--.5IF'JSTUCPVOEBSZBXBSF7JEFP--. 5BDLMJOHUXPDIBMMFOHFTGPS7JEFP--.TUPDBQUVSFFWFOUCPVOEBSJFTJO WJEFP 4DBSDJUZPGMBSHFTDBMFCPVOEBSZBXBSFWJEFPEBUBTFUT Ø 5IFZDSFBUFEBUBTFUTJODPSQPSBUJOHFYJTUJOHUFNQPSBMSFMBUFE 7JEFP6OEFSTUBOEJOHUBTLT
%FTJHOFGGFDUJWFUFNQPSBMSFMBUFEWJEFPUBTLTGPSUSBJOJOH7JEFP--. Ø 5IFZQSPQPTF#PVOEBSZBXBSF5SBJOJOH4USBUFHZ
1SPQPTFE.FUIPE • 1SFMJNJOBSZ5XPUFNQPSBMSFMBUFE7JEFP6OEFSTUBOEJOHUBTLT 6 5FNQPSBM7JEFP (SPVOEJOH 57( %FOTF7JEFP $BQUJPOJOH %7$
• UFNQPSBMMPDBMJ[BUJPODPSSFTQPOEJOHWJEFP TFHNFOUTGPSHJWFOUFYUVBMJOQVUT • UFNQPSBMMPDBMJ[BUJPOBOEDBQUJPOJOH GPSBMMFWFOUT 5FNQPSBMMZ(SPVOEJOH/BUVSBM4FOUFODFJO7JEFP <+JOHZVBO &./-1> %FOTF$BQUJPOJOHJOFWFOUT <,SJTIOB *$$7>
1SPQPTFE.FUIPE • #PVOEBSZBXBSF5SBJOJOH4USBUFHZ DPOTJTUT5SBJOJOH4UBHFT 7
1SPQPTFE.FUIPE • 4UBHF'FBUVSF"MJHONFOU • "JNTUPUSBJOUIFWJTVBMBEBQUFS UPBMJHOWJEFP GFBUVSFTXJUI--.`TTFNBOUJDTQBDF • %BUBTFU-$4, JNBHFUFYUQBJSVTFEGPS
QSFUSBJOJOH--B7"<-JV /FVS*14`> • /PUF • 4JODFDPOUFNQPSBSZMBSHFTDBMFWJEFPUFYUEBUBTFUTDPOUBJO TVCTUBOUJBMUFYUVBMOPJTF UIFZVTFUIFJNBHFUFYUEBUBTFU 8
1SPQPTFE.FUIPE • 4UBHF#PVOEBSZ1FSDFQUJPO • "JNTUPGBDJMJUBUJOHUIFVOEFSTUBOEJOHPG WBSJPVTFWFOUTPDDVSSJOHXJUIJOUIFWJEFP • %BUBTFUTJOHMFUVSOBOENVMUJUVSO RVFTUJPOBOTXFSJOH 2"
• GSPN*OUFSO7JE.'-5<8BOH *$-3> CPVOEBSZBXBSFEBUBTFU • *O*OUFSO7JE.'-5 1Z4DFOF%FUFDU BVUPNBUJDBMMZTQMJUVOUSJNNFEWJEFPJOUPNVMUJQMF DMJQT 9
1SPQPTFE.FUIPE • 4UBHF*OTUSVDUJPO5VOJOH • "JNTUPBMJHOXJUIIVNBOJOUFOU • %JBMPHVFNBEFGSPNIJHIRVBMJUZUFNQPSBM SFMBUFEEBUBTFUTFOBCMF75JNF--.NPSF QSFDJTFFWFOUMPDBMJ[BUJPOBOEEFTDSJQUJPO •
%BUBTFU--.HFOFSBUFEWJEFPCBTFE EJBMPHVFTGSPN"DUJWJUZ/FU$BQUJPO <,SJTIOB *$$7> %7$ BOE%J%F.P <)FOESJDLT *$$7> 75( 10 Trained on Stage 2
1SPQPTFE.FUIPE • 1SPNQUTPGUSBJOJOH4UBHF 11 WJEFPGFBUVSF FNCFEEJOHMJTU XJMMCFJOTFSUFEIFSF TZTUFN QSPNQU
1SPQPTFE.FUIPE • 1SPNQUTPGUSBJOJOH4UBHF 12 WJEFPGFBUVSF FNCFEEJOHMJTU XJMMCFJOTFSUFEIFSF TZTUFN QSPNQU Q1
A1 Q2 A2 Q3 A3 Q4 A4 #PUI2BOE"BSF HFOFSBUFECZ--.
&YQFSJNFOUT • *NQMFNFOUBUJPO • --.7JDVOBW # # <;IFOH /FVS*14> •
-PTTGVODUJPOPSJHJOBMBVUPSFHSFTTJWFUSBJOJOHPCKFDUJWFTPGUIF--. DSPTT FOUSPQZ • *OTFDPOEBOEUIJSETUBHFT UIFZPOMZUSBJO-P3"QBSBNFUFST XJUIGSFF[JOH QSFUSBJOFEQBSBNFUFST • 5SBJOJOH359(16 • #BTFMJOFT • &YJTUJOH7JEFP--.T 7JEFP$IBU#<-J BS9JW> 7JEFP--B." #<;IBOH &./-1> 7JEFP$IBU(15 #<.BB[ "$-> 13
&YQFSJNFOUT • &WBMVBUFPO5XP Ћ UBTLT • 57( • %BUBTFU"DUJWJUZ/FU$BQUJPOTBOE$IBSBEFT45" •
.FUSJDNFBO*06 N*06 SFDBMM!N 3!N • %7$ • %BUBTFU"DUJWJUZ/FU$BQUJPOT • .FUSJD MPDBMJ[JOH U*P6 DBQUJPOJOH $*%&Sɾ.&5&03 DPNCJOBUJPO 40%"@D • 7JEFP%JBMPHVF • %BUBTFU7JEFP$IBU(15 CFODINBSL • .FUSJD(15BTTJHOTTDPSFT d UPBHFOFSBUFEUFYUXJUIBTQFDUT 14 7JEFP%JBMPHVF&WBMVBUJPO
&YQFSJNFOUT 75( %7$ 7JEFP%JBMPHVF 15 3FTVMUTTIPXT75JNF--.PVUQFSGPSNTCBTFMJOFTBDSPTT WBSJPVTWJEFPVOEFSTUBOEJOHUBTLT
&YQFSJNFOUT "CMBUJPO4UVEZ *NBHF5FYUQBJSJTNPSFTVJUBCMFSBUIFSUIBO7JEFP5FYUQBJSGPS4UBHF 7JTVBM"EBQUFSTIPVMECFGSFF[FBGUFS4UBHF -P3" QBSBNFUFSUSBJOFEPO4UBHFTIPVMECFBEEFEPO4UBHF 16
&YQFSJNFOUT 2VBMJUBUJWFSFTVMUTPOWJEFPEJBMPHVF • 75JNF--. JMMVTUSBUFTHPPEEJBMPHVFQFSGPSNBODF 17
$PODMVTJPO • 5IFZJOUSPEVDF75JNF--. B7JEFP--.DBQBCMFPGDPNQSFIFOEJOH NVMUJQMFFWFOUTXJUIJOBWJEFPBOEQSPWJEJOHQSFDJTFUFNQPSBM CPVOEBSJFT • 5IFZQSPQPTFBUISFFTUBHFUFNQPSBMBXBSFUSBJOJOHGSBNFXPSLXJUI UFNQPSBMSFMBUFE2VFTUJPO"OTXFSJOH MFWFSBHJOHNVMUJFWFOUEBUBTFUTGPS
5FNQPSBM7JEFP(SPVOEJOHBOE%FOTF7JEFP$BQUJPOJOH • &YUFOTJWFFYQFSJNFOUTEFNPOTUSBUFUIBU75JNF--.PVUQFSGPSNT FYJTUJOH7JEFP--.TTJHOJGJDBOUMZBDSPTTWBSJPVTWJEFPVOEFSTUBOEJOH UBTLT 18
5IBOLZPV 19
.ZDPODFSOT • $PNQBSJTPOXJUI%7$ 57(POMZNPEFMT • 1FSGPSNBODFPOPUIFSEPNBJO%7$EBUBTFUT • &WBMVBUJPOWBMJEJUZGPSQFSGPSNBODFPO7JEFP%JBMPHVF 20
zaemon1251hesty
lauravandoore
gurzu
cassininazir
kimpetersen
katiecoart
colly
honnibal
aleyda
addyosmani
emna__ayadi
eileencodes
