, , . , #
2
n , , . ,
n LM
n . 9 4
n . 0123
n . A
C
@Takarasawa_
Slide 3
Slide 3 text
- - ,
n - -
n - ,
- . : - -, - L C M - - . O
3
CVPR2020
4
Slide 4
Slide 4 text
1 9 )
blOi n fc
1 1 ? h blOF Cfc
n ,1 9 ? p 1 ?
,1 9 ? V hO I 1 ?
n , . SO T ( . OL g a Moed
4
Scene Flow
Slide 5
Slide 5 text
E: IN DE E E .I
O,J I E I I ) EL 3 3JF GK 3 D ) EL ( I C I EDP G . -
E DI EJ ] 3 D ) EL rv
G I :EG E N ED I D N E c
wi JF GK GD D c n
O F G D FI ) EL IE 3 D ) EL GEJ FI (MF D EDP G . -
dfdghe] c FI ) EL csm Wb 3 D ) EL x
FI (MF D EDSo 3 D ) EL rv TU ul]rv apty[rvc n
0 2 2 0
5
Slide 6
Slide 6 text
. , .,. .
6
Slide 7
Slide 7 text
, CF 9 B +C
n -- F 2B C 2C , . 0 E C 2 1
n 0 LhpA D 4 0 0 0 4 dl
n 0 G sb Lo o e
n , C C2B C9 C C 2 E ➤ B E
4 4 0 O f
n dl ti
n 0 Ira G
7
c w
Scene flow
Optical flow
R
EPC++ T n dluv [C. Luo et. al., 2018]
Slide 8
Slide 8 text
. , .,. .
8
CVPR2020
kzykmyzw
EPC++
EPC
Slide 9
Slide 9 text
1 . 2 3
hfwN 3 T 2 D v a - - IV b
1
. C 1 C D
- 3
3 2 C 3 2 2 D
, 2 C , 1 D
plV y F pl untTe NV dVs N g SL
➤ oiPVmM k cr
9
Slide 10
Slide 10 text
, - .
+
D C LMT PS - E
10
V. Guizilini et. al., “3D Packing for Self-Supervised Monocular Depth Estimation”, CVPR 2020
Slide 11
Slide 11 text
, . , +
3, 3 T RCD
L M CD D , 3
11
S. Wu et. al., “Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild”, CVPR 2020
Slide 12
Slide 12 text
, . , +
3, 3 T RCD
L M CD D , 3
12
S. Wu et. al., “Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild”, CVPR 2020
!
scene flow & depth
"
Slide 13
Slide 13 text
- .
) ( ., f , , CW T N
n ) , , , bi h P LO ,
n , h Pf M ec , lp , , dg
n m a . , ., f op n
082 1 21 2 - 2 . , 2 0
13
PWC-Net
"
!
#$
Slide 14
Slide 14 text
, - .
- ad P C SL
n c - ad . - h O MWTN
n - g b e - f
14
PWC-Net optical flow estimator
Scene flow Optical flow
2
Slide 15
Slide 15 text
- .
., T Ta M W b
n y ., T M ST , M . Tt e c N
n . T rlh N g ML ., Ts
, oC d h N f T g M Op Oo
, Tt i n D MO wP
15
X X’
0 0
depth
B
f f
!"#$%&"'( = * − *, =
-.
!/$'ℎ
Slide 16
Slide 16 text
., . -
BCD .
n . D A
n , 3., , ,.
F
F
16
Slide 17
Slide 17 text
( .
➤ fhrcu T
dnoTfhrc on g fhrcu w T
. , ( t
y gb lf M fp iase C T
( L Dfhrcu L . Tw t
17
Slide 18
Slide 18 text
, . ,
, , Tm c
c Tb ! "# faDi L M hMl
c _ kn {%&
, %&()
} eTkSgC
{+&
, +&()
} {",-
, ".-
} Tdn fa
18
Disparity loss Scene Flow loss
Slide 19
Slide 19 text
, . - ,
. 1, 1 . , T P hMd e
. 1,
L 1 h i M Cb hMag
h . 1, M c ,
19
Photometric loss Smoothness loss
Synthesized left image
Occlusion mask
➤ Disparity loss occlusion mask disparity map
Left image
SSIM
L1
Slide 20
Slide 20 text
,2 2 -1 2 .
1 2- 1 P lSet ah
1 ➤ . . . 1
i E .2 2 i M m
p r sS
C l conSd T b L 1 sSgM
20
Photometric loss Smoothness loss
Disparity
Slide 21
Slide 21 text
+2 2 .,1 2. -
,. . L L Lc d 1 .
,. . 1 . 2,
-2 2 ,. . e L bC
L b a b L 1 . 2,
,, -.- T S LM
21
Scene flow photometric loss 3D point reconstruction loss Smoothness loss
Synthesized t image
Occlusion mask ➤ occlusion mask flow
t image
➤ Edge-aware 2nd order smoothness
Depth
Optical flow
Slide 22
Slide 22 text
- + .3 2
. 1 L C 3 M T
. .
aS , c aSD d
b , , c . 1 eD
22
Scene flow photometric loss 3D point reconstruction loss Smoothness loss
d(t+1)!
Occlusion mask
d(t)
Scene flow !
➤ Edge-aware 2nd order smoothness
➤ depth map " scene flow
Slide 23
Slide 23 text
. , .,. .
L C C
23
!"
# !"
$
!"
!"%&
'"
#
()*
'" '"%&
'"
stereo config
()*
Disparity loss
t+1
Scene flow loss
backward
'"
()*
Slide 24
Slide 24 text
0 CG 2 : 9 /C6
.-22- d be
n .-22- 3F 63C3 C I
n .-22- 1 F 2 3 9I 39 ,2 K c e beT sv
n .-22- 1 F 2 3 9 p .-22- 3F 63C3 C M ul
C 3 3 gibe
n 0 1 2 1
n 1 2 1
1 6 C 9 hd S
.-22- 1 F 2 3 9 d mfba o Sr
6 C 3 F / 6 3 CG / Lnt
-
24
Slide 25
Slide 25 text
3 F 1 ) . 5
-,11, D emu
l pfTxs ( tSn I I T C L
bSrM C Tg
n % 2 2 T 5 2 F
n 2 2 Sa c K wdiT 5 2 F
n % 2 2 D T 5
n 2 ohM pfT T D L Tpf M
-
25
Slide 26
Slide 26 text
, . - ,
L FC M , , S
26
Multi-task methods
(Depth, Ego-motion, Optical flow)
Energy-optimization
(semi-supervised)
• D1-allreference frame
disparity
• D2-allreference frame
disparity
• F1-alloptical flow
end-point
• SF-all
scene flow
Slide 27
Slide 27 text
. , .,. .
27
Semi-supervised fine tuning
Slide 28
Slide 28 text
& 3 . ,
& 3 , 3
n D C T
3
n 3 M O L
n , 3 3 3
29
Slide 29
Slide 29 text
. . ,.
TS C L SM
.
30
Slide 30
Slide 30 text
. , .,. .
31
LMC
Slide 31
Slide 31 text
. , .
32
D D
L M D T D C D
. D
Slide 32
Slide 32 text
. , .
33
D D
L M D T D C D
. D
Disparity loss Scene Flow loss
><@2! ()
4/
0,><@2! ()
loss
@>1"')+%"C =
AB*% -
.>
:?3
$+$+-
disparity loss 875!; 69
Slide 33
Slide 33 text
., . -
,. . . l . . . . . .- ,. . yrl
, ,. . . b . . .- . l
n ,. . l vx wDrb
, wDrb N fwD tn l i i
n . ,. . b l d T . . . b
d , l c M
n -. c . e , b aC
n oxupDsbLM a T c h
l S b c bMg
n ,. . a m Th
, , . . . ,. . . , .>
34