第三回 全日本コンピュータビジョン勉強会（後編）/ Self-Mono-SF

第三回　全日本コンピュータビジョン勉強会 CVPR2020読み会（後編）にて発表した際に使用した資料です。

"Self-supervised Monocular Scene Flow Estimation"

Transcript

1. . , .,. . PTV F EC S 0 - - -2 R M
2. , , . , # 2 n , , . , n LM n . 9 4 n . 0123 n . A C 
   @Takarasawa_
3. - - , n - - n - , - . : - -, - L C M - - . O 
    3 CVPR2020 4
   
4. 1 9 ) blOi n fc 1 1 ? h blOF Cfc n ,1 9 ? p 1 ? ,1 9 ? V hO I 1 ? n , . SO T ( . OL g a Moed 4 Scene Flow
5. E: IN DE E E .I O,J I E I I ) EL 3 3JF GK 3 D ) EL ( I C I EDP G . - E DI EJ ] 3 D ) EL rv G I :EG E N ED I D N E c wi JF GK GD D c n O F G D FI ) EL IE 3 D ) EL GEJ FI (MF D EDP G . - dfdghe] c FI ) EL csm Wb 3 D ) EL x FI (MF D EDSo 3 D ) EL rv TU ul]rv apty[rvc n 0 2 2 0 5
6. . , .,. . 6
7. , CF 9 B +C n -- F 2B C 2C , . 0 E C 2 1 n 0 LhpA D 4 0 0 0 4 dl n 0 G sb Lo o e n , C C2B C9 C C 2 E ➤ B E 4 4 0 O f n dl ti n 0 Ira G 7 c w Scene flow Optical flow R EPC++ T n dluv [C. Luo et. al., 2018]
8. . , .,. . 8 CVPR2020 
   kzykmyzw  EPC++ EPC
9. 1 . 2 3 hfwN 3 T 2 D v a - - IV b 1 . C 1 C D - 3 3 2 C 3 2 2 D , 2 C , 1 D plV y F pl untTe NV dVs N g SL ➤ oiPVmM k cr 9
10. , - . + D C LMT PS - E 10 V. Guizilini et. al., “3D Packing for Self-Supervised Monocular Depth Estimation”, CVPR 2020
11. , . , + 3, 3 T RCD L M CD D , 3 11 S. Wu et. al., “Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild”, CVPR 2020
12. , . , + 3, 3 T RCD L M CD D , 3 12 S. Wu et. al., “Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild”, CVPR 2020  !
     scene flow & depth    "  
13. - . ) ( ., f , , CW T N n ) , , , bi h P LO , n , h Pf M ec , lp , , dg n m a . , ., f op n 082 1 21 2 - 2 . , 2 0 13 PWC-Net   " !   #
    $ 
14. , - . - ad P C SL n c - ad . - h O MWTN n - g b e - f 
     
     14 PWC-Net  optical flow estimator Scene flow 
    Optical flow   2 
15. - . ., T Ta M W b n y ., T M ST , M . Tt e c N n . T rlh N g ML ., Ts , oC d h N f T g M Op Oo , Tt i n D MO wP 
     
     15  X X’ 0 0 depth B f f !"#$%&"'( = * − *, = -. !/$'ℎ 
    
16. ., . - BCD . n . D A n , 3., , ,. F F 16
17. ( . ➤ fhrcu T dnoTfhrc on g fhrcu w T . , ( t y gb lf M fp iase C T ( L Dfhrcu L . Tw t 17
18. , . , , , Tm c c Tb ! "# faDi L M hMl c _ kn {%& , %&() } eTkSgC {+& , +&() } {",- , ".- } Tdn fa 18 Disparity loss Scene Flow loss 
    
19. , . - , . 1, 1 . , T P hMd e . 1, L 1 h i M Cb hMag h . 1, M c , 19 Photometric loss Smoothness loss Synthesized left image Occlusion mask ➤  Disparity loss  occlusion mask
     disparity map   Left image SSIM L1  
20. ,2 2 -1 2 . 1 2- 1 P lSet ah 1 ➤ . . . 1 i E .2 2 i M m p r sS C l conSd T b L 1 sSgM 20 Photometric loss Smoothness loss 
     Disparity
21. +2 2 .,1 2. - ,. . L L Lc d 1 . ,. . 1 . 2, -2 2 ,. . e L bC L b a b L 1 . 2, ,, -.- T S LM 21 Scene flow photometric loss 3D point reconstruction loss Smoothness loss Synthesized t image Occlusion mask ➤ occlusion mask  flow  t image ➤ Edge-aware 2nd order smoothness Depth 
    Optical flow  
22. - + .3 2 . 1 L C 3 M T . . aS , c aSD d b , , c . 1 eD 22 Scene flow photometric loss 3D point reconstruction loss Smoothness loss  d(t+1)! Occlusion mask d(t) Scene flow ! ➤ Edge-aware 2nd order smoothness ➤ depth map  "  
    scene flow   
23. . , .,. . L C C 23 !" # !" $ !" !"%& '" # ()* '" '"%& '" stereo config ()* Disparity loss
    t+1  Scene flow loss
    backward  '" ()*
24. 0 CG 2 : 9 /C6 .-22- d be n .-22- 3F 63C3 C I n .-22- 1 F 2 3 9I 39 ,2 K c e beT sv n .-22- 1 F 2 3 9 p .-22- 3F 63C3 C M ul C 3 3 gibe n 0 1 2 1 n 1 2 1 1 6 C 9 hd S .-22- 1 F 2 3 9 d mfba o Sr 6 C 3 F / 6 3 CG / Lnt - 24
25. 3 F 1 ) . 5 -,11, D emu l pfTxs ( tSn I I T C L bSrM C Tg n % 2 2 T 5 2 F n 2 2 Sa c K wdiT 5 2 F n % 2 2 D T 5 n 2 ohM pfT T D L Tpf M - 25
26. , . - , L FC M , , S 26 Multi-task methods (Depth, Ego-motion, Optical flow) Energy-optimization (semi-supervised) • D1-all
    reference frame disparity • D2-all
    reference frame  disparity • F1-all
    optical flow end-point • SF-all
       scene flow   
27. . , .,. . 27 Semi-supervised fine tuning 
    
28. & 3 . , & 3 , 3 n D C T 3 n 3 M O L n , 3 3 3 29
29. . . ,. TS C L SM . 30
30. . , .,. . 31 LMC
31. . , . 32 D D L M D T D C D . D
32. . , . 33 D D L M D T D C D . D Disparity loss Scene Flow loss ><@2! &#() 4/ 0,
    ><@2! &#()  loss @>1"')+%"C  =  
    AB*% - .>  :?3 $+$+-  disparity loss 875 !; 69
33. ., . - ,. . . l . . . . . .- ,. . yrl , ,. . . b . . .- . l n ,. . l vx wDrb , wDrb N fwD tn l i i n . ,. . b l d T . . . b d , l c M n -. c . e , b aC n oxupDsbLM a T c h l S b c bMg n ,. . a m Th , , . . . ,. . . , .> 
     34