第三回 全日本コンピュータビジョン勉強会(後編)/ Self-Mono-SF

第三回 全日本コンピュータビジョン勉強会(後編)/ Self-Mono-SF

第三回 全日本コンピュータビジョン勉強会 CVPR2020読み会(後編)にて発表した際に使用した資料です。

"Self-supervised Monocular Scene Flow Estimation"

C31b5afbe1ffb5588b2fe5a9e39b357b?s=128

Takumi Karasawa

July 18, 2020
Tweet

Transcript

  1. . , .,. . PTV F EC S 0 -

    - -2 R M
  2. , , . , # 2 n , , .

    , n LM n . 9 4 n . 0123 n . A C  @Takarasawa_
  3. - - , n - - n - , -

    . : - -, - L C M - - . O  3 CVPR2020 4
  4. 1 9 ) blOi n fc 1 1 ? h

    blOF Cfc n ,1 9 ? p 1 ? ,1 9 ? V hO I 1 ? n , . SO T ( . OL g a Moed 4 Scene Flow
  5. E: IN DE E E .I O,J I E I

    I ) EL 3 3JF GK 3 D ) EL ( I C I EDP G . - E DI EJ ] 3 D ) EL rv G I :EG E N ED I D N E c wi JF GK GD D c n O F G D FI ) EL IE 3 D ) EL GEJ FI (MF D EDP G . - dfdghe] c FI ) EL csm Wb 3 D ) EL x FI (MF D EDSo 3 D ) EL rv TU ul]rv apty[rvc n 0 2 2 0 5
  6. . , .,. . 6

  7. , CF 9 B +C n -- F 2B C

    2C , . 0 E C 2 1 n 0 LhpA D 4 0 0 0 4 dl n 0 G sb Lo o e n , C C2B C9 C C 2 E ➤ B E 4 4 0 O f n dl ti n 0 Ira G 7 c w Scene flow Optical flow R EPC++ T n dluv [C. Luo et. al., 2018]
  8. . , .,. . 8 CVPR2020   kzykmyzw 

    EPC++ EPC
  9. 1 . 2 3 hfwN 3 T 2 D v

    a - - IV b 1 . C 1 C D - 3 3 2 C 3 2 2 D , 2 C , 1 D plV y F pl untTe NV dVs N g SL ➤ oiPVmM k cr 9
  10. , - . + D C LMT PS - E

    10 V. Guizilini et. al., “3D Packing for Self-Supervised Monocular Depth Estimation”, CVPR 2020
  11. , . , + 3, 3 T RCD L M

    CD D , 3 11 S. Wu et. al., “Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild”, CVPR 2020
  12. , . , + 3, 3 T RCD L M

    CD D , 3 12 S. Wu et. al., “Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild”, CVPR 2020  !   scene flow & depth    "  
  13. - . ) ( ., f , , CW T

    N n ) , , , bi h P LO , n , h Pf M ec , lp , , dg n m a . , ., f op n 082 1 21 2 - 2 . , 2 0 13 PWC-Net   " !   #$ 
  14. , - . - ad P C SL n c

    - ad . - h O MWTN n - g b e - f    14 PWC-Net  optical flow estimator Scene flow  Optical flow   2 
  15. - . ., T Ta M W b n y

    ., T M ST , M . Tt e c N n . T rlh N g ML ., Ts , oC d h N f T g M Op Oo , Tt i n D MO wP    15  X X’ 0 0 depth B f f !"#$%&"'( = * − *, = -. !/$'ℎ  
  16. ., . - BCD . n . D A n

    , 3., , ,. F F 16
  17. ( . ➤ fhrcu T dnoTfhrc on g fhrcu w

    T . , ( t y gb lf M fp iase C T ( L Dfhrcu L . Tw t 17
  18. , . , , , Tm c c Tb !

    "# faDi L M hMl c _ kn {%& , %&() } eTkSgC {+& , +&() } {",- , ".- } Tdn fa 18 Disparity loss Scene Flow loss  
  19. , . - , . 1, 1 . , T

    P hMd e . 1, L 1 h i M Cb hMag h . 1, M c , 19 Photometric loss Smoothness loss Synthesized left image Occlusion mask ➤  Disparity loss  occlusion mask  disparity map   Left image SSIM L1  
  20. ,2 2 -1 2 . 1 2- 1 P lSet

    ah 1 ➤ . . . 1 i E .2 2 i M m p r sS C l conSd T b L 1 sSgM 20 Photometric loss Smoothness loss  Disparity
  21. +2 2 .,1 2. - ,. . L L Lc

    d 1 . ,. . 1 . 2, -2 2 ,. . e L bC L b a b L 1 . 2, ,, -.- T S LM 21 Scene flow photometric loss 3D point reconstruction loss Smoothness loss Synthesized t image Occlusion mask ➤ occlusion mask  flow  t image ➤ Edge-aware 2nd order smoothness Depth Optical flow  
  22. - + .3 2 . 1 L C 3 M

    T . . aS , c aSD d b , , c . 1 eD 22 Scene flow photometric loss 3D point reconstruction loss Smoothness loss  d(t+1)! Occlusion mask d(t) Scene flow ! ➤ Edge-aware 2nd order smoothness ➤ depth map  "   scene flow   
  23. . , .,. . L C C 23 !" #

    !" $ !" !"%& '" # ()* '" '"%& '" stereo config ()* Disparity loss t+1  Scene flow loss backward  '" ()*
  24. 0 CG 2 : 9 /C6 .-22- d be n

    .-22- 3F 63C3 C I n .-22- 1 F 2 3 9I 39 ,2 K c e beT sv n .-22- 1 F 2 3 9 p .-22- 3F 63C3 C M ul C 3 3 gibe n 0 1 2 1 n 1 2 1 1 6 C 9 hd S .-22- 1 F 2 3 9 d mfba o Sr 6 C 3 F / 6 3 CG / Lnt - 24
  25. 3 F 1 ) . 5 -,11, D emu l

    pfTxs ( tSn I I T C L bSrM C Tg n % 2 2 T 5 2 F n 2 2 Sa c K wdiT 5 2 F n % 2 2 D T 5 n 2 ohM pfT T D L Tpf M - 25
  26. , . - , L FC M , , S

    26 Multi-task methods (Depth, Ego-motion, Optical flow) Energy-optimization (semi-supervised) • D1-allreference frame disparity • D2-allreference frame  disparity • F1-alloptical flow end-point • SF-all   scene flow   
  27. . , .,. . 27 Semi-supervised fine tuning 

  28. & 3 . , & 3 , 3 n D

    C T 3 n 3 M O L n , 3 3 3 29
  29. . . ,. TS C L SM . 30

  30. . , .,. . 31 LMC

  31. . , . 32 D D L M D T

    D C D . D
  32. . , . 33 D D L M D T

    D C D . D Disparity loss Scene Flow loss ><@2! &#() 4/ 0,><@2! &#()  loss @>1"')+%"C  =   AB*% - .>  :?3 $+$+-  disparity loss 875 !; 69
  33. ., . - ,. . . l . . .

    . . .- ,. . yrl , ,. . . b . . .- . l n ,. . l vx wDrb , wDrb N fwD tn l i i n . ,. . b l d T . . . b d , l c M n -. c . e , b aC n oxupDsbLM a T c h l S b c bMg n ,. . a m Th , , . . . ,. . . , .>  34