CVPR論文紹介: A Point Set Generation Network for 3D Object Reconstruction from a Single Image

コンピュータビジョン分野のトップカンファレンスCVPR2017で発表された論文の内，ポイントクラウドの推定にフォーカスした表題の論文を紹介するスライドです．

このスライドは関東CV勉強会( https://kantocv.connpass.com/event/62020/ )での発表資料です．

Transcript

1. %఺܈ͷऔѻ͍
   "1PJOU4FU(FOFSBUJPO/FUXPSLGPS
   %0CKFDU3FDPOTUSVDUJPOGSPNB4JOHMF*NBHF
   େ֞ܚհ
   
   

   View Slide

2. "1PJOU4FU(FOFSBUJPO/FUXPSLGPS
   %0CKFDU3FDPOTUSVDUJPOGSPNB4JOHMF*NBHF
   
   ·ͱΊ: ୯؟Point Cloudਪఆ
   ໰୊ઃఆ
   Ұຕը૾͔Βɺ1PJOU$MPVEΛਪఆ͢
   Δ
   ͓΋͠Ζ͍ͱ͜Ζ
   ෼෍ؒͷڑ཭Λ࠷খԽ͢Δ͜ͱͰ௚઀
   ϙΠϯτΫϥ΢υΛग़ྗ͢Δωοτϫʔ
   ΫΛֶशͰ͖Δ
   ݁ہૄͳग़ྗͰ͋ͬͯ΋ɼͳΜͱ͔ີ
   ͳख๏ %FDPOW
   Λ͏·͘ར༻͢Δߏ
   ଄ʹམͱ͠ࠐΉͷؙ͕͍
   ײ૝
   %Ͱ΋%Ͱ΋ɼૄͳσʔλߏ଄ͷֶ
   श͸ɼ໌Β͔ʹ$//ʹಥͬࠐΊ͹͍
   ͍ΑͶܥͷख๏ΑΓ͸·ͩ·ͩߟ͑Δ
   ͜ͱ͕ͨ͘͞Μ͋ͬͯ໘ന͍
   )BPRJBOH'BO )BP4V -FPOJEBT(VJCBT
   ࿦จ'JHΑΓ
   

   View Slide

3. ࣮ࡍʹࢼͯ͠Έ·ͨ͠
   
   IUUQTHJUIVCDPNGBOIRNF1PJOU4FU(FOFSBUJPO
   

   View Slide

4. Ͳ͏͍͏ΞϓϦέʔγϣϯ
   
   ͍͍ײ͡ͷࣸਅΛࡱΔ ର৅෺ΛϚεΫ͢Δ
   ͍͍ײ͡ʹΫϩοϓ͢Δ 1PJOU$MPVE͕ग़ͯདྷΔ ͜͜·Ͱ໿෼
   
   ؤுΔ
   ؤுΔ
   πʔϧ͕ఏڙ͞ΕͯΔ
   
   ࣗಈ
   

   View Slide

5. ࡉ͔ͳ੍໿
   
   ϚεΫ࡞੒͕ΊΜͲ͍ɻɻɻ
   ׂͱ͔ͬ͠Γ࣮ݧ৚݅Λἧ͑ͳ͖Ό͍͚ͳ͍Β͍͠
   ͱ͸͍͑ɺઌͷ݁Ռ͸ɺͳΜͱͳ͘ద౰ʹऔͬͨ
   ࣸਅͰͪΌΜͱҰൃͰ΍ͬͨྫͰ͢ɻ
   Χϝϥ͔Β؍ͯ͜ͷ൒ٿʹϐολϦର৅͕ೖͬͯ
   Δඞཁ͕͋Δɻ
   

   View Slide

6. ࣮૷ৄࡉ
   

   View Slide

7. ୭ʹ޲͚ͯͷൃද
   
   ˕%Ϟσϧͷਂ૚ֶशΛ͍ͨ͠ਓ
   ̋ϥελը૾Ҏ֎ͷը૾ॲཧΛ͍ͨ͠ਓ
   ˚ఆ൪λεΫͷਫ਼౓Λ͍͋͛ͨਓ
   

   View Slide

8. ը૾ੜ੒໰୊ͷ෼ྨ ݸਓͷݟղͰ͢
   
   
   ࣍ݩ
   ࣍ݩ
   ີ ૄ
   <>IUUQTSFTFBSDIHPPHMFCMPHDPNUFBDIJOHNBDIJOFTUPESBXIUNM
   <>6/461&37*4&%3&13&4&/5"5*0/-&"3/*/(8*5)%&&1$0/70-65*0/"-(&/&3"5*7&"%7&34"3*"-/&5803,4
   <>%FFQFS%FQUI1SFEJDUJPOXJUI'VMMZ$POWPMVUJPOBM3FTJEVBM/FUXPSLT
   <>%3/%3FDVSSFOU3FDPOTUSVDUJPO/FVSBM/FUXPSL
   <>(3"44(FOFSBUJWF3FDVSTJWF"VUPFODPEFSTGPS4IBQF4USVDUVSFT
   %FDPOWPMVUJPOܥ %$("/౳
   <> 4LFUDI3//<>
   %FQUIਪఆ<> %%FDPOW<>
   1PJOU$MPVE
   ϝογϡͷ௚઀ੜ੒<>
   ݚڀͷϑϩϯςΟΞ
   

   View Slide

9. ը૾ੜ੒໰୊ͷ෼ྨ ݸਓͷݟղͰ͢
   
   
   ࣍ݩ
   ࣍ݩ
   ີ ૄ
   <>IUUQTSFTFBSDIHPPHMFCMPHDPNUFBDIJOHNBDIJOFTUPESBXIUNM
   <>6/461&37*4&%3&13&4&/5"5*0/-&"3/*/(8*5)%&&1$0/70-65*0/"-(&/&3"5*7&"%7&34"3*"-/&5803,4
   <>%FFQFS%FQUI1SFEJDUJPOXJUI'VMMZ$POWPMVUJPOBM3FTJEVBM/FUXPSLT
   <>%3/%3FDVSSFOU3FDPOTUSVDUJPO/FVSBM/FUXPSL
   <>(3"44(FOFSBUJWF3FDVSTJWF"VUPFODPEFSTGPS4IBQF4USVDUVSFT
   %FDPOWPMVUJPOܥ %$("/౳
   <> 4LFUDI3//<>
   %FQUIਪఆ<> %%FDPOW<>
   1PJOU$MPVE
   ϝογϡͷ௚઀ੜ੒<>
   ݚڀͷϑϩϯςΟΞ
   1SPT
   w $POW͕ڧ͍ɽΰχϣΰχϣ
   ͠ͳͯ͘΋σΟʔϓϥʔχ
   ϯά͕ͳΜͱ͔ͯ͘͠ΕΔ ཁ
   ग़య
   
   $POT
   w ղ૾౓͕ωοτϫʔΫͷαΠ
   ζʹґଘ͢Δ͜ͱ͕ଟ͍
   1SPT
   w ύϥϝλԽ͞Εͨදݱͷ৔߹ɺ
   ࣗ༝ʹߴղ૾౓ʹͰ͖Δ ϥ
   ελը૾ϕΫλը૾
   
   w ղऍՄೳͳ
   ύϥϝλ͕ग़ྗ
   ʹͳΔͷͰɺਓ͕ؒޙॲཧͰ
   ͖Δ
   $POT
   w $POWͷΑ͏ʹίϞσΟςΟ
   Խͯ͠ͳ͍ͷͰɺωοτϫʔ
   Ϋ΍ϩεΛࣗ෼Ͱߟ͑ͳ͖Ό
   ͍͚ͳ͍
   

   View Slide

10. ࣮૷ʹؔ͢Δͭͷ࿦఺
    
    ϩε͸Ͳ͏͢Δ
    w ڭࢣ͋ΓֶशΛ͢Δ͕ɺϙΠϯτΫϥ΢υಉ࢜ͷڑ཭͸Ͳ͏΍ͬͯఆٛ
    ͢Δ
    ωοτϫʔΫͷܗঢ়͸
    w ग़ྗ͕ը૾͡Όͳ͍͜ͷϞσϧͷωοτϫʔΫ͸Ͳ͏΍ͬͯઃܭ͢Δ
    ֶशσʔλ͸
    w ը૾ͱϙΠϯτΫϥ΢υͷϖΞϫΠζσʔλ͸Ͳ͏΍ͬͯूΊΔ
    ໘ന͍ٞ୊͸ҎԼͷ఺
    

    View Slide

11. ϩε͸
    
    ϙΠϯτ͸ɺ఺͝ͱͷϖΞϫΠζσʔλ͸࡞Εͳ͍ͷͰɺϙΠϯτΫϥ΢υ ఺܈
    ಉ࢜ͷڑ཭Λ࢖͏͜ͱ
    ݁ہ࠷΋͍ۙ఺ΛϖΞʹ͍ͯ͠ΔɻͨͩɺYUͱUYͷ྆ํͷରԠ෇Λऔ͍ͬͯΔ఺͕
    ୯ͳΔϖΞϫΠζͱ͸ҧ͏ɽ
    w ࠷ۙ๣ͷڑ཭ΛऔΔUGOOEJTUBODFϞδϡʔϧΛஶऀΒ͕ެ։ͯ͠Δ
    4Λ4ʹมԽͤ͞ΔͨΊͷίετɺΛ࠷খԽɽ8BTTFSTUFJOڑ཭ͱ΋ɽ
    ͿͬͪΌ͚ͲͬͪͰ΋͏·͘ߦͬͯΔɻ$IBNGFSͷํ͕ПΛٻΊΔඞཁ͕ͳ͘ܭࢉ͕͍ܰ
    

    View Slide

12. ࣮૷ʹؔ͢Δͭͷ࿦఺
    
    ϩε͸Ͳ͏͢Δ
    w ڭࢣ͋ΓֶशΛ͢Δ͕ɺϙΠϯτΫϥ΢υಉ࢜ͷڑ཭͸Ͳ͏΍ͬͯఆٛ
    ͢Δ
    w ෼෍ؒͷڑ཭ $%΍&.%
    Λ࢖͏͜ͱͰ௚઀࠷খԽͰ͖Δ
    ωοτϫʔΫͷܗঢ়͸
    w ग़ྗ͕ը૾͡Όͳ͍͜ͷϞσϧͷωοτϫʔΫ͸Ͳ͏΍ͬͯઃܭ͢Δ
    ֶशσʔλ͸
    w ը૾ͱϙΠϯτΫϥ΢υͷϖΞϫΠζσʔλ͸Ͳ͏΍ͬͯूΊΔ
    ໘ന͍ٞ୊͸ҎԼͷ఺
    

    View Slide

13. 
    ωοτϫʔΫߏ଄͸
    ࠷ߴਫ਼౓ͷϞσϧ·Ͱஈ֊ͷෳࡶ͕͋͞ΔɽͦΕͧΕ͕εʔύʔηοτʹͳͬͯΔ
    

    View Slide

14. 
    ωοτϫʔΫߏ଄͸ -W
    &ODPEFS%FDPEFSϞσϧ
    ग़ྗ͸఺Y%
    ೖྗը૾Ҏ֎ʹϥϯμϜϕΫτϧΛ͍ΕΔ
    w ໰୊͕JMMQPTFEͰ͋Γɼͦͷࣗ༝౓ΛϞσϧԽ͢Δඞཁ
    ͕͋Δ
    w 7BSJBUJPOBM"VUP&ODPEFSͷ࿮૊ΈͰτϨʔχϯά͢Δ
    

    View Slide

15. 
    ωοτϫʔΫߏ଄͸ -W
    %FDPOWΛ௨͞ͳ͍'$ଆ΋࢒ͯ͋͠ΔɽͦΕ
    ͧΕಠཱʹग़ྗͨ͠఺܈Λ݁߹
    w '$͕୺Λॲཧͯ͠$POWଆ͕ຊମΛੜ੒
    ͍ͯ͠Δͱͷใࠂ
    ͋Εɼɼ%FDPOW
    w 8JEUIY)FJHIUY Y Z [
    ͱ͍͏νϟϯ
    ωϧը૾ͱΈͳ͢
    w ཭ࢄ஋Ͱ͸ͳ͍΋ͷͷɼϙΠϯτΫϥ΢
    υ΋ͦΕͳΓʹີ͔ͩΒ͜͏͍͏ߏ଄Ͱ
    ߦ͚ΔΒ͍͠
    w ͤ΍Ζ͔
    

    View Slide

16. 
    ωοτϫʔΫߏ଄͸ -W
    &ODPEFSͱ%FDPEFS͸தؒ૚Ͱ΋ͭͳ͕ͬͯΔ 6/&5෩
    
    $POW &ODPEFS
    %FDPOW %FDPEFS
    ͷ͋ͱʹɺ͞Βʹ$POW%FDPOWΛ܁Γ
    ฦ͢ࣄʹΑͬͯ(MPCBMͱ-PDBMͳಛ௃ΛޮՌతʹ·ͱΊΒΕΔͱ͍ͯ͠Δ
    w ि໨ΛεΩοϓ͢Δͷ΋ೖΕͯΔɻɻɻ΋͏Կ͕ͳΜ΍Β
    w 4UBDLFE)PVSHMBTTOFUXPSLͱݴ͏Β͍͠ɻQPTFਪఆͷطଘݚڀ͕͋Δ
    w ͪͳΈʹ͜ͷ࠷ऴόʔδϣϯ͸$713൛ʹ͸ೖͬͯͳͯ͘BS9JWͰͷ௥Ճ
    ໎࿏ɻɻɻ
    

    View Slide

17. ࣮૷ʹؔ͢Δͭͷ࿦఺
    
    ϩε͸Ͳ͏͢Δ
    w ڭࢣ͋ΓֶशΛ͢Δ͕ɺϙΠϯτΫϥ΢υಉ࢜ͷڑ཭͸Ͳ͏΍ͬͯఆٛ
    ͢Δ
    w ෼෍ؒͷڑ཭ $%΍&.%
    Λ࢖͏͜ͱͰ௚઀࠷খԽͰ͖Δ
    ωοτϫʔΫͷܗঢ়͸
    w ग़ྗ͕ը૾͡Όͳ͍͜ͷϞσϧͷωοτϫʔΫ͸Ͳ͏΍ͬͯઃܭ͢Δ
    w ݁ہ%FDPOWʹམͱ͢͜ͱ͕Ͱ͖Δɽ'$΋ϒϥϯνͱͯ݁͠߹͢Δ
    ֶशσʔλ͸
    w ը૾ͱϙΠϯτΫϥ΢υͷϖΞϫΠζσʔλ͸Ͳ͏΍ͬͯूΊΔ
    ໘ന͍ٞ୊͸ҎԼͷ఺
    

    View Slide

18. ֶशσʔλ͸Ͳ͏͢Δʁ
    
    %ϞσϧͷσʔληοτͷΈΛ࢖͏ɻೖྗը૾͸%ϞσϧΛϨϯμϦϯάͨ͠΋ͷ
    5SBJOJOH%BUB(FOFSBUJPOCZ4ZOUIFTJT
    
    w ͦΕͰ΋ͪΌΜͱ࣮σʔλͰ΋ಈ͘
    w 4IBQF/FUσʔληοτΛར༻
    w ,Ϟσϧ
    w %%ܥͷ෼໺Ͱ͸΋͏͜ΕͰߦ͚Δ͜ͱ͕Α͘Θ͔ͬͯΔ
    w %FQUI΍ϘΫηϧਪఆͰطʹ࢖ΘΕͯΔ
    w ΞϓϦέʔγϣϯͷͱ͜ΖͰड़΂ͨΑ͏ʹɺࠓճ͸ӈਤͷΑ͏ͳΧϝϥͱ෺ମͷҐஔؔ܎Ͱ͔ͳ
    Γ౷੍͞Εͨσʔληοτʹ͍ͯ͠Δ
    w ඞཁʹԠͯ͡র໌৚݅΍ΧϝϥΛ΋ͬͱ͙ͪΌ͙ͪΌʹͰ͖Δ
    

    View Slide

19. ࣮૷ʹؔ͢Δͭͷ࿦఺
    
    ϩε͸Ͳ͏͢Δ
    w ڭࢣ͋ΓֶशΛ͢Δ͕ɺϙΠϯτΫϥ΢υಉ࢜ͷڑ཭͸Ͳ͏΍ͬͯఆٛ
    ͢Δ
    w ෼෍ؒͷڑ཭ $%΍&.%
    Λ࢖͏͜ͱͰ௚઀࠷খԽͰ͖Δ
    ωοτϫʔΫͷܗঢ়͸
    w ग़ྗ͕ը૾͡Όͳ͍͜ͷϞσϧͷωοτϫʔΫ͸Ͳ͏΍ͬͯઃܭ͢Δ
    w ݁ہ%FDPOWʹམͱ͢͜ͱ͕Ͱ͖Δɽ'$΋ϒϥϯνͱͯ݁͠߹͢Δ
    ֶशσʔλ͸
    w ը૾ͱϙΠϯτΫϥ΢υͷϖΞϫΠζσʔλ͸Ͳ͏΍ͬͯूΊΔ
    w %Ϟσϧ͔Β%ը૾Λੜ੒͢Δ͜ͱͰͨ͘͞Μ࡞ΕΔ
    ໘ന͍ٞ୊͸ҎԼͷ఺
    

    View Slide

20. ࣮ݧ
    

    View Slide

21. ઌߦݚڀͱͷൺֱ࣮ݧ
    
    ͜ͷ໰୊ࣗମͷઌߦݚڀ͸ͳ͍
    ͷͰɺߋʹผͷυϝΠϯʹ໭ͯ͠ൺ
    ֱ͢Δ
    w ϙΠϯτΫϥ΢υ͔ΒϘΫηϧ
    ʹม׵্ͨ͠ͰɺϘΫηϧੜ੒
    ख๏ͱಆ͏ʂ
    w ϙΠϯτΫϥ΢υΛ཭ࢄԽͯ͠
    ϘΫηϧͱΈͳ্ͨ͠Ͱɺߋʹ
    ૚ͷ%$POWΛ࢖ͬͯ࠷ऴతͳ
    ϘΫηϧදݱΛ࡞Δ
    w ͱ͍͏͜ͱ͕TVQQMFNFOUBMʹ͠
    ͔ॻ͍ͯͳ͍
    ࢹ఺ೖྗͰ͸ڧ͍ɽͨͩ͠ઌߦ
    ݚڀ%3/͸ෳ਺ࢹ఺ೖྗ͠
    ͯਫ਼౓্͛ΒΕΔڧΈ΋͋Δ
    

    View Slide

22. ͱ͜ΖͰ
    

    View Slide

23. 1PJOU/FU%FFQ-FBSOJOHPO1PJOU4FUTGPS%$MBTTJpDBUJPOBOE
    4FHNFOUBUJPO
    
    ࠓճ঺հͨ͠ϙΠϯτΫϥ΢υͷੜ੒
    ͱಉ࣌ʹɺϙΠϯτΫϥ΢υͷೝࣝ΋
    $713Ͱൃද͞Εͨ
    ಉ༷ʹɺ$POW͕࢖͑ͳ͍͜ͱ͕໰୊
    ͱͳΔ͕ɼ֤఺ΛͦΕͧΕे෼ʹඇઢ
    ܗม׵ͨ͠ޙʹNBYQPPMJOH͢Δ͜ͱ
    Ͱશମͷಛ௃Λଊ͑ΒΕΔͱ͍͏໘ന
    ͍ใࠂɽ
    ৄࡉʹ͍ͭͯ͸ɼ!QFJTVLF͞Μͷε
    ϥΠυ͕ެ։͞ΕͯΔͷͰࢀর͞ΕΔ
    ͱ͍͍ͱࢥ͍·͢ɽ
    w IUUQTXXXTMJEFTIBSFOFU
    'VKJNPUP,FJTVLFQPJOUOFU
    3D఺܈(Point cloud)Λͦͷ··Nx3ͷ࠲ඪྻͱͯ͠ѻ͏ೝࣝ
    ࿦จ'JHΑΓ
    

    View Slide

24. 1PJOU/FU%FFQ-FBSOJOHPO1PJOU4FUTGPS%$MBTTJpDBUJPOBOE
    4FHNFOUBUJPO
    
    ໰୊ઃఆ
    ఺܈Λͦͷ··ೖྗͱͨ͠ಛ௃நग़
    #PYFMͱҧͬͯ੔વͱฒΜͰͳ͍ͷͰɺ
    %$//͕࢖͑ͳ͍
    ΩʔΞΠσΞ
    ֤఺Λಠཱʹඇઢܗม׵ͨ͠ޙʹNBY
    QPPMJOHͱΔ͚ͩͰ݁ߏੑೳ͕ग़Δ
    w ηάϝϯςʔγϣϯͷͱ͖Ͱ΋ɺQPPMJOH
    ͨ݁͠ՌΛݩͷO఺ͷಛ௃ʹDPODBU͢
    Δ͚ͩͰ0,
    ٙ໰఺ɾײ૝
    άϥϑίϯϘϦϡʔγϣϯ͸Ͳ͏ͳΜͰ͢
    ͔ͶɻXLUL
    ਫ਼౓͚ͩͩͱϚϧνϏϡʔ$//͕Ұ൪͍
    ͍ͷ͔͊ɻͰ΋͔֬ʹϚϧνϏϡʔ͕μα
    ͍ͷ͸Θ͔Δ
    3D఺܈(Point cloud)Λͦͷ··Nx3ͷ࠲ඪྻͱͯ͠ѻ͏ೝࣝ
    ࿦จ'JHΑΓ
    

    View Slide

25. 1PJOU/FU%FFQ-FBSOJOHPO1PJOU4FUTGPS%$MBTTJpDBUJPOBOE
    4FHNFOUBUJPO
    
    .7$// ଟࢹ఺͔Βͷը૾$//ͷ౷߹
    ͕Ұ൪ೝࣝਫ਼౓͸ߴ͍Μ͚ͩ
    Ͳɺύϥϝλ਺΋ଟͯ͘ܭࢉ࣌ؒ΋͔͔Δɻ1PJOU/FU͸ਫ਼౓΋े෼Ͱ
    ߴ଎ɻ
    

    View Slide

26. ͓ΘΓ
    

    View Slide

27. ·ͱΊ
    
    ͓΋͠Ζ͍ͱ͜Ζ
    w ෼෍ؒͷڑ཭Λ࠷খԽ͢Δ͜ͱͰ௚઀ϙΠϯτΫϥ΢υΛग़ྗ
    ͢ΔωοτϫʔΫΛֶशͰ͖Δ
    w ݁ہૄͳग़ྗͰ͋ͬͯ΋ɼͳΜͱ͔ີͳख๏ %FDPOW
    Λ͏·͘
    ར༻͢Δߏ଄ʹམͱ͠ࠐΉͷؙ͕͍
    %Ͱ΋%Ͱ΋ɼૄͳσʔλߏ଄ͷֶश͸ɼ໌Β͔ʹ$//ʹ
    ಥͬࠐΊ͹͍͍ΑͶܥͷख๏ΑΓ͸·ͩ·ͩߟ͑Δ͜ͱ͕ͨ͘
    ͞Μ͋ͬͯ໘ന͍
    

    View Slide