第11回 全日本コンピュータビジョン勉強会 CVPR2022 "A Self-Supervised Descriptor for Image Copy Detection"

第11回 全日本コンピュータビジョン勉強会
https://kantocv.connpass.com/event/253626/

紹介論文：A Self-Supervised Descriptor for Image Copy Detection
https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html

Transcript

1. ୈճશ೔ຊίϯϐϡʔλϏδϣϯษڧձ$713ಡΈձ ޙฤ
   
   "4FMG4VQFSWJTFE%FTDSJQUPS
   GPS*NBHF$PQZ%FUFDUJPO
   2022/08/21 Yamato OKAMOTO
   

   View Slide

2. Ԭຊେ࿨ʢ:BNBUP0,".050ʣ
   w ژେʢඒೱݚʣˠΦϜϩϯˠ-*/&
   w ݚڀ։ൃ͔Βࣄۀ૑଄·Ͱखֻ͚ͯ·ͨ͠
   w $PNQVUFS7JTJPO-BCͷ্ཱͪ͛ਓͱͯ͠-*/&ʹస৬
   w ͓͔͛͞·Ͱ$7-͸प೥Λܴ͑·ͨ͠ʙ🎉🎉🎉
   w ಌΕΒΕΔݚڀڌ఺Λ࡞Δ΂͘ɺژ౎͔Βຖ೔ָ͘͠ฃಆͯ͠·͢ʂʂ
   

   View Slide

3. ຊ೔ͷτϐοΫ
   Image Copy Detection
   

   View Slide

4. ಥવͰ͕͢ɺ͜Μͳܦݧ͋Γ·ͤΜ͔ʁʁ
   4/4ʹ౤ߘͨࣸ͠ਅ͕ແஅͰ࢖ΘΕ͍ͯͨʂʂ
   Αͬ͠Όɺࣸਅࡱͬͯ΋Ζͨ͠ɺ
   4/4ʹ౤ߘͨ͠Ζͬʂ
   Μʜʁ͋ͷը૾ɺΘ͠ͷͪΌ͏͔ʂʁ
   ແஅͰ޿ࠂૉࡐͱͯ͠࢖ΘΕͯΔ΍Μʂʂ
   ˞ϑΟΫγϣϯͰ͢
   >∩( ^ ω ^)∩< 📲 Σ ƅЧƅ|||)
   ઌண໊ʹ
   ͚ͩڭ͑·͢
   ͍·͙͢ສԁՔ͙ํ๏
   ͙͢ΫϦοΫ
   ޙ೔
   ͜͜·Ͱώυ͍ྫͰ͸ͳ͘ͱ΋ɺࢲ͸෦׆ಈͰࡱӨͨ͠ϥάϏʔͷࢼ߹ࣸਅΛɺ
   νϥγͷഎܠը૾ͱͯ͠ແஅར༻͞Εͨܦݧ͕͋Γ·͢😱
   

   View Slide

5. .PUJWBUJPOɿίϐʔը૾Λݟ͚ͭग़͍ͨ͠
   ແஅͷը૾ར༻͸͝๏౓ʹ΋͔͔ΘΒͣɺͦͷΑ͏ͳ໰୊͸ઈ͑ͳ͍ɻ
   ྫ͑͹ԼਤͷΑ͏ʹɺ৽ن౤ߘ͞Εͨը૾͕ίϐʔը૾ͳͷ͔Ͳ͏͔ϞχλϦϯά͢Δػೳ͕ٻΊΒΕΔɻ
   ৽ن౤ߘʢ2VFSZʣ
   DB
   ίϐʔݩͱࢥΘ͖͠ը૾͕ଘࡏ
   ݕࡧ
   DB
   ෆਖ਼ར༻͞Ε͔ͨ΋
   ໰୊ͳ͍ͱ൑அ
   https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
   ৽ن౤ߘʢ2VFSZʣ )JUͳ͠
   ݕࡧ
   

   View Slide

6. Ͳ͏΍ͬͯίϐʔը૾ݕग़Λ࣮ݱ͢Δ͔ʁʁ
   ίϐʔը૾ͱίϐʔݩը૾͔Β͸ɺྨࣅͨ͠ಛ௃ྔΛநग़Մೳͳ͸ͣʂʂ
   ΠέͯΔಛ௃ྔදݱΛֶश͢Ε͹ɺಛ௃ྔۭؒͰίϐʔݩͷը૾͕ۙ๣ͱͳΓݕग़Ͱ͖ΔʂʂʢԾઆʣ
   https://github.com/google-research/simclr
   
   
   https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
   45&1ɿΠέͯΔಛ௃ྔදݱΛֶश͢Δ 45&1ɿը૾Λಛ௃ྔۭؒʹࣹӨ͢Δ
   45&1ɿۙ๣ʹσʔλ͕ଘࡏ͢Ε͹2VFSZΛίϐʔͩͱ൑ఆ͢Δ
   ྫ͑͹ɺ4JN$-3ͰେྔͷࣗݾڭࢣֶशΛ͢Δ
   DMPTF
   

   View Slide

7. ݁ՌɿͦΕ͚ͩͰ͸μϝͰͨ͠ʢɾТɾʆʣ
   • ͳͥίϐʔը૾ͱݩը૾͕࠷ۙ๪ʹͳΒͳ͔͔ͬͨʁ
   - ॻ͖ࠐΈจࣈɺ഑৭ɺղ૾౓ͳͲͷཁૉ͕ྨࣅ͢Δ
   ଞͷը૾͕࠷ۙ๣ʹͳͬͯ͠·ͬͨ
   • *NBHF$PQZ%FUFDUJPOͷ೉͠͞
   - Ճ޻͞Εͯ΋ݩը૾͕ಉ͡ͳΒಛ௃ྔ͕ྨࣅ͢ΔΑ
   ͏ʹ͍ͨ͠
   - ͍͔ʹɺͦͷΑ͏ͳಛ௃ྔදݱΛֶश͢Δ͔ʁ
   ࣮ࡍʹ4JN$-3Ͱಛ௃ྔදݱΛֶशͯ͠ɺίαΠϯྨࣅ౓Ͱۙ๣ը૾Λநग़ͨ݁͠Ռ͕ӈԼɻ
   ը૾Ճ޻Ͱಛ௃ྔ͕एׯԕ͔ͬͨ͟෼ɺଞͷը૾͕࠷ۙ๣ʹͳͬͯ͠·ͬͨͱਪ࡯͞ΕΔɻ
   ͜ΕΛݕग़͔ͨͬͨ͠
   https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
   

   View Slide

8. ͦ͜Ͱɺ4JN$-3Λϕʔεʹ%FTDSJQUPSͷֶशख๏ΛఏҊʂʂ
   https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
   

   View Slide

9. ิ଍ɿࠓ೔ͷൃදͰ͸%FTDSJQUPSΛGFBUVSFWFDUPSͷҰछͱͯ͠ղऍ͠·͢ɻ
   ʢwebliࣙॻΑΓҾ༻ʣ
   
   
   هड़ࢠʢಡΈํɿ͖͡Ύͭ͠ʣ
   
   
   ʲӳʳdescriptor
   
   
   هड़ࢠͱ͸ɺϓϩηε͕OSΛ௨ͯ͡ೖग़ྗૢ࡞Λ
   ߦ͏ࡍʹࢀর͞ΕΔɺϑΝΠϧͷߏ଄΍಺༰ͷཁ
   ໿ɺϑΝΠϧଐੑͳͲ͕ه͞Εͨ੔਺ͷ͜ͱͰ͋
   ΔɻϓϩάϥϜͷߏ଄͕ه͞ΕΔ৔߹΋͋Δɻ
   
   
   ༗ࣝऀίϝϯτʢ˞ҹ৅ͷ࿩Ͱ͢ʣ
   ࿦จͷΞϒετ
   
   
   https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
   

   View Slide

10. ิ଍ɿ4JN$-3ͱ͸ಛ௃ྔදݱΛ֫ಘ͢ΔͨΊͷରরֶशͷ̍ͭ
    https://github.com/google-research/simclr
    4JN$-3Ͱ͸ɺಉ͡ը૾ʹҟͳΔม׵Λࢪͯ͠΋ྨࣅͨ͠ಛ௃ྔ͕நग़Ͱ͖ΔΑ͏ʹֶश͢Δɻ
    ͦΕͱಉ࣌ʹɺҟͳΔը૾͔Β͸ҟͳΔಛ௃ྔ͕நग़Ͱ͖ΔΑ͏ʹֶश͢Δɻ
    ͳ͓ɺม׵ʹ͸$301΍৭ม׵΍#MVS͕Α͘࢖ΘΕΔɻ
    ಉ͡ը૾͔Β͸ಉ͡ಛ௃ྔΛ֫ಘ͍ͨ͠ ҟͳΔը૾͔Β͸ҟͳΔಛ௃ྔΛ֫ಘ͍ͨ͠
    4JN$-3ͷଛࣦؔ਺͸*OGP/$&
    ɾT͸ίαΠϯྨࣅ౓
    ɾ1͸1PTJUJWF1BSFͷू߹
    QPTJUJWFQBSF
    OFHBUJWFQBSF
    

    View Slide

11. ఏҊख๏ͷશମ૾
    "Self Supervised Copy Detection (SSCD)
    "
    
    

    View Slide

12. "Self Supervised Copy Detection (SSCD)"ͷશମ૾
    4JN$-3Λϕʔεʹෳ਺ͷվྑΛࢪͯ͠ɺ$PQZ%FUFDUJPOʹಛԽͨ͠ಛ௃ྔදݱΛֶशͨ͠ɻ
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    &OUSPQZ-PTTΛ௥Ճͨ͠
    4JNJMBSJUZ/PSNBMJ[BUJPOΛ௥Ճͨ͠
    "WFSBHF1PPMJOHͰ͸ͳ͘
    (F.1PPMJOHΛ࠾༻
    $//ޙͷ.-1Λઢܗม׵ʹஔ͖׵͑ͨ
    ന৭ԽΛಋೖ
    ը૾ม׵ͷύλʔϯΛ௥Ճͨ͠
    

    View Slide

13. ޻෉͕ͯΜ͜੝ΓͳͷͰɺ
    ʮ͕͜͜ϙΠϯτͩʂʯͱࢥͬͨՕॴͷΈൈਮ͠·͢
    

    View Slide

14. &OUSPQZ-PTTʹΑΓෛྫΛԕ͚͟Δ੍໿ΛڧΊͨ
    &OUSPQZ-PTTʹΑͬͯɺ࠷΋ۙ͘ʹ͋ΔෛྫαϯϓϧΛԕ͚͟ΔΑ͏ʹֶशͤ͞Δɻ
    ͜ΕʹΑΓɺਖ਼ྫಉ࢜ͷڑ཭෼෍͸ͦͷ··ʹɺෛྫͱͷڑ཭෼෍Λԕ͚͟Δ͜ͱʹ੒ޭͨ͠ɻ
    ɾ[͸ಛ௃ྔ
    ɾ1͸ਖ਼ྫϖΞͷू߹
    ੨͕ਖ਼ྫͱͷڑ཭෼෍ɺ੺͕ෛྫͱͷڑ཭෼෍
    &OUSPQZ-PTTಋೖલʢ্ʣʹൺ΂ͯɺಋೖޙʢԼʣ͸ෛྫ͕ԕ͔͍ͬͯ͟Δ
    *OGP/$&ͱ߹Θͤͯ࢖༻͢Δͱɺ
    ͓ؾ࣋ͪతʹ͸USJQMFUMPTTʹ͍͔ۙ΋ʁ🤔
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

15. ิ଍ɿ&OUSPQZ-PTTͱλεΫͷ૬ੑ
    &OUSPQZ-PTTʹΑͬͯෛྫͱͷڑ཭෼෍Λԕ͚͟Δͱɺ$PQZ%FUFDUJPOͰ͸͝རӹ͕͋ͬͨɻ
    ͔͠͠ɺྫ͑͹$MBTTJGJDBUJPOͰ͸ٯޮՌͩͬͨͱใࠂ͞Ε͍ͯΔɻ
    ԣ࣠Е͸&OUSPQZ-PTTΛͲΕ͚ͩڧ͘͢Δ͔ɻ
    ੨͸*NBHF/FUͷ$MBTTJGJDBUJPOͷਫ਼౓ˠԼ͕͍ͬͯΔ
    ྘ͱԫ͸$PQZ%FUFDUJPOͷਫ਼౓ˠ্͕͍ͬͯΔ
    &OUSPQZ-PTT͸$PQZ%FUFDUJPOʹ͸༗ޮ͕ͩɺ
    $MBTTJGJDBUJPOʹ͸ٯޮՌͱͳͬͨɻ
    (FOFSBMͳಛ௃ྔΛ֫ಘͨ͠ͱ͍͏ΑΓɺ
    ಉҰੜ൑ఆʹಛԽͨ͠ͷ͔΋ʁ🤔
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

16. $//ޙͷ.-1Λઢܗม׵ʹஔ͖׵͑ͨ
    .-1Λઢܗม׵ʹஔ͖׵͑Δͱɺ
    ҟͳΔը૾ม׵Λͯ͠΋ݩը૾͕ಉ͡ͳΒ͹ɺ$//͕ಉ͡ಛ௃ྔΛநग़͠Α͏ͱͯ͘͠ΕΔɻ
    w .-1Ͱ͸USBOTGPSNBUJPODPWBSJBOUGFBUVSFͷ֫ಘ͕ظ଴͞ΕΔɻ
    w ͜ΕʹΑΓɺಉ͡ը૾ʹҟͳΔը૾ม׵Λͨ͠ͱ͖ɺԾʹ$//͕ҟͳΔಛ௃ྔΛந
    ग़ͯ͠΋ɺ.-1ͷ͓͔͛Ͱ࠷ऴతʹಉ͡Α͏ͳಛ௃ྔ͕நग़Ͱ͖Δɻ
    w ݴ͍׵͑Δͱɺ.-1͕Χόʔͯ͘͠ΕΔͷͰɺಉ͡ը૾ʹҟͳΔը૾ม׵Λͨ͠ͱ
    ͖ɺ$//͸ҟͳΔಛ௃ྔΛநग़͍ͯ͠ΔՄೳੑ͕͋Δɻ
    w ීஈͳΒͦΕͰ΋͍͍͕ɺࠓճͷλεΫͰ͸JOWBSJBOU͕๬·͍͠ɻ
    w ಉ͡ը૾ʹҟͳΔը૾ม׵Λͯ͠΋ಉ͡ಛ௃ྔΛநग़͍ͨ͠ɻ
    w ͦ͜ͰɺࠓճͷλεΫͰ͸.-1Λઢܗม׵ʹஔ͖׵͑Δɻ
    w ಉ͡ը૾ʹҟͳΔը૾ม׵Λͯ͠΋ɺಉ͡ಛ௃ྔΛநग़ͤ͞Δૂ͍ɻ
    w $//͸ը૾ม׵͕͞Εͯ΋ɺݩ͕ಉ͡ը૾ͳΒಉ͡ಛ௃ྔΛநग़͠ͳ͍ͱ͍͚ͳ͘
    ͳΔɻʢ.-1͸$//Λ؁΍͔͔ͨ͠΋͠Εͳ͍͕ɺઢܗม׵͸༰ࣻͤΜŕʂʣ
    w ࠷ऴతʹར༻͢Δಛ௃ྔʹ͸ީิ͕͋ΓɺఏҊख๏44$%Ͱ͸ᶄΛར༻ͨ͠ɻ
    w ᶃ$//ग़ྗʢUSVOLGFBUVSFʣ
    w ᶄઢܗม׵ͨ͠΋ͷʢQSPKFDUJPOGFBUVSFʣ
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

17. ิ଍ɿDPWBSJBOUͱJOWBSJBOU
    5XJUFSͰरͬͨਤ͕Θ͔Γ΍͔ͬͨ͢ͷͰܝࡌ͓͖ͯ͠·͢
    IUUQTUXJUUFSDPN"LJP5PNJZBTUBUVT TUI00);8,1HH4KD"$BC@Y2
    https://twitter.com/AkioTomiya/status/1404844708099198982?s=20&t=hOOHZWK7Pg8gSjcACab_xQ
    

    View Slide

18. 4DPSF4JNJMBSJUZ/PSNBMJ[BUJPOΛಋೖͨ͠
    ಛ௃ྔۭؒʹࣹӨͨ͠ͱ͖ɺͲΕ͘Β͍ͷڑ཭ͳΒ͹ʮ͍ۙʯͱݴ͑Δͷ͔ʁ
    ͜ͷई౓͕σʔλʹґଘͤͣɺҰ؏ੑΛ࣋ͭΑ͏ʹਖ਼نԽͯ͠΍Δɻ
    w Ͳ͏΍ͬͯਖ਼نԽ͢Δͷ͔ʁʁ
    

    ɹˠैདྷͷίαΠϯྨࣅ౓ʹόΠΞεΛ଍͢
    w όΠΞε͸ۙ๣αϯϓϧͱͷڑ཭ฏۉʹΑͬͯࢉग़͢Δ
    w ࠓճ͸όΠΞεࢉग़ઐ༻ͷCBDLHSPVOEEBUBTFUΛ༻ҙͨ͠ʢόΠΞεࢉग़ʹSFGFSFODFը૾͸࢖Θͳ͍ʣ
    w R͸ΫΤϦʢྫ͑͹৽ن౤ߘ͞Εͨը૾ʣ
    w S͸ϦϑΝϨϯεʢྫ͑͹౤ߘࡁΈͷը૾ʣ
    w T͸ίαΠϯྨࣅ౓
    w ࠷ۙ๪ͷOdOFOE൪໨·ͰͷαϯϓϧΛར༻
    w ЌͰOPNBMJ[BUJPOͷ͔͔Γ۩߹Λௐ੔
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

19. ը૾ม׵ͷύλʔϯΛ௥Ճͨ͠
    4JN$-3Ͱѻ͍ͬͯͨύλʔϯʹɺ͞Βʹ4USPOH#MVSɺ3PUBUJPOͳͲͷύλʔϯΛ௥Ճͨ͠ɻ
    UFYUॲཧͷ௥Ճ͸ɺσʔληοτ%*4$ʹ
    ߹Θͤʹߦ͍ͬͯΔΑ͏ͳؾ΋͢Δ͕ɺɺɺ🤔
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

20. ࣮ݧͱ݁Ռ
    

    View Slide

21. ࣮ݧͷ֓ཁ
    σʔληοτɿ%*4$
    w ֶश༻ʹສຕɺධՁ༻ʹສຕʢRVFSZʣʴສຕʢSFGFSFODFʣ
    w RVFSZը૾ͷ͏ͪສຕ͸SFGFSFODFը૾Λฤूͯ͠࡞੒ͨ͠DPQZͰ͋Δɻ
    λεΫ
    w SFGFSFODFը૾Λࢀরͯ͠ɺRVFSZը૾͕DPQZ͔൱͔Λ൑ఆ͢ΔλεΫɻ
    ධՁࢦඪɿNJDSPBWFSBHFQSFDJTJPO Ÿ"1
    
    w αϯϓϧ୯ҐͰ"WFSBHF1SFDJTJPOͷฏۉΛࢉग़ͨ͠΋ͷɻ
    RVFSZ SFGFSFODF
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

22. ࣮ݧ݁Ռɿ·ͣ͸4P5"
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

23. ߟ࡯ɿ&OUSPQZ-PTTͱ4DPSF/PSNBMJ[BUJPO͕ಛʹޮ͍͍ͯΔ༷ࢠ
    w4JN$-3͕ϕʔεϥΠϯ
    wUSVOL͸$//ͷग़ྗΛར༻
    wQSPK͸ઢܗม׵ͨ͠΋ͷΛར༻
    wϕʔεϥΠϯͱఏҊख๏ͷதؒతଘࡏͱͯ͠
    4JN$-3$%΋ධՁ
    w͜Ε͸ɺ&OUSPQZ-PTTҎ֎ͷఏҊख๏Λ
    

    ద༻ͨ͠΋ͷʢ˞.-1ஔ͖׵͑΍(F.1PPMʣ
    w4JN$-3$%ˠ44$%΁ͷมԽɺ
    

    ͭ·Γ&OUSPQZ-PTTಋೖ͚ͩͰ
    

    Ж"1͕໿޲্ʢˠʣ
    wЖ"14/͸4DPSF/PNBSJ[BUJPOΛ
    

    ಋೖͨ͠৔߹
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    

    View Slide

24. "CMBUJPOT
    (F.1PPMJOH΍4USPOH#MVSͳͲɺ͍ͣΕͷ޻෉΋ޮՌ͕͋ͬͨ͜ͱΛ֬ೝͰ͖͍ͯΔɻ
    ಛච͢΂͖͸ɺQSPKFDUJPOGFBUVSF͸&OUSPQZ-PTTͱηοτͰಛʹ༗ޮͱͳΔ఺ɻ
    https://openaccess.thecvf.com/content/CVPR2022/html/Pizzi_A_Self-Supervised_Descriptor_for_Image_Copy_Detection_CVPR_2022_paper.html
    DPWBSJBOUͱJOWBSJBOUͷٞ࿦Λ౿·͑Δͱɺ
    4JN$-3ͰUSVOLͷํ͕ߴੑೳͳͷ͸Կނʁ🤔
    

    View Slide

25. ·ͱΊ
    w RVFSZը૾͕༩͑ΒΕͨͱ͖ɺSFGFSFODFը૾Λࢀরͯ͠ίϐʔ͔൱͔൑ఆ͢Δ
    *NBHF$PQZ%FUFDUJPOʹ௅ઓ
    w 4JN$-3Λϕʔεʹɺ&OUSPQZ-PTTͱ͍ͬͨ޻෉Λಋೖͯ͠ੑೳվળͨ͠
    ߟ࡯
    w ఏҊख๏͸ޮՌత͕ͩͬͨʮͦΕ͸Կނʁʯͱ͍͏ղ໌͕΋ͬͱਐΉͱخ͍͠
    

    View Slide