Upgrade to Pro — share decks privately, control downloads, hide ads and more …

atmaCup#11 1st place solution

18f59d026896c6eb578c01f698192164?s=47 ishikei
August 06, 2021

atmaCup#11 1st place solution

18f59d026896c6eb578c01f698192164?s=128

ishikei

August 06, 2021
Tweet

Transcript

  1. 2021/8/5 @ishikei4 atmaCup#11 1st place solution & 忙しい⼈のための画像コンペとの向き合い⽅

  2. ࣗݾ঺հ • ੴ ܓҰ࿠ 5XJUUFS!JTIJLFJ • &OHJOFFS!3JTU*OD • ,BHHMF.BTUFS •

    1"1",BHHMFS ່ࡀ
  3. ໨࣍  TUQMBDFTPMVUJPO  ʢ͓·͚ʣ๩͍͠ਓͷͨΊͷը૾ίϯϖͱͷ޲͖߹͍ํ

  4. atmaCup#11 1st place solution

  5. TUQMBDFTPMVUJPO %JTDVTTJPO • IUUQTXXXHVSVHVSVTDJFODFDPNQFUJUJPOTEJTDVTTJPOTEBBGDFBDCDCF $PEF • IUUQTHJUIVCDPNJTIJLFJLBUNB@TU@TPMVUJPO ஫ҙ఺ • ग़ͯ͘Δख๏ͷࡉ͔͍આ໌͸هࡌ͍ͯ͠·ͤΜ

    • ख๏ͦͷ΋ͷΑΓɺͳͥͦΕΛબΜͩͷ͔ɺͲΜͳํ਑Ͱಈ͍ͨͷ͔ɺ ͋ͨΓͷઆ໌͕ଟΊͰ͢
  6. 0WFSWJFX 1IPUPT 1SFQSPDFTT .PEFM5SBJOJH 4UBDLJOH 1PTUQSPDFTT • $7 • 1VCMJD

    • 1SJWBUF 'JOBM4VC *NBHF1BEEJOH • ը૾ͷ௕ลʹ߹Θͤͯਖ਼ํܗʹ 1BEEJOH͔ͯ͠Β3FTJ[F %*/0QSFUSBJOJOH 'JOFUVOJOH • SFTOFUE 7J5@TNBMM 7J5@CBTF • 3FHSFTTJPO 4DSBUDI USBJOJOH • SFTOFUE • 3FHSFTTJPO $MBTTJGJDBUJPO -JHIU(#. • $MBTTJGJDBUJPOͷ৔߹͸֤MBCFM ͷQSPCBMJMJUZ΋ೖྗ OQDMJQ QSFE  
  7. 44- 44-ɿ 4FMG4VQFSWJTFE-FBSOJOH ࣗݾڭࢣ͋Γֶश • ϥϕϧͳ͠ը૾Ͱֶशͯ͠ɺಛ௃දݱΛ֫ಘ͢Δख๏ • *NBHF/FUQSFUSBJOFE͕࢖͑ͳ͍͜ͱ΁ͷରԠ %*/0ɿTFMG%*TUJMMBUJPOXJUI /0

    MBCFMT ϥϕϧͳࣗ͠ݾৠཹ • UFBDIFS TUVEFOUͭͷϞσϧͰৠཹ ͨͩ͠UFBDIFS͸TUVEFOUͷҠಈฏۉ
  8. 44- ͳͥ%*/0Λબ୒ͨ͠ʁ • ެ࣮ࣜ૷ͷ3&"%.&͕ॆ࣮͍ͯͯͦ͠ͷ··ϩʔΧϧͰಈ͔͠΍͍͢ ˠޮՌ֬ೝ·Ͱͷ͕࣌ؒ୹ͯ͘͢Ή • ίϯϖॳظ͔Β%JTDVTTJPOʹ্͕͍ͬͯͨ4JN4JBN͸͏·ֶ͘शͤ͞ΒΕͳ͔͕ͬͨɺ ͖ͪΜͱಈ͚͹44-ͷ֤ख๏Ͱͦ͜·Ͱେ͖ͳҧ͍͸ͳ͍Μ͡Όͳ͍͔ɺͱ༧૝ ˠΑΓ؆୯ʹ֬ೝͰ͖Δख๏Λબ୒͔ͨͬͨ͠ʢͥΜͿࢼ࣌ؒ͢͸ͳ͔ͬͨ😇ʣ

  9. 44- • .PEFMSFTOFUE 7J5@TNBMM 7J5@CBTF • &QPDIͷΈຊλεΫͷਫ਼౓Λ֬ೝ͠ͳ͕Βˠʹ૿΍ͨ͠ • 44-QSFUSBJOXFJHIUΛ࢖༻ͯ͠'JOFUVOJOH ॏΈݻఆͳ͠

    FQPDI Model SSL epoch SSL loss CV Public Private ViT_small_patch16 100 7.587 0.7422 0.7154 0.7002 ViT_small_patch16 300 3.814 0.6890 0.6574 0.6408
  10. 44- • 44-ͷFQPDI਺Λ͞Βʹ૿΍ͤ͹ɺ΋ͬͱ TJOHMFNPEFMͷਫ਼౓͸վળͰ͖͔ͨ΋ • ͨͩɺࠓճ͸TUBDLJOHͰׂͱείΞ্͕͕Δ ײ͕֮͋ͬͨͨΊɺTJOHMFNPEFMͷ௥ٻ͸ ఔʑʹͯ͠Ϟσϧͷ਺Λ૿΍͢ํ਑Ͱ࣮ݧΛ ਐΊͨ 44-ͷUSBJOJOH@MPTT

    FQPDI਺Λ૿΍͢΄Ͳ؇΍͔ʹݮগ͍ͯ͘͠ҹ৅
  11. *NBHF1BEEJOH • ը૾ͷॎԣൺ͕೥୅ʹΑͬͯۇ͔ʹ ภ͍ͬͯΔΑ͏ʹݟ͑ͨ • ը૾ͷ௕ลʹ߹Θͤͯਖ਼ํܗʹ QBEEJOH͔ͯ͠Β໨తͷղ૾౓ʹ 3FTJ[F • ͜ΕʹΑͬͯಉ͡QJYFM

    TJ[FͰ΋ *NBHF1BEEJOHΛೖΕͨ΄͏͕ είΞ͕վળͨ͠ Model Img Padding CV Public Private resnet18d - 0.6779 0.6758 0.6522 resnet18d ✓ 0.6674 0.6609 0.6474
  12. *NBHF1BEEJOH • 5BSHFU • TJ[FY • 5BSHFU • TJ[FY •

    5BSHFU • TJ[FY ݩը૾ ͦͷ··3FTJ[F *NBHF1BEEJOH
  13. 4DSBUDI5SBJOJOH • ࠷ॳ͸ %JTDVTTJPOʹ্͕͍ͬͯͨ 4JN4JBNΛࢼ͍͕ͯͨ͠શ͘͏·͘ ͍͔ͣ • ίϯϖத൫·Ͱ͸4DSBUDIֶशͷ &GGJDJFOU/FU# 3FT/FUEΛ

    ࢖͍ͬͯͨ • ʢ࠷ऴTVCʹ͸4DSBUDIֶशͷ 3FT/FUEͷΈ࢖༻ʣ 5SBJOJOH1BSBNFUFS • FQPDI • PQUJNJ[FS"EBN8 XFJHIUEFDBZ • MFBSOJOHSBUF $PTJOFEFDBZ • JNBHF@TJ[FY • CBUDI@TJ[F "VHNFOUBUJPO • )PSJ[POUBM'MJQ • 4IJGU4DBMF3PUBUF • #MVS • 3BOEPN#SJHIUOFTT$POUSBTU • .JYVQ $VUNJY
  14. 4DSBUDI5SBJOJOH ͳͥFQPDIʁ • ࠓճɺ*NBHF/FUQSFUSBJOΛ࢖Θͣʹ TDSBUDIͰϞσϧΛֶश͢Δͷ͕ॳΊͯ ͩͬͨͨΊɺUJNNͷ5SBJOJOH&YBNQMFTͷ ϖʔδΛ͔ͳΓࢀߟʹͨ͠ • CBUDI@TJ[F͸Ҏ֎ͱେ͖͘ͳͯ͘େৎ෉ 

    FQPDI਺͸ࢥͬͯͨΑΓ͔ͳΓ௕͘ͱΔɺ ͳͲצॴ͕Θ͔ͬͯͦͷޙͷ࣮ݧ͕ޮ཰తʹ ਐΊΒΕͨ
  15. 4DSBUDI5SBJOJOH ֶशۂઢ • SFTOFUE JNH@TJ[FY  SFHSFTTJPO 44-ͳ͠ • $71VCMJD

    1SJWBUF • ·ͩFQPDI਺৳͹ͤΔؾ΋͢Δ
  16. 3FHSFTTJPO WT$MBTTJGJDBUJPO • جຊ͸3FHSFTTJPOΛ࢖͍͕ͬͯͨɺ͍͔ͭ͘$MBTTJGJDBUJPOϞσϧ΋ֶश • ͦ͜·Ͱେ͖ͳҧ͍͸ͳ͔͕ͬͨɺޙ͔Β֬ೝ͢ΔͱTJOHMFCFTU͸$MBTTJGJDBUJPO ͩͬͨΈ͍ͨ Model SSL Img

    size CV Public Private ViT_base_patch16 ✓ 224x224 0.6759 0.6400 0.6306 • $MBTTJGJDBUJPOͷ৔߹͸ͪ͜Βͷ%JTDVTTJPOʹ ͋ΔΑ͏ʹ༧ଌ஋Λ࿈ଓ஋ʹม׵
  17. 4UBDLJOH • -JHIU(#.Λ࢖ͬͨ4UBDLJOH • 3FHSFTTJPO $MBTTJGJDBUJPO૒ํͷϞσϧΛ࢖͏͕ɺ$MBTTJGJDBUJPOʹؔͯ͠͸ɺ ֤MBCFMͷQSPCBCJMJUZ΋ೖྗͱͯ͠Ճ͑ͨ 0 1 2

    3 5BSHFU 0.3 1.2 2.2 2.9 .PEFM 3FHSFTTJPO 0.1 1.3 2.1 2.8 .PEFM $MBTTJGJDBUJPO QSFE QSFE 0.8 0.1 0.0 0.0 QSPCB@    0.1 0.65 0.1 0.1 0.05 0.2 0.7 0.3 0.05 0.05 0.2 0.6 ➕ ɿ -JHIU(#. 4UBDLJOH
  18. 4UBDLJOH ͳͥ4UBDLJOHʁ • ॳظʹ&OTFNCMFͱ4UBDLJOH྆ํࢼ͍ͯͨ͠ˠͦͷ࣌఺Ͱ૒ํʹਫ਼౓ࠩ͸ͳ͔ͬͨ • $7-#ͷ૬͕ؔߴ͔ͬͨ͜ͱɺίϯϖͷੑ࣭ɺओ࠵ऀͷੑ֨ͳͲ͔Β1VCMJD1SJWBUF ΋ׂͱ૬͍ؔͯ͠ΔͩΖ͏ɺͱ༧ଌ ˠ$7ʹ0WFSGJUUJOH͠΍͍͢4UBDLJOHͰ΋-#Ͱਫ਼౓Λอͬͯ͘ΕΔͱ౿ΜͰ࠾༻ • ݁ہɺͦͷޙ$MBTTJGJDBUJPOͷQSPCBCJMJUZΛՃ͑ͨ͜ͱͰείΞ΋CPPTUͨ͠͠ྑ͍൑அ

    ͩͬͨ
  19. TPMVUJPO·ͱΊ • .PEFM3FT/FUE 7J5@TNBMM 7J5@CBTF • %*/0QSFUSBJO FQPDI 'JOFUVOJOH4DSBUDIUSBJOJOJH MPOHFQPDI

    • *NBHF1BEEJOH • 3FHSFTTJPO$MBTTJGJDBUJPO • -(#.ͰTUBDLJOH $MBTTJGJDBUJPO͸֤MBCFMͷQSPCBCJMJUZ΋ೖྗ
  20. (おまけ)忙しい⼈のための画像コンペとの向き合い⽅

  21. ʢ͓·͚ʣ๩͍͠ਓͷͨΊͷը૾ίϯϖͱͷ޲͖߹͍ํ ࿩͢͜ͱ • ීஈࣗ෼͕ը૾ίϯϖ΍Δͱ͖ʹؾʹͯ͠Δ͜ͱ • ಛʹɺ৭ʑͳࣄ৘ʹΑΓͳ͔ͳ͔ίϯϖ΍Δ͕࣌ؒͱΕͳ͍ਓ޲͚ ஫ҙ఺ • ҰԠը૾ίϯϖʹߜͬͯ࿩͠·͢ •

    ʢଞͷυϝΠϯͷίϯϖͰ΋ڞ௨ͳ෦෼͸͋Δ͔΋஌Ε·ͤΜ͕ʣ • ͋͘·Ͱ෼ੳίϯϖͱ͍͏ʮωοτήʔϜʯͰͲ͏্ҐΛૂ͏͔ɺʹয఺ ʢ࣮຿ͱ͸ผͷ؍఺ʣ
  22. ը૾ίϯϖಛ༗ͷ೰Έ Ұճͷֶशʹ͕͔͔࣌ؒΔͨΊɺࢼߦճ਺Λ૿΍ͤͳ͍ • ݸਓతʹ͸͜Εʹ͖ͭΔ • ׬શʹϞσϧͷֶशʹੜ׆ϦζϜΛ߹ΘͤΒΕΕ͹ྑ͍͕ɺͦ͏΋͍͔ͳ͍ • ࢓ࣄɺֶߍɺҭࣇɺFUD

  23. ը૾ίϯϖಛ༗ͷ೰Έ ྫ͑͹ࠓճͷίϯϖͷ৔߹ • SFTOFUE JNH@TJ[F CBUDI@TJ[F 359࢖༻ͰFQPDIඵ ˠFQPDIYGPME࣌ؒ • ೔Ϟσϧճͤͯ΋िؒͷίϯϖͩͱճ͔͠ϑϧϞσϧΛ࡞Εͳ͍

  24. ը૾ίϯϖಛ༗ͷ೰Έ Ͳ͏͢Δʁ • FQPDIʹ͔͔Δ࣌ؒΛ୹͘͢Δ • ʮ΍Δ͜ͱʯʮ΍Βͳ͍͜ͱʯΛ໌֬ʹ͢Δ

  25. Ͳ͏͢Δʁ FQPDIʹ͔͔Δ࣌ؒΛ୹͘͢Δ • ͓ۚͰͳ͙Δ • (16Λޮ཰తʹ࢖͏ • ྫɿը૾ΛOQZܗࣜͰ࣋ͬͨΓϝϞϦʹࡌ͓͍ͤͯͨΓ EBUBTFU಺Ͱ͸ಛ௃͸OQBSSBZͰ࣋ͭɺͳͲ •

    (16Ҏ֎Ͱॲཧʹ͕͔͔͍࣌ؒͬͯΔՕॴΛಛఆ͢Δ • খ͍͞ը૾αΠζɺGPMEͷΈɺҰ෦ͷը૾ͷΈͰ࣮ݧΛߴ଎ʹճ͢ • ͨͩ͠ɺ͋·Γಉ͡GPMEͰͷΈ֬ೝͯ͠ΔͱͦͷGPMEʹͲΜͲΜաֶशͯ͠͠·͏ͷͰ΄Ͳ΄Ͳʹ • ؾΛ͚ͭͳ͍ͱɺࢼߦճ਺㲈TIBLF֬཰ʹͳͬͯ͠·͍͕ͪʢͳؾ͕͍ͯ͠Δʣ
  26. Ͳ͏͢Δʁ ʮ΍Δ͜ͱʯʮ΍Βͳ͍͜ͱʯΛ໌֬ʹ͢Δ • ཧ૝తʹ͸શ෦ࢼͯͦ͠ͷத͔Βબ୒͍ͨ͠ʢ͕ɺͦΜͳ࣌ؒ͸ͳ͍ʣ • ʮޮ͖ͦ͏ͳ͜ͱʯʮޮ͔ͳͦ͏ͳ͜ͱʯΛίϯϖͷσʔλ΍എܠ͔Βߟ࡯ʢໝ૝ʣ ͨ͠ΓɺͦͷͨΊͷॳظݕ౼ʹ࣌ؒΛ͔͚Δ

  27. Ͳ͏͢Δʁ ίϯϖং൫Ͱؾʹ͍ͯ͠Δ͜ͱ • ίϯϖಛ༗ͷੑ࣭͸Կ͔ • ࠓճͳΒ.BUFSJBMTͳͲͷ৘ใɺ֎෦σʔλېࢭɺը૾ൺ཰ʁ • ৭ʑࢼͯ͠ແཧͦ͏ͳΒ͙͢ଞͷํ਑ʹ੾Γସ͑Δ • ʢ݁ہ.BUFSJBMTͳͲͷ৘ใ͏·͘࢖͑·ͤΜͰͨ͠ʣ

    • &OTFNCMF 4UBDLJOHͰͷ্͕Γ෯͸ͲΕ͘Β͍͔ʁ • TJOHMFNPEFMʹݻࣥͨ͠ํ͕ྑ͍ͷ͔ɺϞσϧͷଟ༷ੑΛॏࢹͨ͠ํ͕ྑ͍ͷ͔ • ࣗ෼͸ͭҎ্͍͍ײ͡ͷϞσϧ͕Ͱ͖ͨ࣌఺Ͱ֬ೝ͍ͯ͠Δ • ࠷ऴతʹ͸͜͜ͷ൑அׂ͕ͱॏཁʹͳΔؾ͕͢Δ
  28. Ͳ͏͢Δʁ ٯʹਂ௥͍͠ͳ͍Α͏ʹ͍ͯ͠Δ͜ͱ • Ϟσϧͷֶशύϥϝʔλ΍"VHNFOUBUJPOͷࡉ͔͍ௐ੔ • ˞΋ͪΖΜ͕͜͜ॏཁʹͳΔίϯϖ΋͋Δ͠ɺݸਓͷײ૝Ͱ͢ • ࣗ෼ͷ৔߹ɺPQUJNJ[FS΍-3 TDIFEVMFSؚΉ جຊతͳBVHNFOUBUJPO͸ׂΓͱݻఆ

    • ϞσϧͷDVTUPNIFBE ࠷ऴతʹ༧ଌ͢Δ෦෼ ͷ޻෉ • ΍ͬͨΒޮ͔͘΋͚ͩͲɺଞͷ༏ઌ౓ͷํ͕ߴ͍
  29. ͦͷଞ ʢࣗ෼͕ʣ৸͍ͯΔ࣌ؒΛ׆༻͢Δ • ઈରʹͳʹֶ͔शΛճ͔ͯ͠Β৸Δ ͱ͖ʹ͸ਓಡΈ΋׆༻͢Δ • ͦͷ)045͕աڈ։࠵ͨ͠ίϯϖͰͷTIBLF۩߹ • ίϯϖͷ։࠵໨తͳͲ͔ΒͲͷΑ͏ʹ1VCMJD1SJWBUF͕෼͔Ε͍ͯΔ͔ •

    ্Ґਞ͕ͲΜͳ͜ͱΛͯͦ͠͏͔ FUD
  30. ·ͱΊ • FQPDIʹ͔͔Δ࣌ؒΛ୹͘͢Δ • ʮ΍Δ͜ͱʯʮ΍Βͳ͍͜ͱʯΛ໌֬ʹ͢Δ • ब৸࣌ؒΛ༗ޮʹ࢖͏ • Ϟσϧͷύϥϝʔλௐ੔ʹ࣌ؒΛ͔͚͗͢ͳ͍ •

    ਓಡΈ΋׆༻͢Δ
  31. ご静聴ありがとうございました!!