atmaCup#11 1st place solution

2021/8/5 @ishikei4 atmaCup#11 1st place solution & 忙しい⼈のための画像コンペとの向き合い⽅

ࣗݾ঺հ • ੴ ܓҰ࿠ 5XJUUFS!JTIJLFJ • &OHJOFFS!3JTU*OD • ,BHHMF.BTUFS •
1"1",BHHMFS ່ࡀ

໨࣍ TUQMBDFTPMVUJPO ʢ͓·͚ʣ๩͍͠ਓͷͨΊͷը૾ίϯϖͱͷ޲͖߹͍ํ

TUQMBDFTPMVUJPO %JTDVTTJPO • IUUQTXXXHVSVHVSVTDJFODFDPNQFUJUJPOTEJTDVTTJPOTEBBGDFBDCDCF $PEF • IUUQTHJUIVCDPNJTIJLFJLBUNB@TU@TPMVUJPO ஫ҙ఺ • ग़ͯ͘Δख๏ͷࡉ͔͍આ໌͸هࡌ͍ͯ͠·ͤΜ
• ख๏ͦͷ΋ͷΑΓɺͳͥͦΕΛબΜͩͷ͔ɺͲΜͳํ਑Ͱಈ͍ͨͷ͔ɺ ͋ͨΓͷઆ໌͕ଟΊͰ͢

0WFSWJFX 1IPUPT 1SFQSPDFTT .PEFM5SBJOJH 4UBDLJOH 1PTUQSPDFTT • $7 • 1VCMJD
• 1SJWBUF 'JOBM4VC *NBHF1BEEJOH • ը૾ͷ௕ลʹ߹Θͤͯਖ਼ํܗʹ 1BEEJOH͔ͯ͠Β3FTJ[F %*/0QSFUSBJOJOH 'JOFUVOJOH • SFTOFUE 7J5@TNBMM 7J5@CBTF • 3FHSFTTJPO 4DSBUDI USBJOJOH • SFTOFUE • 3FHSFTTJPO $MBTTJGJDBUJPO -JHIU(#. • $MBTTJGJDBUJPOͷ৔߹͸֤MBCFM ͷQSPCBMJMJUZ΋ೖྗ OQDMJQ QSFE

44- 44-ɿ 4FMG4VQFSWJTFE-FBSOJOH ࣗݾڭࢣ͋Γֶश • ϥϕϧͳ͠ը૾Ͱֶशͯ͠ɺಛ௃දݱΛ֫ಘ͢Δख๏ • *NBHF/FUQSFUSBJOFE͕࢖͑ͳ͍͜ͱ΁ͷରԠ %*/0ɿTFMG%*TUJMMBUJPOXJUI /0
MBCFMT ϥϕϧͳࣗ͠ݾৠཹ • UFBDIFS TUVEFOUͭͷϞσϧͰৠཹ ͨͩ͠UFBDIFS͸TUVEFOUͷҠಈฏۉ

44- ͳͥ%*/0Λબ୒ͨ͠ʁ • ެ࣮ࣜ૷ͷ3&"%.&͕ॆ࣮͍ͯͯͦ͠ͷ··ϩʔΧϧͰಈ͔͠΍͍͢ ˠޮՌ֬ೝ·Ͱͷ͕࣌ؒ୹ͯ͘͢Ή • ίϯϖॳظ͔Β%JTDVTTJPOʹ্͕͍ͬͯͨ4JN4JBN͸͏·ֶ͘शͤ͞ΒΕͳ͔͕ͬͨɺ ͖ͪΜͱಈ͚͹44-ͷ֤ख๏Ͱͦ͜·Ͱେ͖ͳҧ͍͸ͳ͍Μ͡Όͳ͍͔ɺͱ༧૝ ˠΑΓ؆୯ʹ֬ೝͰ͖Δख๏Λબ୒͔ͨͬͨ͠ʢͥΜͿࢼ࣌ؒ͢͸ͳ͔ͬͨ😇ʣ

44- • .PEFMSFTOFUE 7J5@TNBMM 7J5@CBTF • &QPDIͷΈຊλεΫͷਫ਼౓Λ֬ೝ͠ͳ͕Βˠʹ૿΍ͨ͠ • 44-QSFUSBJOXFJHIUΛ࢖༻ͯ͠'JOFUVOJOH ॏΈݻఆͳ͠
FQPDI Model SSL epoch SSL loss CV Public Private ViT_small_patch16 100 7.587 0.7422 0.7154 0.7002 ViT_small_patch16 300 3.814 0.6890 0.6574 0.6408

44- • 44-ͷFQPDI਺Λ͞Βʹ૿΍ͤ͹ɺ΋ͬͱ TJOHMFNPEFMͷਫ਼౓͸վળͰ͖͔ͨ΋ • ͨͩɺࠓճ͸TUBDLJOHͰׂͱείΞ্͕͕Δ ײ͕֮͋ͬͨͨΊɺTJOHMFNPEFMͷ௥ٻ͸ ఔʑʹͯ͠Ϟσϧͷ਺Λ૿΍͢ํ਑Ͱ࣮ݧΛ ਐΊͨ 44-ͷUSBJOJOH@MPTT
FQPDI਺Λ૿΍͢΄Ͳ؇΍͔ʹݮগ͍ͯ͘͠ҹ৅

*NBHF1BEEJOH • ը૾ͷॎԣൺ͕೥୅ʹΑͬͯۇ͔ʹ ภ͍ͬͯΔΑ͏ʹݟ͑ͨ • ը૾ͷ௕ลʹ߹Θͤͯਖ਼ํܗʹ QBEEJOH͔ͯ͠Β໨తͷղ૾౓ʹ 3FTJ[F • ͜ΕʹΑͬͯಉ͡QJYFM
TJ[FͰ΋ *NBHF1BEEJOHΛೖΕͨ΄͏͕ είΞ͕վળͨ͠ Model Img Padding CV Public Private resnet18d - 0.6779 0.6758 0.6522 resnet18d ✓ 0.6674 0.6609 0.6474

*NBHF1BEEJOH • 5BSHFU • TJ[FY • 5BSHFU • TJ[FY •
5BSHFU • TJ[FY ݩը૾ ͦͷ··3FTJ[F *NBHF1BEEJOH

4DSBUDI5SBJOJOH • ࠷ॳ͸ %JTDVTTJPOʹ্͕͍ͬͯͨ 4JN4JBNΛࢼ͍͕ͯͨ͠શ͘͏·͘ ͍͔ͣ • ίϯϖத൫·Ͱ͸4DSBUDIֶशͷ &GGJDJFOU/FU# 3FT/FUEΛ
࢖͍ͬͯͨ • ʢ࠷ऴTVCʹ͸4DSBUDIֶशͷ 3FT/FUEͷΈ࢖༻ʣ 5SBJOJOH1BSBNFUFS • FQPDI • PQUJNJ[FS"EBN8 XFJHIUEFDBZ • MFBSOJOHSBUF $PTJOFEFDBZ • JNBHF@TJ[FY • CBUDI@TJ[F "VHNFOUBUJPO • )PSJ[POUBM'MJQ • 4IJGU4DBMF3PUBUF • #MVS • 3BOEPN#SJHIUOFTT$POUSBTU • .JYVQ $VUNJY

4DSBUDI5SBJOJOH ͳͥFQPDIʁ • ࠓճɺ*NBHF/FUQSFUSBJOΛ࢖Θͣʹ TDSBUDIͰϞσϧΛֶश͢Δͷ͕ॳΊͯ ͩͬͨͨΊɺUJNNͷ5SBJOJOH&YBNQMFTͷ ϖʔδΛ͔ͳΓࢀߟʹͨ͠ • CBUDI@TJ[F͸Ҏ֎ͱେ͖͘ͳͯ͘େৎ෉
FQPDI਺͸ࢥͬͯͨΑΓ͔ͳΓ௕͘ͱΔɺ ͳͲצॴ͕Θ͔ͬͯͦͷޙͷ࣮ݧ͕ޮ཰తʹ ਐΊΒΕͨ

4DSBUDI5SBJOJOH ֶशۂઢ • SFTOFUE JNH@TJ[FY SFHSFTTJPO 44-ͳ͠ • $71VCMJD
1SJWBUF • ·ͩFQPDI਺৳͹ͤΔؾ΋͢Δ

3FHSFTTJPO WT$MBTTJGJDBUJPO • جຊ͸3FHSFTTJPOΛ࢖͍͕ͬͯͨɺ͍͔ͭ͘$MBTTJGJDBUJPOϞσϧ΋ֶश • ͦ͜·Ͱେ͖ͳҧ͍͸ͳ͔͕ͬͨɺޙ͔Β֬ೝ͢ΔͱTJOHMFCFTU͸$MBTTJGJDBUJPO ͩͬͨΈ͍ͨ Model SSL Img
size CV Public Private ViT_base_patch16 ✓ 224x224 0.6759 0.6400 0.6306 • $MBTTJGJDBUJPOͷ৔߹͸ͪ͜Βͷ%JTDVTTJPOʹ ͋ΔΑ͏ʹ༧ଌ஋Λ࿈ଓ஋ʹม׵

4UBDLJOH • -JHIU(#.Λ࢖ͬͨ4UBDLJOH • 3FHSFTTJPO $MBTTJGJDBUJPO૒ํͷϞσϧΛ࢖͏͕ɺ$MBTTJGJDBUJPOʹؔͯ͠͸ɺ ֤MBCFMͷQSPCBCJMJUZ΋ೖྗͱͯ͠Ճ͑ͨ 0 1 2
3 5BSHFU 0.3 1.2 2.2 2.9 .PEFM 3FHSFTTJPO 0.1 1.3 2.1 2.8 .PEFM $MBTTJGJDBUJPO QSFE QSFE 0.8 0.1 0.0 0.0 QSPCB@ 0.1 0.65 0.1 0.1 0.05 0.2 0.7 0.3 0.05 0.05 0.2 0.6 ➕ ɿ -JHIU(#. 4UBDLJOH

4UBDLJOH ͳͥ4UBDLJOHʁ • ॳظʹ&OTFNCMFͱ4UBDLJOH྆ํࢼ͍ͯͨ͠ˠͦͷ࣌఺Ͱ૒ํʹਫ਼౓ࠩ͸ͳ͔ͬͨ • $7-#ͷ૬͕ؔߴ͔ͬͨ͜ͱɺίϯϖͷੑ࣭ɺओ࠵ऀͷੑ֨ͳͲ͔Β1VCMJD1SJWBUF ΋ׂͱ૬͍ؔͯ͠ΔͩΖ͏ɺͱ༧ଌ ˠ$7ʹ0WFSGJUUJOH͠΍͍͢4UBDLJOHͰ΋-#Ͱਫ਼౓Λอͬͯ͘ΕΔͱ౿ΜͰ࠾༻ • ݁ہɺͦͷޙ$MBTTJGJDBUJPOͷQSPCBCJMJUZΛՃ͑ͨ͜ͱͰείΞ΋CPPTUͨ͠͠ྑ͍൑அ
ͩͬͨ

TPMVUJPO·ͱΊ • .PEFM3FT/FUE 7J5@TNBMM 7J5@CBTF • %*/0QSFUSBJO FQPDI 'JOFUVOJOH4DSBUDIUSBJOJOJH MPOHFQPDI
• *NBHF1BEEJOH • 3FHSFTTJPO$MBTTJGJDBUJPO • -(#.ͰTUBDLJOH $MBTTJGJDBUJPO͸֤MBCFMͷQSPCBCJMJUZ΋ೖྗ

（おまけ）忙しい⼈のための画像コンペとの向き合い⽅

ʢ͓·͚ʣ๩͍͠ਓͷͨΊͷը૾ίϯϖͱͷ޲͖߹͍ํ ࿩͢͜ͱ • ීஈࣗ෼͕ը૾ίϯϖ΍Δͱ͖ʹؾʹͯ͠Δ͜ͱ • ಛʹɺ৭ʑͳࣄ৘ʹΑΓͳ͔ͳ͔ίϯϖ΍Δ͕࣌ؒͱΕͳ͍ਓ޲͚ ஫ҙ఺ • ҰԠը૾ίϯϖʹߜͬͯ࿩͠·͢ •
ʢଞͷυϝΠϯͷίϯϖͰ΋ڞ௨ͳ෦෼͸͋Δ͔΋஌Ε·ͤΜ͕ʣ • ͋͘·Ͱ෼ੳίϯϖͱ͍͏ʮωοτήʔϜʯͰͲ͏্ҐΛૂ͏͔ɺʹয఺ ʢ࣮຿ͱ͸ผͷ؍఺ʣ

ը૾ίϯϖಛ༗ͷ೰Έ Ұճͷֶशʹ͕͔͔࣌ؒΔͨΊɺࢼߦճ਺Λ૿΍ͤͳ͍ • ݸਓతʹ͸͜Εʹ͖ͭΔ • ׬શʹϞσϧͷֶशʹੜ׆ϦζϜΛ߹ΘͤΒΕΕ͹ྑ͍͕ɺͦ͏΋͍͔ͳ͍ • ࢓ࣄɺֶߍɺҭࣇɺFUD

ը૾ίϯϖಛ༗ͷ೰Έ ྫ͑͹ࠓճͷίϯϖͷ৔߹ • SFTOFUE JNH@TJ[F CBUDI@TJ[F 359࢖༻ͰFQPDIඵ ˠFQPDIYGPME࣌ؒ • ೔Ϟσϧճͤͯ΋िؒͷίϯϖͩͱճ͔͠ϑϧϞσϧΛ࡞Εͳ͍

ը૾ίϯϖಛ༗ͷ೰Έ Ͳ͏͢Δʁ • FQPDIʹ͔͔Δ࣌ؒΛ୹͘͢Δ • ʮ΍Δ͜ͱʯʮ΍Βͳ͍͜ͱʯΛ໌֬ʹ͢Δ

Ͳ͏͢Δʁ FQPDIʹ͔͔Δ࣌ؒΛ୹͘͢Δ • ͓ۚͰͳ͙Δ • (16Λޮ཰తʹ࢖͏ • ྫɿը૾ΛOQZܗࣜͰ࣋ͬͨΓϝϞϦʹࡌ͓͍ͤͯͨΓ EBUBTFU಺Ͱ͸ಛ௃͸OQBSSBZͰ࣋ͭɺͳͲ •
(16Ҏ֎Ͱॲཧʹ͕͔͔͍࣌ؒͬͯΔՕॴΛಛఆ͢Δ • খ͍͞ը૾αΠζɺGPMEͷΈɺҰ෦ͷը૾ͷΈͰ࣮ݧΛߴ଎ʹճ͢ • ͨͩ͠ɺ͋·Γಉ͡GPMEͰͷΈ֬ೝͯ͠ΔͱͦͷGPMEʹͲΜͲΜաֶशͯ͠͠·͏ͷͰ΄Ͳ΄Ͳʹ • ؾΛ͚ͭͳ͍ͱɺࢼߦճ਺㲈TIBLF֬཰ʹͳͬͯ͠·͍͕ͪʢͳؾ͕͍ͯ͠Δʣ

Ͳ͏͢Δʁ ʮ΍Δ͜ͱʯʮ΍Βͳ͍͜ͱʯΛ໌֬ʹ͢Δ • ཧ૝తʹ͸શ෦ࢼͯͦ͠ͷத͔Βબ୒͍ͨ͠ʢ͕ɺͦΜͳ࣌ؒ͸ͳ͍ʣ • ʮޮ͖ͦ͏ͳ͜ͱʯʮޮ͔ͳͦ͏ͳ͜ͱʯΛίϯϖͷσʔλ΍എܠ͔Βߟ࡯ʢໝ૝ʣ ͨ͠ΓɺͦͷͨΊͷॳظݕ౼ʹ࣌ؒΛ͔͚Δ

Ͳ͏͢Δʁ ίϯϖং൫Ͱؾʹ͍ͯ͠Δ͜ͱ • ίϯϖಛ༗ͷੑ࣭͸Կ͔ • ࠓճͳΒ.BUFSJBMTͳͲͷ৘ใɺ֎෦σʔλېࢭɺը૾ൺ཰ʁ • ৭ʑࢼͯ͠ແཧͦ͏ͳΒ͙͢ଞͷํ਑ʹ੾Γସ͑Δ • ʢ݁ہ.BUFSJBMTͳͲͷ৘ใ͏·͘࢖͑·ͤΜͰͨ͠ʣ
• &OTFNCMF 4UBDLJOHͰͷ্͕Γ෯͸ͲΕ͘Β͍͔ʁ • TJOHMFNPEFMʹݻࣥͨ͠ํ͕ྑ͍ͷ͔ɺϞσϧͷଟ༷ੑΛॏࢹͨ͠ํ͕ྑ͍ͷ͔ • ࣗ෼͸ͭҎ্͍͍ײ͡ͷϞσϧ͕Ͱ͖ͨ࣌఺Ͱ֬ೝ͍ͯ͠Δ • ࠷ऴతʹ͸͜͜ͷ൑அׂ͕ͱॏཁʹͳΔؾ͕͢Δ

Ͳ͏͢Δʁ ٯʹਂ௥͍͠ͳ͍Α͏ʹ͍ͯ͠Δ͜ͱ • Ϟσϧͷֶशύϥϝʔλ΍"VHNFOUBUJPOͷࡉ͔͍ௐ੔ • ˞΋ͪΖΜ͕͜͜ॏཁʹͳΔίϯϖ΋͋Δ͠ɺݸਓͷײ૝Ͱ͢ • ࣗ෼ͷ৔߹ɺPQUJNJ[FS΍-3 TDIFEVMFSؚΉ جຊతͳBVHNFOUBUJPO͸ׂΓͱݻఆ
• ϞσϧͷDVTUPNIFBE ࠷ऴతʹ༧ଌ͢Δ෦෼ ͷ޻෉ • ΍ͬͨΒޮ͔͘΋͚ͩͲɺଞͷ༏ઌ౓ͷํ͕ߴ͍

ͦͷଞ ʢࣗ෼͕ʣ৸͍ͯΔ࣌ؒΛ׆༻͢Δ • ઈରʹͳʹֶ͔शΛճ͔ͯ͠Β৸Δ ͱ͖ʹ͸ਓಡΈ΋׆༻͢Δ • ͦͷ)045͕աڈ։࠵ͨ͠ίϯϖͰͷTIBLF۩߹ • ίϯϖͷ։࠵໨తͳͲ͔ΒͲͷΑ͏ʹ1VCMJD1SJWBUF͕෼͔Ε͍ͯΔ͔ •
্Ґਞ͕ͲΜͳ͜ͱΛͯͦ͠͏͔ FUD

·ͱΊ • FQPDIʹ͔͔Δ࣌ؒΛ୹͘͢Δ • ʮ΍Δ͜ͱʯʮ΍Βͳ͍͜ͱʯΛ໌֬ʹ͢Δ • ब৸࣌ؒΛ༗ޮʹ࢖͏ • Ϟσϧͷύϥϝʔλௐ੔ʹ࣌ؒΛ͔͚͗͢ͳ͍ •
ਓಡΈ΋׆༻͢Δ

ご静聴ありがとうございました！！

atmaCup#11 1st place solution

atmaCup#11 1st place solution

ishikei

Other Decks in Technology

Featured

Transcript

2021/8/5 @ishikei4 atmaCup#11 1st place solution & 忙しい⼈のための画像コンペとの向き合い⽅

ࣗݾ঺հ • ੴ ܓҰ࿠ 5XJUUFS!JTIJLFJ • &OHJOFFS!3JTU*OD • ,BHHMF.BTUFS •

໨࣍ TUQMBDFTPMVUJPO ʢ͓·͚ʣ๩͍͠ਓͷͨΊͷը૾ίϯϖͱͷ޲͖߹͍ํ