画像ディープラーニングコンペの基本

上位Kagglerに学ぶ~画像コンペの戦い方~
2025-08-07

Transcript

1. ը૾σΟʔϓϥʔχϯάίϯϖͷجຊ ্Ґ KagglerʹֶͿɹʙը૾ίϯϖͷઓ͍ํʙ 2025-08-07 Jun Koda ᅳ෎ాɹ३ʢί΢μɹδϡϯʣ https://hakuhodo-technologies.connpass.com/event/361499/
2. ෺ཧ GMɻը૾ GM Ͱ͸ͳ͍ ⇧ ᠳΜͰ࡛ۄͷʮ࡛ۄʹॅΜͰΔͷʹ౦ژͬͯॻౕ͘ʯͰக໋ইΛෛͬͨ AI ʹΑͬͯ Notebook ͕ෆཁʹͳΔલʹ Notebook GM ʹͳΖ͏͔ͳ ϙευΫͱͯ͠Ӊ஦෺ཧΛ΍͖ͬͯͨ ࠓ͸ۀ຿ҕୗͰػցֶशݚڀ։ൃ
3. ඈߦػӢ segmentation. U-Net 70×1000 ͱ͍͏αΠζ͚ͩͲ U-Net ͓ೃછΈͷҩྍը૾εϥΠε ৴߸ॲཧཁૉ͋Γͷը૾෼ྨ ͸͡Ίͯͷը૾ίϯϖ ଟνϟϯωϧ͚ͩͲ2Dը૾෼ྨ RTX2080Ti Λങͬͯ௅ઓ 3D ݂؅ segmentation ը૾ίϯϖ
4. G2Net (2021) ͷࢥ͍ग़ ϒϥοΫϗʔϧ߹ମ৴߸͕͋Δ͔Ͳ͏͔Λ൑ఆ͢Δ ࣌ܥྻ৴߸Λ spectrogram ͬΆ͘ը૾ʹ͢Δ “࠷େͷϒϨʔΫεϧʔ͕ learning rate Λ্͛Δͩͬͨ” • Public notebook Λࣸܦ • ৴߸ॲཧΛ޻෉͢Δ΋ϝμϧݍ͔Β΄ Ͳԕ͍ • ৽͍͠ public notebook ΛΈΔ • lr 1e-5 → 1e-4 ͰείΞര্͕Γ
5. https://medium.com/@junkoda/kaggle-ॏྗ೾Ͱۜϝμϧ-1c7135e69817 ౰࣌ͷϒϩάΑΓ খ͍͞ lr ͸ local minimum ʹϋϚΔ G2Net (2021) ͷࢥ͍ग़
6. ɾɾɾ ɾɾɾ 0.005 ͸େࠩ ↑ ͸ prize ↓ ͸ݍ֎
7. Hyperparameter tuning Ͱ Kaggle Λউͭ͜ͱ͸Ͱ͖ͳ͍ ͕ͩෛ͚Δ͜ͱ͸Ͱ͖Δ
8. جຊతͳσΟʔϓϥʔχϯά܇࿅ͷ ίίϩ͸ॻ͍ͯ͋Δ learning rate warm up, batch size, ͳͲͳͲ খ͍͞ batch size ͸ over fi t Λ๷͙͚ͲɺͦͷͨΊʹখ͘͢͞Δͷ͸अಓͳؾ͕͢Δ
9. https://www.kaggle.com/competitions/hms-harmful-brain-activity-classi fi cation/discussion/488083 େGrandmater ΋ҙ֎ͱࡉ͔͘ௐ੔ͯ͠Δ
10. Grid search ʹ͢Δʁ Optunaʁ ͦΕͱ΋ G • S • Dʁ ͱ͜ΖͰ Hyperparameter optimization Ͳ͏ͯ͠·͔͢ʁ
11. Kaggle Ͱ͸ graduate student ͸ࢲͨͪࣗ਎ɻࢲ͸खͰͪ·ͪ·ௐ੔ͯ͠Δ
12. Augmentation ͸େࣄ
13. Albumentatations RandomRain: https://explore.albumentations.ai/transform/RandomRain ˚ ܇࿅σʔλʹͳ͍ʹ΋ରԠͰ͖ΔΑ͏ʹͳΔʢe.g. Ӎͷࣸਅʣ ˕ σʔλΛ૿΍ͯ͠௕͘܇࿅Ͱ͖ΔΑ͏ʹ͢Δ Augmentation ͷޮՌ
14. Augmentation ͍͢͝ͷਤ ඈߦػӢίϯϖ ࠷ॳͷ 10 epoch ͚ͩͩͱҧ͍͸গͳ͍ ௕͘܇࿅Ͱ͖Δͷ͕ϙΠϯτ
15. b d ͜Ε͸ b HFlip ͜Ε͸ d Augmentation Λͯ͠͸͍͚ͳ͍৔߹ ҰจࣈΞϧϑΝϕοτ෼ྨͰ Horizontal fl ip Ϟσϧ΋ࠔ࿭
16. ΠϯυਓΛӈʹ ϋϯυϧΛࠨʹ ੔߹ੑͷऔΕΔΑ͏ʹ൓స - atmacup ंͷيಓ༧ଌ hakubisin ͞Μ 1st place solution https://speakerdeck.com/hakubishin3/turing-x-atmacup-number-18-1st-place-solution ը૾ɾϋϯυϧ֯౓ɾ΢ΟϯΧʔͳͲΛҰ؏ͯ͠൓స͢Δ ※ ͜ΕͰ΋ं͕ӈଆΛ૸Γͩͨ͠Βେมͳ৔߹͸μϝ
17. ରশੑΛճ෮ͤ͞Δ - ճసɾ൓స͕͏·͍͔͘ͳͯ͘΋मਖ਼Ͱ͖Δ͜ͱ΋ https://www.kaggle.com/competitions/waveform-inversion/writeups/ruby-14th-place-solution ஍਒೾ίϯϖɾ൓సͤ͞Δͱ਒ݯ͕̍ͭζϨΔ ͳΒɺ൓సͤͯ͞1 pixel ͣΒͤ͹͍͍͡Όͳ͍ tascj ͞ΜͷඈߦػӢ solution ΋ 0.5 pixel ͣΒ͢ ڥք͕ؾʹͳΔ͚Ͳେৎ෉Β͍͠ ← ൓సͯͣ͠Β͢ ճసɾ൓స͕Ͱ͖ͳͯ͘΋ͻͱ޻෉ͰͰ͖Δ৔߹΋
18. σʔλ͕গͳ͍ͱ͖͸ Augmentation Λڧ͘ɺϞσϧ͸খ͞Ί Theo Viel https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/writeups/on-strike-2nd-place-solution Cut Mix Yun et al (2019) https://arxiv.org/pdf/1905.04899 ϥϯμϜʹը૾ΛࠞͥΔ ϥϕϧ͸໘ੵൺ soft label ͦΜͳཚ๫ͳ! RSNA 2023 ೣ͕4෼ʹݘ͕6෼ʂ
19. Augmentation ͸ڧ͍ ௥Ճσʔλ͸΋ͬͱڧ͍ G2Net (2021) ѹ౗త1Ґ https://www.kaggle.com/competitions/g2net-gravitational-wave-detection/writeups/kdl-top-1-solution-deep-learning-part ஍਒೾ (Geophysical Waveform Inversion) https://www.kaggle.com/competitions/waveform-inversion/leaderboard 13TB Ҏ্ͷσʔλΛੜ੒ On-the- fl y σʔλੜ੒ͷͨΊʹ CUDA Λॻ͍ͨ Augmentation ͸ٖࣅతͳ௥Ճσʔλͳͷ͔ͩΒͦΕ͸ͦ͏ ࢲ: 660 GB ͷσʔλ͕େ͖ͯ͘ਏ͍ → ශऑ!
20. ը૾ίϯϖ͸஍ಓͳվળ͕ඞཁ • େ͖ͳΞΠσΟΞͰυϯͱείΞ͕Α͘ͳΔ΋ͷͰ͸ͳ͍ • ಉ͡Α͏ʹݟ͑ͯ΋খ͞ͳվળͷੵΈॏͶ͕େ͖ͳࠩʹ զʑ͸ RSNA ίϯϖϨδΣϯυͷΑ͏ʹ͸͍͔ͳ͍ ஍ಓʹίπίπࢼߦࡨޡɾܦݧΛੵΉʁ ࡉ͔͍ςΫχοΫΛ prize solution ίʔυΛಡΜͰձಘ͠Α͏ʢࢲ͸Ͱ͖ͯͳ͍ʣ
21. ʮʓʓ͚ͨ͠Ͳޮ͔ͳ͔ͬͨʯ͸ҙຯ͕ͳ͍ • ৚͕݅ͦΖͬͯॳΊͯޮՌΛൃش͢Δ͜ͱ͕Α͋͘Δ • ྫ͑͹ը૾Λେ͖ͨ͘͠ͱ͖ɺϞσϧΛେ͖ͨ͘͠ͱ͖ɺ௕͘܇࿅ͨ͠ͱ͖ʹॳΊͯ ޮՌΛൃش͢Δ • ؆୯ͳྫͩͱ augmentation ͨ͠Β epoch Λ૿΍͢ • ͦͷ··ͩͱɺ୯ʹσʔλ͕ྼԽ͚ͨͩ͠ tattaka ͞ΜʮҰ౓ࣺͯͨΞΠσΟΞʹҙࣝతʹཱͪฦΖ͏ͱߟ͑ͯΔʯ ؔ౦ Kaggler ձ 2025 य़ private communication ޙ͔Βߟ͑Δͱਖ਼ղͷۙ͘ΛԿ౓΋௨Γա͗ͯΔɺͱ͍͏͜ͱ͸Կ౓΋͋Δ
22. Kenshin ͞Μ https://tech.preferred.jp/ja/blog/kaggle-contrails-3rd-place/ https://blog.knshnb.com/posts/journey-to-grandmaster/ ඈߦػӢίϯϖ ࢲΛؚΊͨଟ͘ͷਓʮ2.5D Ϟσϧ͸͏·͍͔͘ͳ͍ʯ ৴೦ͱࢼߦࡨޡʂ
23. G2Net Ͱҹ৅ʹ࢒ͬͨ͜ͱ έϩοϐઌੜʮը૾Ͱ͸ 0.88 ͸௒͑ΒΕͳ͍ɺ௚઀৴߸Λݟͳ͍ͱʯ ࣮ࡍɺ্Ґ solution ͸ͦͷ௨Γͩͬͨ GMʹͳΔͱඍௐ੔Ͱ͸௒͑ΒΕͳ͍ݶք͕೺ѲͰ͖ΔΒ͍͠ ͳͷͰ Hyperparameter tuning ΋ࡉ͔͍ௐ੔΋͖ͬͱྑ͍ܦݧ ͍Ζ͍Ζࢼߦࡨޡͯ͠ɺͲͷ͘Β͍·ͰվળͰ͖Δ͔Θ͔Ε͹Ұਓલ ʢࢲ͸Θ͔Βͳ͍ʣ
24. • ແ৬Λଓ͚Δ͓͕ۚ΋͏ͳ͍ • Grandmaster ʹͳΓ͍ͨ • ͓͕ۚͳ͍ͷͰϥετνϟϯε • Cloud GPU Λ࢖ͬͯͰ΋ۚϝμϧ • ࠷ޙͷ1िؒʹ A100 Λ 4~5୆ • େ͖ͳϞσϧ → ࣦഊ • ը૾Λେ͖͘ • 256 pixel ͷೖྗը૾Λ 1024 ʹ Ryushi ͞Μ͔Βͷ͓୊ ্Ґೖ৆࣌ʹߟ͑ͨ͜ͱ ্Ґ..ೖ৆...? ΋͏௕͍͜ͱۚϝμϧ΋औͬͯͳ͍ͷͰ๨Ε·ͨ͠ ը૾ΛͰ͔ͬ͘ 1024×1024 ඈߦػӢίϯϖ Google Research - Identify Contrails to Reduce Global Warming ৴߸͕ࡉ͍͔Βޮ͍ͨͷ͔΋ɻG2Net Ͱ΋ޮ͍ͨ
25. ίϯϐϡʔλࣄ৘ ϩʔΧϧϚγϯ Ubuntu RTX 4090 (RAM 24 GB) Intel Core i5 13500 (TDP 65W) ਫྫྷΛආ͚ۭͯྫྷʹ͢ΔͨΊ߇͑Ίʹ͚ͨ͠ͲɺίΞ͕ͨ͘͞Μ͋Δͱ͸͍͑ҙ֎ͱCPU΋࢖͏ M.2 SSD WD Black 2TB (Gen 4 ~7000MB/s) σʔλಡΈࠐΈ଎౓΋ҙ֎ͱେࣄ ۚϝμϧ͕औΕͦ͏ͳΒ࠷ޙͷ1िؒʹ Cloud GPU Fractal Design "North"
26. औΕ·ͤΜͰͨ͠!!! Yale/UNC-CH - Geophysical Waveform Inversion
27. ۚϝμϧ͕औΕͦ͏ͳΒ࠷ޙͷ1िؒʹ Cloud GPU ʮۚϝμϧΛऔΔʯͳͲͱࢤ͕௿͍͔ΒۜϝμϧʹͳΔͷͰ͢ Cloud GPU ʹ͓ۚΛ஫͗ࠐΉͳΒ1ҐΛऔΓͳ͍͞
28. 1Ґ͕औΕͳ͍ͳΒ Cloud GPU ʹ͓ۚΛ஫͗ࠐΉͷ͸΍Ίͳ͍͞ ରۮ ͦΜͳ͜ͱΑΓ prize solution Λࣸܦ͠Α͏ ϓϥΠεϨε 👍
29. ѱຐͷᅤ͖ ৆ۚͰޙ͔Βิరͱ͔͍͏ ѱຐͷ༠࿭ʹෛ͚ͳ͍ ୤๏Χ...ʢͦΕҎ্͍͚ͳ͍ʂ
30. 30ສԁΛਓੜʹ౤ࢿʁ RTX 5070 (12G), 5070Ti (16G) 10ສԁલޙͷ GPU ͔Βελʔτ͢Δͷ͸ΞϦͳؾ͕͢Δ
31. ࢲͷ͓͢͢Ί͸ϩʔυόΠΫ ݈߁େࣄ ӡಈ͸೴ʹ΋ޮ͘ 10೥͸࢖͑Δ 30ສԁΛਓੜʹ౤ࢿʁ RTX 5070 (12G), 5070Ti (16G) 10ສԁલޙͷ GPU ͔Βελʔτ͢Δͷ͸ΞϦͳؾ͕͢Δ Shimano 105 Λ࢖ͬͯΔ΍ͭ