第3回関東kaggler会 🤔 妙だな... (Jun Koda)

1. 🤔 ົͩͳ... ᅳ෎ాɹ३ Jun Koda
3. ܦྺ • ςΩαεେֶͰ෺ཧͷ PhD • Ӊ஦ͷ਺஋γϛϡϨʔγϣϯ • ͦͷޙɺΦʔετϥϦΞɺΠλϦΞͰ ϙευΫݚڀऀ Ϗοάόϯ௚ޙ͸΄΅Ұ༷ͩͬͨμʔΫϚλʔ͕ ສ༗ҾྗͰҾ͔Ε͍͋ͬͯ͘
4. ܦྺ • ςΩαεେֶͰ෺ཧͷ PhD • Ӊ஦ͷ਺஋γϛϡϨʔγϣϯʢμʔΫϚλʔಉ࢜ͷສ༗Ҿྗʣ • ͦͷޙɺΦʔετϥϦΞɺΠλϦΞͰϙευΫݚڀऀ • ίϩφՒͷϩοΫμ΢ϯͰମௐΛ่͠೔ຊʹ໭Γྍཆ • Kaggle ͷ৆ۚͰ Kaggle Λ΍Ε͹ྑ͍ͷͰ͸ʁˠ ಇ͖ʹग़Α͏ͱͯ͠Δ
5. গ਺ͷਓ͔͠ؾ͍͍ͮͯͳ͍͜ͱʹؾ͍ͮͯҰ఺ಥഁͰۚϝμϧΛऔΔλΠϓ
6. ճస augmentationͰείΞ͕Լ͕Δʁ 🤔 ົͩͳ... • ࢲͷҰ൪ͷۀ੷ • ͜ΕΛ͖͔͚ͬʹ1Ґʹ Google Research - Identify Contrails to Reduce Global Warming (2023)
7. Benetech 4 + 3 min
8. ਪ࿦ͷ͕࣌ؒ୹͘ͳͬͨ 🤔 ົͩͳ... • άϥϑը૾͔Β x y ͷ஋ʢ਺஋ɾจࣈʣΛಡΉ • ࢲ͸ؾ͔ͮͳ͔͚ͬͨͲ discussion ʹ͋ͬͨ • A. Ϟσϧͷੑೳ্͕͕Ε͹ͦ͏͍͏͜ͱ΋͋ΔΑ => Ұཧ͋Δ • ੜ੒Ϟσϧͷ௕͍ҙຯෆ໌ͷग़ྗͰ࣌ؒΛແବʹ࢖͏ Benetech - Making Graphs Accessible (2023)
9. https://x.com/aryyyyy221/status/1670952873872744449 Private ςετσʔλ͕ͬͦ͜Γফ͑ͯͨ
10. Line Scatter 50 - 65% Dot ΍Δҙຯͳ͠ Horizontal Bar ΍Δҙຯͳ͠ Vertical Bar ͻͲ͍
11. • Scatter ͷ৳ͼ୅͸ͨ͘͞Μ͋ͬͨͷͰɺۚϝμϧऔΕ͔ͨ΋ • άϥϑͷׂ߹͸೺Ѳ͓ͯ͘͠΂͖ͩͬͨ • Ͱ΋ݮΔલʹௐ΂͍ͯͨΒɺݮͬͨ͜ͱΛ࡯஌Ͱ͖͔ͨʁ 15Ґ → ௐ΂Δํ๏͸ "LB Probing" bilzard ͞Μʮࠓ·ͰʹKaggleίϯϖͰ࢖ͬͨLB Probingख๏ʹ͍ͭͯʯ https://zenn.dev/bilzard/articles/lb-probing-technique
12. USPTO 7 + 4 min
13. είΞ͕਺ֶతʹ͋Γ͑ͳ͍ 🤔 ົͩͳ... USPTO - Explainable AI for Patent Professionals ʢಛڐίϯϖʣ ͜ͷ໘ੵ 0.5 λʔήοτ͸ৗʹ 50 ݸͰ ਖ਼ղ࠷େͰ 25ݸ ਺ֶతʹ࠷ߴ0.5ͩͱࢥͬͨΒ submission Ͱ 0.55? 0.8 ΋௒͑ΔΒ͍͠ ͕࣌ؒͳͯ͘͏΍Ή΍ʹ ؒҧͬͨධՁࢦඪΛ࠷దԽͯ͠͠·ͬͨ 🔺ʮKaggle ͱ͸ධՁࢦඪΛ࠷దԽ͢Δڝٕʯ ਅͷධՁࢦඪΛ೺ѲͰ͖ͳ͔ͬͨͱ͸൓ল ؔ౦ Kagger ձʢୈ1ճʣcharmq ͞Μ Kaggle΁ͷऔΓ૊Έํ ~validationฤ~ TP 25ݸͰείΞ 0.8 Λӽ͑Δ Theo Viel https://www.kaggle.com/competitions/uspto-explainable-ai/discussion/522199 Competition metric ͕࿩ͱҧ͏ 1? ධՁࢦඪ MAP@50 ໘ੵ ~ 0.85
14. είΞ͕ 0.99? 🤔 ົͩͳ... ख࣋ͪͷΞϧΰϦζϜ΍࠷దԽख๏͕ҧ͍͗͢ΔͷͰɺ ͦ͏ݴ͏͜ͱ΋͋Δ͞ => ഊ๺ऀ ࣮͸ࢲʹ΋Ͱ͖Δ hack / magic ͕͋ͬͨ
15. աૄίϯϖͰϫϯνϟϯۚϝμϧ Ͱ΋ɺ̍̌೔͘Β͍͔͠ࢀՃͯ͠ͳ͍͔Βʙ ͱݴ͏ಀ͛ಓ͢Βഁյ͞ΕΔ ← ίί
16. Ariel Ͱ΋͋ͬͨ Competition metric ͕࿩ͱҧ͏ OK Ṗͷఆ਺ NeurIPS - Ariel Data Challenge 2024 Metric ͕յΕ͍ͯΔͱࢥͬͨΒʁ Test set ͷෆ֬ఆੑͱ metric ͷ͍͕ٙࠞ͟ΔͷͰ೉͍͠ LB probing ͱಉ͘͡ ੍ޚͰ͖Δ΋ͷΛ (σ) ΛมԽͤͯ͞ είΞͷมԽΛݟΔ AmbrosM https://www.kaggle.com/competitions/ariel-data-challenge-2024/discussion/528114
17. select ͸յΕ͍ͯͳ͍ The Pragmatic Programmer ɹʰୡਓϓϩάϥϚʔʱ • ·ͣ͸ࣗ෼Λ͓ٙ͏ • ྫྷ੩ʹߟ͑ͯ΋͓͔͔ͬͨ͠Β discussion Ͱ૬ஊͯ͠ΈΑ͏ʢಛʹ evaluation metric ͷ͜ͱͳΒʣ • CZIIΛٹͬͨέϩοϐઌੜ͸ྲྀੴ
18. Ventilator 11 + 5 min ۚϝμϧʹ݁ͼ͍ͭͨكͳྫ (1)
19. ਫ਼౓͕ྑ͗͢Δ 🤔 ົͩͳ... Google Brain - Ventilator Pressure Prediction (2021) https://www.kaggle.com/competitions/ventilator-pressure-prediction/ Input: ਓ޻ݺٵثͷٵೖόϧϒ ut Target: ഏʢ໛ܕʣ಺ͷۭؾѹྗ pt ࣌ܥྻίϯϖ u0 p0 u1 p1 u2 p2 ut pt ɾɾɾ ɾɾɾ ૒ํ޲ LSTM ͳͲ Input Target ࣌ؒ →
20. 1. ࠷ॳ͸௚ײ 2. ௥ٻɾ୯७Խ: ࠷ॳͷ p0 ͳΜͯΘ͔ΔΘ͚ͳ͍ͷͰ͸ʁ 3. ࣮ݧ: ࣮ࡍ u0 ͚ͩͰ͸ p0 ͷޡࠩ͸ 0.5 (MAE) 4. Bi-LSTM ͩͱ p0 ͷޡࠩ 0.15 5. ·͢·͓͔͍͢͠ u0 p0 t0 ͚ͩ ޡࠩ Δp0 ~ 0.5 u0 p0 u1 u2 ut Input Target Bi-LSTM ޡࠩ Δp0 ~ 0.15 (MAE) ະདྷͷೖྗ͕աڈͷѹྗʹӨڹΛ༩͑Δ͸͕ͣͳ͍ Կނʁ ? ɾɾɾ ਫ਼౓͕ྑ͗͢Δ ࣌ؒ → ୯७Խ
21. PID ੍ޚ Proportional - Integral - Derivative Controller u0 p0 u1 u2 ut Input Target • աڈͷग़ྗ͕ະདྷͷೖྗʹӨڹΛ༩͍͑ͯͨʂ • ग़ྗͷઢܕࣸ૾ͷ݁ՌͰ͋Δೖྗ͔Βग़ྗΛٯࢉ ͢Δઢܕ୅਺ͷ໰୊ͩͬͨʂ • ܎਺ K ͕৭ʑͳͷͰ LSTM ͸׬શʹ͸ղ͚ͳ͍ ɾɾɾ ࣌ؒ → <latexit sha1_base64="cb481LZ0rfS2icOpNjk8woetUrI=">AAACKHicbZDNSgMxFIUz/tb6V3XpJlhERSgzIupGLLoR3FSwttDWIZPJtMFMJiR3hDL0cdz4Km5EFOnWJzFTu9DqhcDHOfdyc0+gBDfgukNnanpmdm6+sFBcXFpeWS2trd+aJNWU1WkiEt0MiGGCS1YHDoI1lWYkDgRrBPcXud94YNrwRN5AX7FOTLqSR5wSsJJfOkt3YQ+f4itfYZXjvkWO21yC795Bru3s4RB2RkaI25EmNAvVIAthYPv9UtmtuKPCf8EbQxmNq+aXXtthQtOYSaCCGNPyXAWdjGjgVLBBsZ0apgi9J13WsihJzEwnGx06wNtWCXGUaPsk4JH6cyIjsTH9OLCdMYGemfRy8T+vlUJ00sm4VCkwSb8XRanAkOA8NRxyzSiIvgVCNbd/xbRHbBRgsy3aELzJk//C7UHFO6ocXh+Wq+fjOApoE22hXeShY1RFl6iG6oiiR/SM3tC78+S8OB/O8Lt1yhnPbKBf5Xx+ATQ9ooM=</latexit> u(t) = Kpp(t) + Ki Z t 0 p(t0)dt0 + Kd dp dt (t) ग़ྗ (target)ɺͦͷඍ෼ੵ෼Λೖྗʹ࢖͏ͱ͍͍ײ͡ʹ੍ޚͰ͖Δ
22. Ұ෦ͷޡࠩΛ׬શʹ 0 ʹͯ͠4Ґ ࠷ऴ೔
23. Ventilator ·ͱΊ 1. ҧ࿨ײɾ௚ײ 2. ҧ࿨ײΛߜΓࠐΉɻݴޠԽ 3. ࣮ݧͯ͠໰୊Λ໌֬ʹ͢Δ 4. ԾઆʢLeakage?ʣɾݕূ => ͦ͏Ͱ΋ͳ͍ 5. ౴͑Λݟͨʢओ࠵ऀͷ࿦จΛಡΜͩʣ
24. ඈߦػӢ 16 + 8 min
25. ճస augmentationͰੑೳ͕Լ͕Δʁ 🤔 ົͩͳ... • Ӵ੕ը૾͔ΒඈߦػӢΛݟ͚ͭΔ Semantic Segmention λεΫ • 7 timestep ͘Β͍ͷ࣌ؒํ޲΋͋Δ 2 + 1 ࣍ݩσʔλɻσʔλ͸๺൒ٿʹଟΊ • ౰࣌ͷ࣮ݧϩά Gmail ΛৼΓฦͬͯΈΑ͏ Google Research - Identify Contrails to Reduce Global Warming (2023) https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/
26. ೰ΜͰΔ D4 (rot90 ͱ൓స) augmentation Λ ΍ͬͨΒείΞ͕མͪͯɺ͍Ζ͍ Ζ΍ͬͯ΋ݩʹ໭Βͳ͍ ʮରশੑΛ࣋ͨͳ͍͜ͱͳΜͯ͋Δʁʯ ෺ཧֶ͸ରশੑେ޷͖ ճసରশੑ͕͋Δ͸ͣͱͩ͜ΘͬͯΔ rot90 ͱ൓స augmentation Ͱ Dice score 0.666 → 0.638 ΋Լ͕Δ 7 / 3 U-Net ʹ rot90 + ൓స augmentation ΛೖΕͨ
27. ཌ೔ 7 / 4 ށ࿭ͬͯΔ ॳΊͯͷ͜ͱͰށ࿭͍ͬͯΔ ՄࢹԽͷ༧ఆ
28. • ࣌ؒ࣍ݩΛ࢖͏ϓϩδΣΫτ΋ฒߦͯ͠΍ͬͯͨͷͰɺͦͬͪʹ໭ͬͯΔ • ؾ෼స׵େࣄ • ͻͱͭͷࣄʹϋϚͬͯ͠·ͬͯ͸ྑ͘ͳ͍ 7 / 5 ಀආͨ͠ʂ
29. 7 / 6 ಥવ͢΂ͯΛཧղͯ͠Δ 0.1 pixel ͸ޡ২ɻ0.5 pixel ͷ͜ͱͩͱࢥ͏ ೖྗΛճస͢ΔͱͲ͏ͳΔʁ͸མͪண͍ͯϖϯͱࢴΛ࢖ͬͯߟ͑ͯΈΔඞཁ͕͋Δͱࢥ͏ 1st-place solution Jun Koda https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/430618 ճసͨ͠ೖྗΛϞσϧʹೖΕΔ࣮ݧ ճసͨ͠ϥϕϧͱൺֱ ՄࢹԽ & είΞܭࢉ ճసͯ͠΋είΞ͸ಉ͡Ͱ͋ͬͯ΄͍͠
30. ·ͩԿϐΫηϧͣΕͯΔ͔෼͔ͬͯͳ͍ ※ 0.5 pixel γϑτ͸ torch grid_sample() Λ࢖ͬͯΔ ೖྗΛճస͢Δͱճసͨ͠ग़ྗͱҰக͢ΔΑ͏ʹͳͬͨ ճసͨ͠ೖྗΛϞσϧʹೖΕΔ࣮ݧ ճసͨ͠ϥϕϧͱൺֱ ՄࢹԽ & είΞܭࢉ ճసͯ͠΋είΞ͸ಉ͡Ͱ͋ͬͯ΄͍͠
31. resnest26d Local CV No augmentation: 0.666 (10 epochs) Bad rot aug: 0.638 0.5 pixel fi xed rot: 0.687 (60 epochs) ͨͿΜ͜ΕͰۜݍʹೖͬͨͷͰɺۜݍҎ্ͷਓ͸ΈΜͳ 0.5 pixel ิਖ਼ͯ͠Δͷ͔ͱࢥͬͨɻ Discussion Ͱ΋ճసTTAͱ͔ݴͬͯͨ͠ɻ ໎ݴʮΘΓͱΈΜͳؾ෇͍ͨͱࢥ͏͚Ͳʯ 7 / 8 ζϨΛิਖ਼ͨ͠ϞσϧͰείΞ͕৳ͼͯΔ
32. 1. ճస augmentation ͰείΞ͕େ͖͘མͪΔ͸͕ͣͳ͍ͱͩ͜ΘΔɻগͳ͘ͱ΋ؙҰ೔ߟ͑ͨ 2. είΞ͕ѱԽ͢ΔϝΧχζϜΛߟ͑Δɻ 3. ࿈ଓ࠲ඪ͔Β pixel ΁ͷ fl oor ໰୊ʁʢԾઆʣ 4. ճసͨ͠ೖྗΛϞσϧʹೖΕͯΈΔͱ͍͏࣮ݧʢݕূʣ 5. ཧղ ඈߦػӢ·ͱΊ • ແҙࣝʹͣͬͱॲཧ͞Ε͍ͯΔͷ͔ɺ;ͱؾͮ͘ɻࢄา΍ϩʔυόΠΫ • ೴ͷ݈߁ͷͨΊʹ΋ӡಈ͸େࣄ • ৔߹ʹΑͬͯ͸క੾ΛܾΊͯɺͩ͜ΘΓΛࣺͯͯఫୀ͢Δඞཁ΋ • ॏྗ೾ίϯϖͰ͸೉͗͢͠Δ͜ͱʹͩ͜Θͬͯɺਖ਼͍͠ಓʹΪϦΪϦͰ෮ؼͰ͖ͨ
33. 🤔 ົͩͳ • ົͩͳ͸ Kaggle ͷ໾ʹཱͭ • ົͩͳ͸ത࢜՝ఔͰ஁͑ΒΕͨ • ਖ਼ղΛ୭΋஌Βͳ͍ݚڀͰ͸ɺ݁Ռ͕ਖ਼͍͔͠ɾཧղͰ͖Δ͔ɾ͓͔͍͠ࣄ͸ແ͍͔ٞ࿦͢Δ • ςΩαεͷࢣঊʹ஁͑ΒΕͨ • ົͩͳ͸ݚڀ΍ۀ຿ͷ໾ʹͨͭ • ͲΜͳ༏ΕͨਓͰ΋ʢಛʹઐ໳֎Ͱ͸ʣؒҧ͏ɻAI͕׬ᘳʹͳͬͯ΋࢖͏ਓؒ͸ෆ׬શ • ົͩͳ͸ҧ࿨ײʹؾ͖ͮɺṖΛղ͍ͯɺؒҧ͍Λగਖ਼͢Δྗ • ࣗ෼ͷؒҧ͍΋͋Ε͹νʔϜͷऑ఺΋Χόʔ͢Δ͜ͱ΋ • ࠷ॳͷҧ࿨ײ͕೉͍͠ɻֶΜͰߟ͑ͨܦݧ 24 + 1 min
34. 1000ԁࡳͰλόί1ݸ… ົͩͳ… ໊୳ఁίφϯ (18) ੨ࢁ߶ণ গ೥αϯσʔίϛοΫε