Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Turing × atmaCup #18 - 1st Place Solution
Search
Shuhei Goda
December 13, 2024
Technology
0
750
Turing × atmaCup #18 - 1st Place Solution
Turing × atmaCup #18 の表彰式での登壇資料です
https://turing.connpass.com/event/338583/
Shuhei Goda
December 13, 2024
Tweet
Share
More Decks by Shuhei Goda
See All by Shuhei Goda
ジョブマッチングサービスにおける相互推薦システムの応用事例と課題
hakubishin3
3
860
とある事業会社にとっての Kaggler の魅力
hakubishin3
8
2.2k
課題の解像度が荒かったことで意図した改善ができなかった話
hakubishin3
3
960
Wantedly におけるマッチング体験を最大化させるための推薦システム
hakubishin3
4
1.1k
Recommendation Industry Talks #1 Opening
hakubishin3
1
360
会社訪問アプリ「Wantedly Visit」での シゴトに関する興味選択機能と推薦改善
hakubishin3
0
600
論文紹介: Improving Implicit Feedback-Based Recommendation through Multi-Behavior Alignment(Xin Xin et al., 2023)
hakubishin3
0
570
Feedback Prize - English Language Learning における擬似ラベルの品質向上の取り組み
hakubishin3
0
950
ウォンテッドリーにおける推薦システムのオフライン評価の仕組み
hakubishin3
7
6.7k
Other Decks in Technology
See All in Technology
あれは良かった、あれは苦労したB2B2C型SaaSの新規開発におけるCloud Spanner
hirohito1108
2
550
データの品質が低いと何が困るのか
kzykmyzw
6
1.1k
現場の種を事業の芽にする - エンジニア主導のイノベーションを事業戦略に装着する方法 -
kzkmaeda
2
2k
室長と気ままに学ぶマイクロソフトのビジネスアプリケーションとビジネスプロセス
ryoheig0405
0
360
データ資産をシームレスに伝達するためのイベント駆動型アーキテクチャ
kakehashi
PRO
2
520
2.5Dモデルのすべて
yu4u
2
840
7日間でハッキングをはじめる本をはじめてみませんか?_ITエンジニア本大賞2025
nomizone
2
1.8k
君も受託系GISエンジニアにならないか
sudataka
2
430
アジャイル開発とスクラム
araihara
0
170
AndroidデバイスにFTPサーバを建立する
e10dokup
0
250
開発組織のための セキュアコーディング研修の始め方
flatt_security
3
2.3k
地方拠点で エンジニアリングマネージャーってできるの? 〜地方という制約を楽しむオーナーシップとコミュニティ作り〜
1coin
1
220
Featured
See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
How GitHub (no longer) Works
holman
314
140k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.8k
How STYLIGHT went responsive
nonsquared
98
5.4k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.3k
Side Projects
sachag
452
42k
Site-Speed That Sticks
csswizardry
4
380
Reflections from 52 weeks, 52 projects
jeffersonlam
348
20k
Designing for humans not robots
tammielis
250
25k
For a Future-Friendly Web
brad_frost
176
9.5k
The Invisible Side of Design
smashingmag
299
50k
Transcript
© 2024 Wantedly, Inc. 1st Place Solution Dec. 14 2024
- Shuhei Goda Turing × atmaCup #18 දজࣜ&ৼΓฦΓձ
© 2024 Wantedly, Inc. ໊લɿ ߹ా पฏ Shuhei Goda
ॴଐͱׂɿ ΥϯςουϦʔגࣜձࣾ ɾData Team Manager ɾMachine Learning Tech Lead ɾProduct Manager Kaggle Tierɿ Kaggle Competitions Grandmaster @jy_msc ࣗݾհ https://www.kaggle.com/shuheigoda
© 2024 Wantedly, Inc. Turing × atmaCup #18 ʹ͍ͭͯ •
։࠵ظؒɿ2024/11/15 17:30 ʙ 2024/11/24 18:00 • ࣗಈंͷߦγʔϯͷΧϝϥը૾ं྆ͷঢ়ଶσʔλͳͲ͔Βɺ0.5 ~ 3s ޙͷ ࣗंͷҐஔΛਪఆ͢ΔλεΫʢي༧ଌʣ
© 2024 Wantedly, Inc. ࠓճͷ Public LeaderBoard : 1st Place
🎉 Private LeaderBoard : 1st Place 🎉
© 2024 Wantedly, Inc. λΠϜϥΠϯʢ11/19͔ΒࢀՃʣ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 1.ੳͱํܾΊ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 2. 1st-stageͷվળ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 3. 2nd-stageͷվળ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 4. Ξϯαϯϒϧͷ४උͱ࣮ࢪ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ EDAϕʔεϥΠϯΞϓϩʔν͔ΒɺΛѲͯ͠ํΛߟ͑Δ • σʔλଟ͘ͳ͘ɺγϯϓϧͳ͕Βςʔϒϧಛྔ͕ڧͦ͏ •
ͱ͍ͬͯɺ༧ଌʹ͓͍ͯը૾ͷใ༗ޮͦ͏ʹݟ͑Δʢৄࡉޙड़ʣ → ଞࢀՃऀͱͷେ͖ͳࠩҟͱͳΓಘΔͷʮը૾ใͷѻ͍ํʯͩͱߟ͑ͨ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ ਐΊํͱ࣌ؒͷ͍ํΛܾΊΔ • 1st-stage Ϟσϧʢը૾Λѻ͏ϞσϧʣͷվળΛॏతʹΔ
• 2nd-stage ϞσϧʢςʔϒϧಛྔϝΠϯͷϞσϧʣͷվળΛগ͠Δ • ͋ͱϞσϧΛՔ͙࡞ۀΛߦ͍Ξϯαϯϒϧ͢Δ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ ࣮ݧʹ༻ͨ͠ϕʔεϥΠϯ • tk@tnkcoder ͞Μ͕ެ։ͨ͠ϕʔεϥΠϯϞσϧΛར༻
• [CV 0.2008/LB 0.2017] LightGBM + CNN stacking baseline (LightGBM + CNN) 1st-stage: CNN 2nd-stage: GBDT Image Tabular 1st-stage Predictions Submission
© 2024 Wantedly, Inc. 2. 1st-stageͷվળ
© 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤ Target ͷ࠷େͰׂͬͨ
Target Λֶशɾ༧ଌ͢Δ • ݁ՌɿX ͱ Y ͷ༧ଌੑೳ͕վળɻt ͕খ͍͞΄ͲޮՌ͕େ͖͍ • ղऍɿ༧ଌ࣌Ͱλʔήοτͷεέʔϧ͕େ͖͘ҟͳΔɻεέʔϧΛ߹ΘͤࠐΉ͜ͱͰɺ ֤༧ଌ࣌ͷใΛ·ͱΊͯޮՌతʹֶशͰ͖ΔͷͰͳ͍͔
© 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤༧ଌ࣌ͷՃΛ Auxiliary Target
ͱֶͯ͠शɾ༧ଌ͢Δ • ྫ͑ Target ͷ x_0, x_1 ͔Β vx_1 Λࢉग़͢Δ͜ͱ͕Ͱ͖Δ • ݁Ռͱͯ͠ɺ1st-stage CV: 0.2312 → 0.2288 (-0.0024) ʹվળ • ·ͨɺAuxiliary Target ʹର͢Δ༧ଌΛޙஈͷಛྔͱͯ͠Ճ͢Δ͜ͱͰ ɺ2nd-stage ͷείΞ͕վળʢ2nd-stage CV: 0.1963 → 0.1933ʣ
© 2024 Wantedly, Inc. 2-2. HorizontalFlip ࢥͬͨ͜ͱɿࣗಈं͔ΒࡱӨ͞Εͨը૾ɺਫฏసͤͯ͞ҧײ͕গͳ͍ Ͳ͕ͬͪΦϦδφϧʁ
© 2024 Wantedly, Inc. 2-2. HorizontalFlip ֶश࣌ɾਪ࣌ʹ HorizontalFlip ΛՃ͑Δ •
ֶश࣌ɿp=0.5 Ͱ HorizontalFlip • ਪ࣌ɿΦϦδφϧը૾ͷਪ݁Ռͱਫฏసͨ͠ਪ݁ՌΛฏۉ͢Δ • ͜ΕΒʹΑͬͯείΞ৳ͼΔ͕ɺ1st-stage Ϟσϧʹೖྗ͢Δςʔϒϧಛྔ ͷసΛΕͯ͠·͏ͱείΞ͕ٯʹԼ͕ͬͯ͠·͏ͷͰҙ • 1st-stageͰѻ͏ςʔϒϧಛྔۃྗγϯϓϧʹ͑Δඞཁ͕͋ΔɻΘ Γʹ 2nd-stage ʹෳࡶͳFEΛدͤΔ͜ͱ͕Ͱ͖Δ {“steeringAngleDeg”: 15, “leftBlinker”: True, “rightBlinker”: False} → {“steeringAngleDeg”: -15, “leftBlinker”: False, “rightBlinker”: True}
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮తใΛ͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔͳ༧ଌͰ͖ΔΑ͏ʹͳΔͷͰʁΑΓظతͳӡసঢ়گͷѲ͕ॏཁ
ྫ1ɿ sec=2.0, t-0.5 sec=2.0, t-1.0 sec=2.0 sec=12.0 12secޙͷใ͔Βɺͦͷ··ਐ͢Ε ྑ͔ͬͨ͜ͱ͕Θ͔Δ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮తใΛ͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔͳ༧ଌͰ͖ΔΑ͏ʹͳΔͷͰʁΑΓظతͳӡసঢ়گͷѲ͕ॏཁ
ྫ2ɿ sec=2.0 sec=12.0 12secޙͷใ͔ΒɺࣼΊʹਐΊ ྑ͔ͬͨ͜ͱ͕Θ͔Δ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ظͷมԽʢ1ඵ୯Ґʣ ɾٸͳૢ࡞มԽ ɾՃݮ ɾंઢมߋ
ɾӈࠨં ظͷมԽʢ୯Ґʣ ɾߦత ɾӡసελΠϧ ɾతͷܦ࿏
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ϘτϧωοΫͱͳ͍ͬͯΔʮظมԽʯΛޮՌతʹϞσϧԽ͢Δʹʁ • ϕʔεϥΠϯͰ2nd-stageʹ͓͍ͯલޙͷظใΛߟྀͨ͠༧ଌ͕Մೳͩ ͕ɺ1st-stageʹ͓͍֤ͯID͕ಠཱͳͷͱͯ͠ѻ͏&ܦ࿏༧ଌͷ݁Ռͱͯ͠ͷ
ใΛൖͤ͞ΔܗʹͳΔͷͰඇޮʹݟ͑Δ • 1st-stage ͷNNͷஈ֊ͰɺظͷมԽʹجͮ͘ΛֶशͰ͖ΔΑ͏ʹ͢Δ 1st-stage: CNN 2nd-stage: GBDT sceneA,ID1 1st-stage Predictions 1st-stage: CNN 1st-stage Predictions FE(e.g. shift features) sceneA,ID2 Shared
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿScene୯ҐͰ 2.5D-CNN + LSTM
CNN 1st-stage Predictions (B×S×N) BiLSTM … Tabular sec=20 MLP sec=120 Scene
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ sec=20 sec=120 Pad Pad
Pad Pad sec=220 sec=320 sec=520 Pad Pad Pad sec=20 sec=120 sec=220 sec=320 sec=420 sec=520 scene=A scene=B scene=C όονͷ࡞Γํ • Scene͝ͱʹ͕͞ҟͳΔͷͰɺ٧ΊͯPadding • αϯϓϧؒͰ࣌ܥྻతͳҐஔ͕ؔҟͳΔͷͰɺscene_sec scene_num ʢsceneͷத ͰԿ൪ʹొͨ͠ID͔ʣΛಛྔͱͯ͠ೖྗ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿ2.5D-CNN + LSTM Λ࠾༻͢Δ
• ͜ͷΞʔΩςΫνϟʹมߋ͢Δ͜ͱͰɺCNN୯ମͰ Private 4Ґ૬ͷείΞʹ ྫ1ɿ ྫ2ɿ
© 2024 Wantedly, Inc. 3. 2nd-stageͷվળ
© 2024 Wantedly, Inc. ͍͔ͭ͘ͷಛྔͷՃ ͍ͣΕͦͦ͜͜ͷվળʹد༩ͨ͠ • 1st stage ͷ
target (x_0 ~ z_5) ͷ༧ଌʹՃ͑ͯɺaux target ͷ༧ଌಛྔ ͱͯ͠ར༻͢Δ • 2छྨͷं྆ϞσϧʢϢχαΠΫϧϞσϧͱಈྗֶతόΠγΫϧϞσϧʣͷ༧ଌ ݁ՌΛಛྔͱͯ͠ར༻͢Δ
© 2024 Wantedly, Inc. 4. Ξϯαϯϒϧ
© 2024 Wantedly, Inc. Ξϯαϯϒϧ ༷ʑͳόοΫϘʔϯͰϞσϧΛ࡞ͬͯ Weighted Average • جຊతʹϞσϧΛ૿͢΄ͲείΞ͕େ͖͘৳ͼΔɻ࠷ऴతʹ11ݸࠞͥͨɻ
• ͬͨόοΫϘʔϯɿresnext, efficientnet, resnet, swin-transformer ͳͲ
© 2024 Wantedly, Inc. ࠷ऴ݁Ռ
© 2024 Wantedly, Inc. ֤ϞσϧͷύϑΥʔϚϯε model cv public private private
ॱҐ ɹsingle 1st stage 0.1906 0.1958 0.1808 4Ґ ɹsingle 2nd stage 0.1883 0.1928 0.1785 1Ґ ɹensemble 0.1792 0.1885 0.1754 1Ґ
© 2024 Wantedly, Inc. ϕʔεϥΠϯʹൺͯ͏·͍͘͘Α͏ʹͳͬͨྫ - ظతͳঢ়گѲ͕ޮ͍͍ͯΔ าಓʹಥͬࠐ ·ͳ͘ͳͬͨ
นʹಥͬࠐ ·ͳ͘ͳͬͨ ରंઢʹ ৵ೖ͠ͳ͘ͳͬͨ ΨʔυϨʔϧ ʹಥͬࠐ·ͳ ͘ͳͬͨ
© 2024 Wantedly, Inc. ૬มΘΒͣ͏·͍͔͘ͳ͍ྫ - ͦͷʹ͓͚Δঢ়گѲ͕ͳ͔ͳ͔͍͠ ࠨ͔Β ं͕ग़͖ͯͨ
τϥοΫͰ ৴߸͕ݟ͑ͳ͍ ETCϨʔϯ ঃߦ͠ͳ͍ͱ ͍͚ͳ͍ ԣஅาಓۙ͘ʹ ௨ߦਓ͍ͳ͍
© 2024 Wantedly, Inc. ຊίϯϖʹର͢ΔऔΓΈํʹ͍ͭͯ
© 2024 Wantedly, Inc. എܠ ࢠͲ͕ੜ·Ε͔ͯΒɺॳΊͯͷσʔλੳίϯϖͷࢀՃ ύύKagglerʹͳΓ·ͨ͠
© 2024 Wantedly, Inc. എܠ ͔ͤͬ͘ࢀՃ͢ΔͳΒPrizeݍʹೖΓ͍ͨ… Ͱ • ͕ͬͭΓίʔυΛॻ͚Δͷɺൺֱత͘৸ͯ͘ΕΔਂͷΈ •
։࠵ظؒͷલͱޙՈఉͷ༻ࣄͰ1த͕࣌ؒऔΕͳ͍ ͋Μ·Γ࣌ؒऔΕͳ͍ɺͲ͏͠Α͏
© 2024 Wantedly, Inc. Ͳ͏औΓΉ͖͔ Do everything
© 2024 Wantedly, Inc. Ͳ͏औΓΉ͖͔ Do everything Δ͜ͱɾΒͳ͍ ͜ͱΛܾΊΔ
© 2024 Wantedly, Inc. ελϯε Δ͜ͱ • ڝ૪༏ҐͱͳΔٕज़՝ʢղܾ͖͍͢ʣΛਪఆ͠ɺͦΕʹṌ͚ͯऔΓΉ • ֎ΕͨΒૉʹఘΊΔɺΘΜͪΌΜϗʔϜϥϯͶΒ͍
Βͳ͍͜ͱ • ࡉ͔͍վળɺϋΠύϥνϡʔχϯάͳͲ • ܭࢉϦιʔε͕ۭ͍͍ͯͯɺͳΜͱͳ͘Ͱ࣮ݧΛճ͞ͳ͍Α͏ʹ͢Δ
© 2024 Wantedly, Inc. Ͳ͏͍͏՝Λղ͖͔͘Λઃఆ͢Δ Ͳ͏͍͏͍ʢnot Ξϓϩʔνʣ͕ࠩผԽϙΠϯτʹͳΔͷ͔ߟ͑Δ • ΞΠσΞΛεϙοτతʹݕূ͢ΔΑΓɺूத͢Δ͖՝Λઃఆͯ͠ਂ΅ͬ ͨ΄͏͕ɺదͳΞϓϩʔνʹͨͲΓண͖͍͢
ੳޙʹઃఆͨ͠՝ QɿલޙͷγʔϯͷมԽظͷΛ֫ಘ͢Δͷʹ༗ޮ͔ʁ Qɿӡస࣌ͷঢ়گ༧ଌʹͲͷΑ͏ʹӨڹ͢Δͷ͔ʁʢྫ͑ߴಓ࿏ͩͱʁʣ
© 2024 Wantedly, Inc. ੜAIػೳͰ࣮ݧεϐʔυΛૣ͘͢Δ ࣮ݧαΠΫϧ͕ैདྷͷ1/2~1/3ͷ࣌ؒͰճͤΔΑ͏ʹ • ࣮ݧͷઃܭ͔ΒݕূʢσόοάʣʹࢸΔ·Ͱͷ࣌ؒͷେ෯ͳॖ • ΊΜͲ͘͞…
ͱ͍͏৺ཧతϋʔυϧΛେ෯ʹԼ͛Δʢਖ਼͜Ε͕େ͖͍ʣ • ྫ͑ɺID୯Ґ→Scene୯ҐͷมߋɺมߋՕॴ͕ଟͯ͘ਏ͍ • Ͳ͏ઃܭ͢Δ͖͔ɺͲ͏͍͏มߋՕॴ͕͋Δ͔ɺͲ͏࣮͢Δ͔Λ͑ ͯΒ͏ɻͦͯ͠ίέͨΒσόοάͷࡐྉΛΒ͏ ΞΠσΞͷ ݕ౼ ઃܭ ࣮ ݕূ