Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Turing × atmaCup #18 - 1st Place Solution
Search
Shuhei Goda
December 13, 2024
Technology
0
700
Turing × atmaCup #18 - 1st Place Solution
Turing × atmaCup #18 の表彰式での登壇資料です
https://turing.connpass.com/event/338583/
Shuhei Goda
December 13, 2024
Tweet
Share
More Decks by Shuhei Goda
See All by Shuhei Goda
ジョブマッチングサービスにおける相互推薦システムの応用事例と課題
hakubishin3
3
820
とある事業会社にとっての Kaggler の魅力
hakubishin3
8
2.1k
課題の解像度が荒かったことで意図した改善ができなかった話
hakubishin3
3
950
Wantedly におけるマッチング体験を最大化させるための推薦システム
hakubishin3
4
1.1k
Recommendation Industry Talks #1 Opening
hakubishin3
1
350
会社訪問アプリ「Wantedly Visit」での シゴトに関する興味選択機能と推薦改善
hakubishin3
0
590
論文紹介: Improving Implicit Feedback-Based Recommendation through Multi-Behavior Alignment(Xin Xin et al., 2023)
hakubishin3
0
560
Feedback Prize - English Language Learning における擬似ラベルの品質向上の取り組み
hakubishin3
0
930
ウォンテッドリーにおける推薦システムのオフライン評価の仕組み
hakubishin3
7
6.7k
Other Decks in Technology
See All in Technology
今年一年で頑張ること / What I will do my best this year
pauli
1
240
【NGK2025S】動物園(PINTO_model_zoo)に遊びに行こう
kazuhitotakahashi
0
310
0→1事業こそPMは営業すべし / pmconf #落選お披露目 / PM should do sales in zero to one
roki_n_
PRO
1
2.2k
JuliaTokaiとJuliaLangJaの紹介 for NGK2025S
antimon2
1
150
Plants vs thieves: Automated Tests in the World of Web Security
leichteckig
0
130
機械学習を「社会実装」するということ 2025年版 / Social Implementation of Machine Learning 2025 Version
moepy_stats
14
3k
新卒1年目、はじめてのアプリケーションサーバー【IBM WebSphere Liberty】
ktgrryt
0
160
AWSマルチアカウント統制環境のすゝめ / 20250115 Mitsutoshi Matsuo
shift_evolve
0
120
Oracle Exadata Database Service(Dedicated Infrastructure):サービス概要のご紹介
oracle4engineer
PRO
0
12k
RubyでKubernetesプログラミング
sat
PRO
4
160
月間60万ユーザーを抱える 個人開発サービス「Walica」の 技術スタック変遷
miyachin
2
730
LLM活用の現在とこれから:LayerXにおける事例とともに 2025/1 ver. / layerx-llm-202501
yuya4
3
180
Featured
See All Featured
Rails Girls Zürich Keynote
gr2m
94
13k
Documentation Writing (for coders)
carmenintech
67
4.5k
The Cost Of JavaScript in 2023
addyosmani
46
7.2k
Visualization
eitanlees
146
15k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
45
2.3k
Mobile First: as difficult as doing things right
swwweet
222
9k
Become a Pro
speakerdeck
PRO
26
5.1k
A Modern Web Designer's Workflow
chriscoyier
693
190k
Adopting Sorbet at Scale
ufuk
74
9.2k
A designer walks into a library…
pauljervisheath
205
24k
For a Future-Friendly Web
brad_frost
176
9.5k
4 Signs Your Business is Dying
shpigford
182
22k
Transcript
© 2024 Wantedly, Inc. 1st Place Solution Dec. 14 2024
- Shuhei Goda Turing × atmaCup #18 දজࣜ&ৼΓฦΓձ
© 2024 Wantedly, Inc. ໊લɿ ߹ా पฏ Shuhei Goda
ॴଐͱׂɿ ΥϯςουϦʔגࣜձࣾ ɾData Team Manager ɾMachine Learning Tech Lead ɾProduct Manager Kaggle Tierɿ Kaggle Competitions Grandmaster @jy_msc ࣗݾհ https://www.kaggle.com/shuheigoda
© 2024 Wantedly, Inc. Turing × atmaCup #18 ʹ͍ͭͯ •
։࠵ظؒɿ2024/11/15 17:30 ʙ 2024/11/24 18:00 • ࣗಈंͷߦγʔϯͷΧϝϥը૾ं྆ͷঢ়ଶσʔλͳͲ͔Βɺ0.5 ~ 3s ޙͷ ࣗंͷҐஔΛਪఆ͢ΔλεΫʢي༧ଌʣ
© 2024 Wantedly, Inc. ࠓճͷ Public LeaderBoard : 1st Place
🎉 Private LeaderBoard : 1st Place 🎉
© 2024 Wantedly, Inc. λΠϜϥΠϯʢ11/19͔ΒࢀՃʣ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 1.ੳͱํܾΊ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 2. 1st-stageͷվળ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 3. 2nd-stageͷվળ
© 2024 Wantedly, Inc. ϓϩηεʢͬ͘͟Γͱʣ 4. Ξϯαϯϒϧͷ४උͱ࣮ࢪ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ EDAϕʔεϥΠϯΞϓϩʔν͔ΒɺΛѲͯ͠ํΛߟ͑Δ • σʔλଟ͘ͳ͘ɺγϯϓϧͳ͕Βςʔϒϧಛྔ͕ڧͦ͏ •
ͱ͍ͬͯɺ༧ଌʹ͓͍ͯը૾ͷใ༗ޮͦ͏ʹݟ͑Δʢৄࡉޙड़ʣ → ଞࢀՃऀͱͷେ͖ͳࠩҟͱͳΓಘΔͷʮը૾ใͷѻ͍ํʯͩͱߟ͑ͨ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ ਐΊํͱ࣌ؒͷ͍ํΛܾΊΔ • 1st-stage Ϟσϧʢը૾Λѻ͏ϞσϧʣͷվળΛॏతʹΔ
• 2nd-stage ϞσϧʢςʔϒϧಛྔϝΠϯͷϞσϧʣͷվળΛগ͠Δ • ͋ͱϞσϧΛՔ͙࡞ۀΛߦ͍Ξϯαϯϒϧ͢Δ
© 2024 Wantedly, Inc. 1. ੳͱํܾΊ ࣮ݧʹ༻ͨ͠ϕʔεϥΠϯ • tk@tnkcoder ͞Μ͕ެ։ͨ͠ϕʔεϥΠϯϞσϧΛར༻
• [CV 0.2008/LB 0.2017] LightGBM + CNN stacking baseline (LightGBM + CNN) 1st-stage: CNN 2nd-stage: GBDT Image Tabular 1st-stage Predictions Submission
© 2024 Wantedly, Inc. 2. 1st-stageͷվળ
© 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤ Target ͷ࠷େͰׂͬͨ
Target Λֶशɾ༧ଌ͢Δ • ݁ՌɿX ͱ Y ͷ༧ଌੑೳ͕վળɻt ͕খ͍͞΄ͲޮՌ͕େ͖͍ • ղऍɿ༧ଌ࣌Ͱλʔήοτͷεέʔϧ͕େ͖͘ҟͳΔɻεέʔϧΛ߹ΘͤࠐΉ͜ͱͰɺ ֤༧ଌ࣌ͷใΛ·ͱΊͯޮՌతʹֶशͰ͖ΔͷͰͳ͍͔
© 2024 Wantedly, Inc. 2-1. Target ͷվળ ֤༧ଌ࣌ͷՃΛ Auxiliary Target
ͱֶͯ͠शɾ༧ଌ͢Δ • ྫ͑ Target ͷ x_0, x_1 ͔Β vx_1 Λࢉग़͢Δ͜ͱ͕Ͱ͖Δ • ݁Ռͱͯ͠ɺ1st-stage CV: 0.2312 → 0.2288 (-0.0024) ʹվળ • ·ͨɺAuxiliary Target ʹର͢Δ༧ଌΛޙஈͷಛྔͱͯ͠Ճ͢Δ͜ͱͰ ɺ2nd-stage ͷείΞ͕վળʢ2nd-stage CV: 0.1963 → 0.1933ʣ
© 2024 Wantedly, Inc. 2-2. HorizontalFlip ࢥͬͨ͜ͱɿࣗಈं͔ΒࡱӨ͞Εͨը૾ɺਫฏసͤͯ͞ҧײ͕গͳ͍ Ͳ͕ͬͪΦϦδφϧʁ
© 2024 Wantedly, Inc. 2-2. HorizontalFlip ֶश࣌ɾਪ࣌ʹ HorizontalFlip ΛՃ͑Δ •
ֶश࣌ɿp=0.5 Ͱ HorizontalFlip • ਪ࣌ɿΦϦδφϧը૾ͷਪ݁Ռͱਫฏసͨ͠ਪ݁ՌΛฏۉ͢Δ • ͜ΕΒʹΑͬͯείΞ৳ͼΔ͕ɺ1st-stage Ϟσϧʹೖྗ͢Δςʔϒϧಛྔ ͷసΛΕͯ͠·͏ͱείΞ͕ٯʹԼ͕ͬͯ͠·͏ͷͰҙ • 1st-stageͰѻ͏ςʔϒϧಛྔۃྗγϯϓϧʹ͑Δඞཁ͕͋ΔɻΘ Γʹ 2nd-stage ʹෳࡶͳFEΛدͤΔ͜ͱ͕Ͱ͖Δ {“steeringAngleDeg”: 15, “leftBlinker”: True, “rightBlinker”: False} → {“steeringAngleDeg”: -15, “leftBlinker”: False, “rightBlinker”: True}
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮తใΛ͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔͳ༧ଌͰ͖ΔΑ͏ʹͳΔͷͰʁΑΓظతͳӡసঢ়گͷѲ͕ॏཁ
ྫ1ɿ sec=2.0, t-0.5 sec=2.0, t-1.0 sec=2.0 sec=12.0 12secޙͷใ͔Βɺͦͷ··ਐ͢Ε ྑ͔ͬͨ͜ͱ͕Θ͔Δ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ࢥͬͨ͜ͱɿಉҰγʔϯͷલޙͷࢹ֮తใΛ͏͜ͱͰɺ୯ҰID(t-1.0 ~ t)͚ͩͩ ͱࠔͳ༧ଌͰ͖ΔΑ͏ʹͳΔͷͰʁΑΓظతͳӡసঢ়گͷѲ͕ॏཁ
ྫ2ɿ sec=2.0 sec=12.0 12secޙͷใ͔ΒɺࣼΊʹਐΊ ྑ͔ͬͨ͜ͱ͕Θ͔Δ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ظͷมԽʢ1ඵ୯Ґʣ ɾٸͳૢ࡞มԽ ɾՃݮ ɾंઢมߋ
ɾӈࠨં ظͷมԽʢ୯Ґʣ ɾߦత ɾӡసελΠϧ ɾతͷܦ࿏
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ϘτϧωοΫͱͳ͍ͬͯΔʮظมԽʯΛޮՌతʹϞσϧԽ͢Δʹʁ • ϕʔεϥΠϯͰ2nd-stageʹ͓͍ͯલޙͷظใΛߟྀͨ͠༧ଌ͕Մೳͩ ͕ɺ1st-stageʹ͓͍֤ͯID͕ಠཱͳͷͱͯ͠ѻ͏&ܦ࿏༧ଌͷ݁Ռͱͯ͠ͷ
ใΛൖͤ͞ΔܗʹͳΔͷͰඇޮʹݟ͑Δ • 1st-stage ͷNNͷஈ֊ͰɺظͷมԽʹجͮ͘ΛֶशͰ͖ΔΑ͏ʹ͢Δ 1st-stage: CNN 2nd-stage: GBDT sceneA,ID1 1st-stage Predictions 1st-stage: CNN 1st-stage Predictions FE(e.g. shift features) sceneA,ID2 Shared
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿScene୯ҐͰ 2.5D-CNN + LSTM
CNN 1st-stage Predictions (B×S×N) BiLSTM … Tabular sec=20 MLP sec=120 Scene
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ sec=20 sec=120 Pad Pad
Pad Pad sec=220 sec=320 sec=520 Pad Pad Pad sec=20 sec=120 sec=220 sec=320 sec=420 sec=520 scene=A scene=B scene=C όονͷ࡞Γํ • Scene͝ͱʹ͕͞ҟͳΔͷͰɺ٧ΊͯPadding • αϯϓϧؒͰ࣌ܥྻతͳҐஔ͕ؔҟͳΔͷͰɺscene_sec scene_num ʢsceneͷத ͰԿ൪ʹొͨ͠ID͔ʣΛಛྔͱͯ͠ೖྗ
© 2024 Wantedly, Inc. 2-3. Scene୯Ґ ۩ମతͳΞϓϩʔνɿ2.5D-CNN + LSTM Λ࠾༻͢Δ
• ͜ͷΞʔΩςΫνϟʹมߋ͢Δ͜ͱͰɺCNN୯ମͰ Private 4Ґ૬ͷείΞʹ ྫ1ɿ ྫ2ɿ
© 2024 Wantedly, Inc. 3. 2nd-stageͷվળ
© 2024 Wantedly, Inc. ͍͔ͭ͘ͷಛྔͷՃ ͍ͣΕͦͦ͜͜ͷվળʹد༩ͨ͠ • 1st stage ͷ
target (x_0 ~ z_5) ͷ༧ଌʹՃ͑ͯɺaux target ͷ༧ଌಛྔ ͱͯ͠ར༻͢Δ • 2छྨͷं྆ϞσϧʢϢχαΠΫϧϞσϧͱಈྗֶతόΠγΫϧϞσϧʣͷ༧ଌ ݁ՌΛಛྔͱͯ͠ར༻͢Δ
© 2024 Wantedly, Inc. 4. Ξϯαϯϒϧ
© 2024 Wantedly, Inc. Ξϯαϯϒϧ ༷ʑͳόοΫϘʔϯͰϞσϧΛ࡞ͬͯ Weighted Average • جຊతʹϞσϧΛ૿͢΄ͲείΞ͕େ͖͘৳ͼΔɻ࠷ऴతʹ11ݸࠞͥͨɻ
• ͬͨόοΫϘʔϯɿresnext, efficientnet, resnet, swin-transformer ͳͲ
© 2024 Wantedly, Inc. ࠷ऴ݁Ռ
© 2024 Wantedly, Inc. ֤ϞσϧͷύϑΥʔϚϯε model cv public private private
ॱҐ ɹsingle 1st stage 0.1906 0.1958 0.1808 4Ґ ɹsingle 2nd stage 0.1883 0.1928 0.1785 1Ґ ɹensemble 0.1792 0.1885 0.1754 1Ґ
© 2024 Wantedly, Inc. ϕʔεϥΠϯʹൺͯ͏·͍͘͘Α͏ʹͳͬͨྫ - ظతͳঢ়گѲ͕ޮ͍͍ͯΔ าಓʹಥͬࠐ ·ͳ͘ͳͬͨ
นʹಥͬࠐ ·ͳ͘ͳͬͨ ରंઢʹ ৵ೖ͠ͳ͘ͳͬͨ ΨʔυϨʔϧ ʹಥͬࠐ·ͳ ͘ͳͬͨ
© 2024 Wantedly, Inc. ૬มΘΒͣ͏·͍͔͘ͳ͍ྫ - ͦͷʹ͓͚Δঢ়گѲ͕ͳ͔ͳ͔͍͠ ࠨ͔Β ं͕ग़͖ͯͨ
τϥοΫͰ ৴߸͕ݟ͑ͳ͍ ETCϨʔϯ ঃߦ͠ͳ͍ͱ ͍͚ͳ͍ ԣஅาಓۙ͘ʹ ௨ߦਓ͍ͳ͍
© 2024 Wantedly, Inc. ຊίϯϖʹର͢ΔऔΓΈํʹ͍ͭͯ
© 2024 Wantedly, Inc. എܠ ࢠͲ͕ੜ·Ε͔ͯΒɺॳΊͯͷσʔλੳίϯϖͷࢀՃ ύύKagglerʹͳΓ·ͨ͠
© 2024 Wantedly, Inc. എܠ ͔ͤͬ͘ࢀՃ͢ΔͳΒPrizeݍʹೖΓ͍ͨ… Ͱ • ͕ͬͭΓίʔυΛॻ͚Δͷɺൺֱత͘৸ͯ͘ΕΔਂͷΈ •
։࠵ظؒͷલͱޙՈఉͷ༻ࣄͰ1த͕࣌ؒऔΕͳ͍ ͋Μ·Γ࣌ؒऔΕͳ͍ɺͲ͏͠Α͏
© 2024 Wantedly, Inc. Ͳ͏औΓΉ͖͔ Do everything
© 2024 Wantedly, Inc. Ͳ͏औΓΉ͖͔ Do everything Δ͜ͱɾΒͳ͍ ͜ͱΛܾΊΔ
© 2024 Wantedly, Inc. ελϯε Δ͜ͱ • ڝ૪༏ҐͱͳΔٕज़՝ʢղܾ͖͍͢ʣΛਪఆ͠ɺͦΕʹṌ͚ͯऔΓΉ • ֎ΕͨΒૉʹఘΊΔɺΘΜͪΌΜϗʔϜϥϯͶΒ͍
Βͳ͍͜ͱ • ࡉ͔͍վળɺϋΠύϥνϡʔχϯάͳͲ • ܭࢉϦιʔε͕ۭ͍͍ͯͯɺͳΜͱͳ͘Ͱ࣮ݧΛճ͞ͳ͍Α͏ʹ͢Δ
© 2024 Wantedly, Inc. Ͳ͏͍͏՝Λղ͖͔͘Λઃఆ͢Δ Ͳ͏͍͏͍ʢnot Ξϓϩʔνʣ͕ࠩผԽϙΠϯτʹͳΔͷ͔ߟ͑Δ • ΞΠσΞΛεϙοτతʹݕূ͢ΔΑΓɺूத͢Δ͖՝Λઃఆͯ͠ਂ΅ͬ ͨ΄͏͕ɺదͳΞϓϩʔνʹͨͲΓண͖͍͢
ੳޙʹઃఆͨ͠՝ QɿલޙͷγʔϯͷมԽظͷΛ֫ಘ͢Δͷʹ༗ޮ͔ʁ Qɿӡస࣌ͷঢ়گ༧ଌʹͲͷΑ͏ʹӨڹ͢Δͷ͔ʁʢྫ͑ߴಓ࿏ͩͱʁʣ
© 2024 Wantedly, Inc. ੜAIػೳͰ࣮ݧεϐʔυΛૣ͘͢Δ ࣮ݧαΠΫϧ͕ैདྷͷ1/2~1/3ͷ࣌ؒͰճͤΔΑ͏ʹ • ࣮ݧͷઃܭ͔ΒݕূʢσόοάʣʹࢸΔ·Ͱͷ࣌ؒͷେ෯ͳॖ • ΊΜͲ͘͞…
ͱ͍͏৺ཧతϋʔυϧΛେ෯ʹԼ͛Δʢਖ਼͜Ε͕େ͖͍ʣ • ྫ͑ɺID୯Ґ→Scene୯ҐͷมߋɺมߋՕॴ͕ଟͯ͘ਏ͍ • Ͳ͏ઃܭ͢Δ͖͔ɺͲ͏͍͏มߋՕॴ͕͋Δ͔ɺͲ͏࣮͢Δ͔Λ͑ ͯΒ͏ɻͦͯ͠ίέͨΒσόοάͷࡐྉΛΒ͏ ΞΠσΞͷ ݕ౼ ઃܭ ࣮ ݕূ