Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
model_pipeline_final.pdf
Search
Maxwell
September 18, 2018
Science
1
200
model_pipeline_final.pdf
model pipeline and others in Home Credit Default Risk competition.
Thanks to team mates.
Maxwell
September 18, 2018
Tweet
Share
More Decks by Maxwell
See All by Maxwell
Causal Impact -paper summary-
hoxomaxwell
2
600
Great Barrier Reef Model Pipeline: 15th place
hoxomaxwell
1
160
Lecture materials at the University of Tokyo School of Medicine
hoxomaxwell
1
110
Kaggle Hungry Geese
hoxomaxwell
1
84
HuBMAP 17th place model pipeline
hoxomaxwell
1
69
LT: Shallow Dive into Bayes Factor
hoxomaxwell
6
1.2k
Kaggle APTOS 2019 @ U-Tokyo Med
hoxomaxwell
1
400
Cornell Birdcall 36th place solution
hoxomaxwell
2
210
Kaggle Bengali.AI 6 th place solution
hoxomaxwell
4
8k
Other Decks in Science
See All in Science
240510 COGNAC LabChat
kazh
0
130
Introduction to Graph Neural Networks
joisino
PRO
4
2.1k
機械学習を支える連続最適化
nearme_tech
PRO
1
150
【健康&筋肉と生産性向上の関連性】 【Google Cloudを企業で運用する際の知識】 をお届け
yasumuusan
0
330
創薬における機械学習技術について
kanojikajino
13
4.4k
非同期コミュニケーションの構造 -チャットツールを用いた組織における情報の流れの設計について-
koisono
0
140
Celebrate UTIG: Staff and Student Awards 2024
utig
0
460
ICRA2024 速報
rpc
3
5.2k
証明支援系LEANに入門しよう
unaoya
0
350
Science of Scienceおよび科学計量学に関する研究論文の俯瞰可視化_LT版
hayataka88
0
930
ウェーブレットおきもち講座
aikiriao
1
790
作業領域内の障害物を回避可能なバイナリマニピュレータの設計 / Design of binary manipulator avoiding obstacles in workspace
konakalab
0
160
Featured
See All Featured
Fantastic passwords and where to find them - at NoRuKo
philnash
50
2.9k
Testing 201, or: Great Expectations
jmmastey
38
7.1k
Measuring & Analyzing Core Web Vitals
bluesmoon
4
120
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.5k
What's new in Ruby 2.0
geeforr
343
31k
Practical Orchestrator
shlominoach
186
10k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
A Philosophy of Restraint
colly
203
16k
What's in a price? How to price your products and services
michaelherold
243
12k
The Art of Programming - Codeland 2020
erikaheidi
52
13k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
191
16k
Keith and Marios Guide to Fast Websites
keithpitt
409
22k
Transcript
ikiri_DS Model PipeLine 600+1 ( LB804 ) FEATURES 1000+1 (
LB803 ) meta app meta bur Kernel GP Nejumi features Tereka features + LGBM 5 3 tosh 5 + CatBoost 5 2 1 + LGBM * 4 3 1 + CNN 7 Residual 2 + ExtTree 4 3 1 Residual 1 ( corrected with residual regression ) Blending CV 0.8094 Adversarial Stochastic Blending CV 0.8096 Adversarial Stochastic Blending CV 0.81050 * model drawn in next page + NN 1 3 ONODERA Maxwell Nejumi Tereka RK 1 2 3 4 5 6 7 Branden features 8 Branden + NN 1 3 takuoko features 9 Angus features 10 takuoko nejumi feature Angus + Res2 + LGBM 1 6 + Res1 + LGBM 1 6 1 or 2 or 5 + LGBM 1 or 2 or 5 + CatBoost or + LGBM 5 1 or 2 5 + LGBM 8 + LGBM 9 + LGBM 10 Adversarial Stochastic Blending CV : 0.8061 29.Aug.2018 Tam Tam features 11 + LGBM 11 + RGF 1 + LGBM 11 + RNN 7 1 * using hidden layer as additional features to correct residuals. + CNN 7 + hidden + Res3 + LGBM 1 6 + RGF 1 + Res2 + LGBM 1 6 + LGBM 5 RK features 12 + LGBM 12 1 or 2 12 + LGBM 8 1 or 2 8 + LGBM 3 1 5 or 3 2 5 + LGBM 8 1 12 or 8 2 12 Public 0.8085 17 th Private 0.8017 18 th + LGBM 8 + LGBM 9 + LGBM 10 Ireko DAE 13 Ireko8 + NN 1 13 + NN 1 + NN 1 13 Nejumi prediction Public 0.8093 10 th Private 0.8016 18 th Public 0.8080 23 th Private 0.8028 14 th + RNN 7 1 Public 0.8110 3 rd Private 0.8042 5 th Giba Post Processing Public 2nd 0.81241 Private 2nd 0.80561 Home Credit Default Risk partial partial partial + LGBM 8 1 or 2 8 or 12 + LGBM 3 1 or 2 3 or 12 3 + LGBM 6 1 Residual 3 + hidden + LGBM 1 6' or 6' 1 + LGBM 6' 2 Blending
ikiri_DS Model PipeLine 600+1 ( LB804 ) FEATURES 1000+1 (
LB803 ) meta app meta bur Kernel GP Nejumi features Tereka features tosh + LGBM * 4 3 1 + CNN 7 Residual 2 Residual 1 ( corrected with residual regression ) Blending CV 0.8085 Adversarial Stochastic Blending CV 0.8085 Adversarial Stochastic Blending CV 0.8097 * model drawn in next page ONODERA Maxwell Nejumi Tereka RK 1 2 3 4 5 6 7 Branden features 8 Branden + NN 1 3 takuoko features 9 Angus features 10 takuoko nejumi feature Angus + Res2 + LGBM 1 6 + Res1 + LGBM 1 6 + LGBM 8 + LGBM 9 + LGBM 10 Adversarial Stochastic Blending CV : 0.8061 29.Aug.2018 Tam Tam features 11 + LGBM 11 + LGBM 11 + RNN 7 1 * using hidden layer as additional features to correct residuals. + CNN 7 + hidden + Res3 + LGBM 1 6 + RGF 1 + Res2 + LGBM 1 6 + LGBM 5 RK features 12 + LGBM 12 1 or 2 12 + LGBM 8 1 or 2 8 Public 0.8071 26 th Private 0.8009 37 th + LGBM 8 + LGBM 9 + LGBM 10 Ireko DAE 13 Ireko8 + NN 1 13 + NN 1 + NN 1 13 Nejumi prediction Public 0.8082 23 th Private 0.8022 18 th Public 0.8080 23 th Private 0.8028 14 th Public 0.8099 7 th Private 0.8040 6 th Giba Post Processing Home Credit Default Risk partial + LGBM 8 1 12 or 8 2 12 partial 1 or 2 + LGBM + LGBM 6 1 Residual 3 + hidden + LGBM 1 6' or 6' 1 + LGBM 6' 2 Blending + ExtTree 4 3 1 + NN 1 3 + RGF 1 + LGBM 4 3 2 + XGB 4 3 1 + NN 1 + RNN 7 1 + hidden + Res3 + LGBM 1 6 + Res1 + LGBM 1 6 + hidden + Res4 + LGBM 1 6 stacking with LGBM CV 0.8080 Public 0.8070 / Private 0.8015 Stacking prediction Stacking + LGBM 3 1 or 2 3
application bureau bureau balance AUC : 0.683 (SEED71) 0.683 (SEEDs
avg) AUC 0.772 (SEED71) 0.773 (SEEDs avg) XGBoost app meta feature XGBoost prev meta feature 229 features 300 features all data stacking-like Light GBM 5 stratified fold ( shuffle = True ) 5 / 8 SEEDs rank averaged SEED : 71 for model fit SEED : 710, 711, 712, 713, 714 ( 715, 716, 717 ) for OOF prediction hyper parameter tuned for 603 features (reflected on meta features) XGBoost bureau meta feature ONODERA BASIC FEATURES 600 features NEJUMI FEATURES ( interest rate ) 1 feature 603 ( 604 ) features Local CV 0.80641 Public LB / Private LB 0.80569 / 0.79853 100 th / 105 th AUC 0.710 (SEED71) 0.712 (SEEDs avg) previous inst POS_CASH credit 952 features Local CV 0.80646 LB 0.804 ( ~ 0.805 ) Maxwell 603 ( 604 ) selected features based on ONODERA criteria w/o feature selection Stacking-like Light GBM