Kaggle PetFinder 2nd Place Solution

Adoption Prediction 2nd Place Solution Kaggle Tokyo Meetup #6 July
13th, 2019 [kaggler-ja] Wodori

ຊൃදʹ͍ͭͯ • ,BHHMF%JTDVTTJPO IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPO QSFEJDUJPOEJTDVTTJPO • ϙΤϜύʔτ IUUQTLBFSV OBOUPLBIBUFOBCMPHDPNFOUSZ

໨࣍ ίϯϖ֓ཁͱ݁Ռ νʔϜղ๏ͷ঺հ ೖ৆ޙͷల։

ίϯϖ֓ཁͱ݁Ռ

ίϯϖ֓ཁ • ೥݄຤ʙ೥݄຤ʹΘͨͬͯ։࠵ • ʮϚϨʔγΞͷϖοτγϣοϓͰͷݘɾೣ͕Ҿ͖औΒΕΔ ଎͞Λ༧ଌ͢Δʯͱ͍͏͓୊ IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPOQSFEJDUJPO

ར༻Ͱ͖Δσʔλ • ϖοτͷը૾ܗࣜσʔλ • આ໌จ΍ϖοτͷ໊લͳͲͷςΩετܗࣜσʔλ • ਎௕ɾମॏɾଐੑͳͲͷςʔϒϧܗࣜσʔλ • ඞཁʹԠͯ͡֎෦σʔλ΋ར༻Մೳ

UBSHFUͷৄࡉʢ෼ྨλεΫʣ • ݘɾೣ͕Ҿ͖औΒΕΔ଎͞ • ϦετΞοϓͷ౰೔ʢ೔໨ʣ • ϦετΞοϓͷʙ೔໨ •
ϦετΞοϓͷʙ೔໨ • ϦετΞοϓͷʙ೔໨ • Ҿ͖औΒΕͳ͍

ධՁࢦඪ • RVBESBUJDXFJHIUFELBQQBʢॏΈ෇͖Χού܎਺ʣ • ͍ۙ෼ྨͰ֎ͨ࣌͠ͷଛࣦ͸খ͍͕͞ɺԕ͍෼ྨͰ֎͢ͱ ଛࣦ͕େ͖͍ IUUQTCFMMDVSWFKQTUBUJTUJDT CMPHIUNM

TUBHF੍ ,FSOFM0OMZ • TUBHF੍ • UFTUEBUB͕׬શʹ৽͍͠σʔλʹࠩ͠ସΘΔ • ,FSOFM0OMZ • Ұ࿈ͷॲཧΛ,BHHMF಺ͷϓϩάϥϜ࣮ߦ؀ڥʹهड़
• (16࣮ߦ࣌ؒ͸I

ࢀՃऀશମͷେ·͔ͳํ਑ • ෼ྨ໰୊͕ͩɺճؼͰղ͍ͯɺ$7࠷దͳᮢ஋ΛܾΊΔ • QVCMJDLFSOFMͰ͸(#%5͕ओྲྀ • ը૾΍ςΩετ͔Βಛ௃ྔநग़͠ɺςʔϒϧͱ݁߹ • Ұ෦//ܥ΋͕͋ͬͨɺείΞ͸௿͔ͬͨ •
ࠩผԽཁҼͷҰͭͩͬͨ

݁ՌʢQVCMJDҐQSJWBUFҐʣ

νʔϜղ๏ͷ঺հ

ղ๏ͷ֓ཁ • Ϟσϧͷ3JEHFTUBDLJOH • /FVSBM/FUXPSL • -JHIU(#. • -JHIU(#. •
9(#PPTU • ͭͷޙॲཧ • 3BOLBWFSBHJOHͰᮢ஋࠷దԽ • ը૾ॏෳͷൃݟ

νʔϜϝϯόʔ • ೔ຊਓਓνʔϜʢLBHHMFSKBʣ • LBFSVSVSV ZOLUL V UBLVPLP HFHF •
೔ຊޠ 4MBDLͰ΍ΓऔΓ • εϓϨουγʔτͰλεΫͳͲΛ؅ཧ • ίʔυ͸,BHHMF ,FSOFMΛ'PSL

LFSVSVSVιϩࢀՃ • QVCMJDLFSOFMϕʔεʹಔϝμϧݍ಺ ʢऴྃΧ݄൒લʣ • ػցֶशΤϯδχΞ΁ͷస৬ʹ΋੒ޭ • స৬ॕ͍Ͱ V
ZOLUL ͱҿΈձ ʢਓͷग़ձ͍͸લճͷNFFUVQʣ IUUQTUBCFMPHDPNUPLZP""

ਓ͕αϒϚʔδ

ਓମ੍ • LFSVSVSVϞσϧΛ৳͹͢ʢਓ͕޷͖উखίϝϯτʣ • ZOLUL͸͔ΒCBTFMJOFϞσϧΛߏங • V ͸աڈίϯϖؚΊEJTDVTTJPOͳͲΛ୳ࡧ • ۜϝμϧݍ಺ʹ౸ୡ

UBLVPLP • ং൫ʹࢀՃ͍ͯͨ͠ • ਓମ੍ͷ೔ޙ

ਓମ੍ • LFSVSVSVϞσϧΛ৳͹͢ • ZOLULϞσϧ΋஀ੜ ʢZOZULϞσϧ͔Β೿ੜ͠UBLVPLPϞσϧ͕஀ੜʣ • ۚϝμϧ·Ͱɺ΋͏ͻͱԡ͠ ίʔυϚʔδ
CZUBLVPLP ߴ଎Խ CZZOLUL

Ґ ҐͰνʔϜϚʔδ • ڧLFSOFMެ։Ͱएׯ-#͕ ߥΕΔதɺHFHF͞Μ͕ αϒఔ౓ͰҐ·Ͱ౸ୡ • UXJUUFSͰ%.ૹ৴ ʢ໘ࣝ͸ͳ͔ͬͨʣ

ਓମ੍ • LFSVSVSVϞσϧ • UBLVPLPϞσϧ • HFHFϞσϧ • ۚϝμϧݍ಺Ͱऴྃɺ࠷ऴ݁Ռ͸Ґ

Ҏޙɺ֤ਓʹΑΔղઆ • UZV@GFBUVSFT // -(#. • L@GFBUVSFT -(#. • H@GFBUVSFT9(#
• 3*%(&TUBDLJOHCZZOLUL • ͦͷଞ CZV

UBLVPLP

ྲྀΕ • ,FSOFMͷίʔυ͕ղಡ೉ • ZOLUL͞Μ͕શ෦ϦϑΝΫλϦϯάɻͦΕΛஔ͖౔࢈ʹதࠃ ʹཱཱྀͬͨ • UBLVPLP͕ϝΠϯͰҾ͖ܧ͗ɺਫ਼౓޲্

ಛ௃ ςΩετ • %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOU BOOPUT@EFTD CSFFE • 1VODUTͷॲཧ
FYXIBU`T XIBU`T • .JTTQFMMͷॲཧ FYXIBUTXIBUJT • 4UFNNJOHFYHPJOHHP • /VNCFSͷॲཧ FYDBUTDBUT • #08UGJEG#.47%/.'

ಛ௃ ςΩετ • %FTDSJQUJPOPOMZ • 1VODUTͷॲཧ • .JTTQFMMͷॲཧ • /VNCFSͷॲཧ
• &NCFEEJOHXJUIHMPWFBOEGBTUUFYU2VPSBSE NFUIPE • "WFSBHJOHFNCFEEJOHWFDUPSQFSSPXT

ಛ௃ ςΩετ • 2VPSBSE NFUIPE • FNCFEEJOHͰׂ͖ͨ߹ • •
ଞͷલॲཧ΋֤ϓϩηεͰ FNCFEEJOHׂ߹Λ֬ೝ͠ɺ ৗʹ্͕Δํ๏Λબ୒ͨ͠

ಛ௃ ςΩετ • ਫ਼౓্͕͕Βͳ͔ͬͨ΋ͷ • #FSU &MNP • HMPWFUXJUUFSWFDUPSͳͲଞͷFNCFEEJOHT QZNBHOJUVEF
• 4DSBUDIͰֶशͨ͠XW GBTUUFYU • .FUBUFYUGFBUVSFTͷFNCFEEJOHT • -FOHUIͳͲͷUFYU౷ܭྔ • தࠃޠͷલॲཧ

ಛ௃ ը૾ • $//GFBUVSFT • ,FSBT EFOTFOFU *ODFQUJPO3FTOFU7 • ԼهϕϯνϚʔΫΛݩʹɺਫ਼౓ͷߴ͍$//Ϟσϧ͔Βॱʹݕ౼
• IUUQTHJUIVCDPNDHOPSUIDVUUCFODINBSLJOHLFSBTQZUPSDI

ಛ௃ ը૾ • ਫ਼౓্͕͕Βͳ͔ͬͨ΋ͷ • ଞͷ$//NPEFM LFSBTQZUPSDI • /*." •
"WJUPͰ࢖ΘΕ͍ͯͨը૾ͷ౷ܭྔ XIJUFOFTT EVMMOFTT FUDʜ

ಛ௃ ςʔϒϧ • "HHSFHBUJPO • 4JNQMFNFUIPE • 3BUJPNFUIPE • Ձ֨
CSFFEࣲݘ ࣲݘͷฏۉֹͷ৔߹ SBUJP • %JGGNFUIPE • Ձ֨ CSFFEࣲݘ ࣲݘͷฏۉֹͷ৔߹ EJGG

ಛ௃ ςʔϒϧ • ΧςΰϦͷڞى -%"47% • શΧςΰϦͷ૊Έ߹Θͤʢ࣍·Ͱʣ • JOUFSBDUJPOTGFBUVSFT
• FY"HF 2VBOUJUZ • FYUFSOBMEBUB • ֆจࣈ • 4UBUF #SFFE #SFFESBOLJOH

ಛ௃ ͦͷଞ • ,FSOFMͰ͸σʔλʹରͯ͠ɺը૾ຕɺNFUBKTPOGJMF΋ ݸ͔͠࢖༻͍ͯ͠ͳ͔ͬͨ • ࣮ࡍʹ͸ෳ਺͋ΓɺͦΕΒ͸શ෦࢖༻ͯ͠ฏۉΛऔͬͨ • ͋Δσʔλ͸શ෦࢖͏͜ͱ͕ॏཁʂ •
ಛ௃ੜ੒ʹUZV QBSU͚ͩͰTFD͔͚ͨͷͰɺଞΛ͍ͩ Ϳѹഭͨ͠PS[

ಛ௃ ಛ௃બ୒ • શ෦Ͱdͷಛ௃ • -(#.ͷJNQPSUBODFUPQΛબ୒ • BMM
ͳͲͰ୳ࡧ͠ɺ$7ͱ-#ΛݟܾͯΊͨ • ࣮͸΋ͬͱগͳ͍ํ͕$7ͱQSJWBUF͸Α͔ͬͨ

Ϟσϧ • -(#.ͱ// • #FTUTJOHMF͸-(#.ͰUI //ͰUI • ΞϯαϯϒϧͰ࢖༻ͯͨ͠ͷ͸-(#.ͰUI //͸Ґ͘Β͍ ͷ΍ͭ
ಛʹUI //͸$7ߴ͔͕ͬͨ1VCMJD͕গ͠Լ͕ͬͨ • ֶश࣌ؒ͸//ͰTFD -(#.ͰTFD • -(#.ͷύϥϝʔλ • ,FSOFMͷؙύΫϦ

Ϟσϧ // • 0QUJNJ[FS"EBN • -PTT3.4& • 4DIFEVMFS$-3

7JTVBMJ[F 'FBUVSFJNQPSUBODF 1%1

LBFSVSVSV

ํ਑ • $//GFBUVSFT ࣍ݩ EPDWFD ࣍ݩ UGJEG ͨ͘͞Μ
• ߴ࣍ݩϕΫτϧΛ௿࣍ݩʹ௵ͯ͠ར༻͢Δ͜ͱʹࣥ৺ • NFBO WBSͳͲͷ౷ܭྔ΍ 47%ͳͲΛଟ༻͠ɺTDPSF͕৳ͼͨ΋ͷ ͚ͩར༻ ݁Ռతʹ࣌ؒઅ໿ʹ • "HHSFHBUJPOͱͯ΋ޮ͖ͦ͏ • UFYU౷ܭྔಛ௃ͳͲ΋ "HHSFHBUJPOͨ͠ • $7ͪΐͬͱ্͕ͬͨ

ಛ௃ ը૾ • $//GFBUVSFT • ,FSBTEFOTFOFU • 47%Ͱ ࣍ݩ ࣍ݩʹѹॖ

ಛ௃ ը૾ • "WJUPίϯϖͷΧʔωϧͰݟ͚ͭͨը૾ͷαΠζಛ௃ • TJ[F XJEUI IFJHIU ΛΧʔωϧͰެ։ •
LBFSVSVSV NPEFMʹ͸ޮ͔ͣ΋ɺ࠷ޙʹՃೖͨ͠ HFHFࢯ ͕ϕʔεʹ͍ͯͨ͠ΧʔωϧͰ࢖ΘΕ͍ͯͨ

ಛ௃ ςΩετ • γϯϓϧͳલॲཧ • TUPQ@XPSET จࣈ ͷআ֎ •
খจࣈԽɺվߦ λϒ ਺ࣈͷ࡟আ • ه߸ͷஔ׵

ಛ௃ ςΩετ • %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOU BOOPUT@EFTD • -FOHUIͳͲͷ UFYU౷ܭྔ •
-FOHUIͳͲͷ UFYU౷ܭྔΛ 3FTDVFS*% Ͱ HSPVQCZ • UGJEG47%Ͱѹॖ

ಛ௃ ςΩετ • %FTDSJQUJPOPOMZ • %PDWFD࣍ݩͷ෼ࢄදݱΛ NFBO WBSͳͲͰڧҾ ʹ࣍ݩΛམͱͯ͠ར༻

ಛ௃ ςʔϒϧ • "HHSFHBUJPO • 3FTDVFS*% Λ LFZʹ • ໟͷ௕͞ɺେ͖͞ɺख਺ྉɺ݈߁ঢ়ଶͳͲઆ໌ޮ͖ͦ͏ͳ΋ͷ
ʹద༻ ৭ɺੑผͳͲ͸ º • μϝԡ͠Ͱઆ໌จͷ-FOHUIͳͲʹ΋ద༻

8IBUEJEO`UXPSL • ը૾ • 1Z5PSDI൛ EFOTFOFU • EBUB@MPBEFSͰͷฒྻॲཧ OVN@XPSLFS૿ͰΤϥʔ
• ςΩετ • 4$%7 87 TDSBUDI

8IBUEJEO`UXPSL • &YUFSOBMEBUB • DVUFͰݕࡧͯ͠ग़͖ͯͨݘೣը૾σʔληοτͰֶशͨ͠ NPEFMͰ QSFEJDUͯ͠ <bDVUF`>ಛ௃ΛಘΔ • EBUBTFUͷDMFBOJOH͕଍Γͳ͔ͬͨ

'FBUVSFJNQPSUBODF

͓·͚ • 4FFEΨνϟͰ΋ۚݍʢ΄΅ೖ৆ݍʣ trial seed kaeru_seed Private Ranking 6
31416 58 0.44945 3rd 7 310416 1033 0.44659 6th 8 310416 1324 0.44955 3rd 9 310416 1604 0.45096 2nd 10 310416 1757 0.45491 2nd trial seed kaeru_seed Private Ranking 1 77777 133777 0.45358 2nd 2 31414 1625 0.45072 2nd 3 31415 1433 0.44914 3rd 4 31415 1631 0.45271 2nd 5 31415 1850 0.44837 5th

શମ૾ • UFTUͬΆ͘ͳ͍USBJOσʔλΛ࡟ͬͨ *NBHF 5BCMF 5FYU %FOTFOFU *NBHFTJ[F
OVN 47% &YUFSOBM "SFB 1PQVMBUJPO #SFFEEFUBJMJOGP .FUBEBUB 4FOUJNFOU 'SFRVFODZFODPEJOH 5'*%' 47% MJHIUHCN YHCPPTU USBJO EBUB ʢUFTUʹࣅ͍ͯͳ͍σʔλͷΛ࡟আʣ UFTU USBJOUFTUΛ ෼ྨ͢ΔΑ͏ʹֶश

ޮ͍ͨͷʁ • 1VCMJDͰ͸େ͖͘ޮ͍͕ͨɺ1SJWBUFͰ͸ޮ͔ͳ͔ͬͨ • -BUFTVCNJTTJPOͯ͠Έ͕ͨɺ݁Ռ͸΄΅มΘΒͣʢҙຯͳ͠ʣ • ΍ͨΒͱอޢ਺͕ଟ͍3FTDVFS*%Λ࡟͍ͬͯΔΑ͏ʹݟ͑ͨ • ݁ہɺγϯάϧͩͱಔϝμϧݍ

্ख͍͔͘ͳ͔ͬͨू • جຊ͏·͍͔͘ͳ͔ͬͨ • &YUFSOBMEBUBΛ໊دͤͯ͠΋Βͬͨ΍ͭ • 4FFEBWFSBHJOHʢ28,͕ෆ҆ఆͩͬͨͷͰ΍ͬͯΈ͕ͨʣ • ݘೣͷ৭Λॱংई౓ʹ͢Δʢྫɿനˠփ৭ˠࠇʣ •
قઅੑʢςΩετ͔Β+BOVBSZͱ͔ൈ͖औΔʣ • ͳͲ

3JEHF4UBDLJOH • 4JOHMFNPEFM͸͍͍ͤͥNPEFM • $7είΞΛݟͯɺ4UBDLFSΛݕ౼ • -(#."WFSBHF3JEHF • #FTU4JOHMF -(#.
$7 1SJWBUF UI • #FTU4UBDLJOH$7 1SJWBUF OE

3BOLBWFSBHJOH • 3JEHF΁ͷೖྗͷલॲཧͱͯ͠ɺਖ਼نԽΛߦͬͨ • TDJQZTUBUTSBOLEBUBΛ࠾༻ • SBOLԽ͢Δ͜ͱͰɺGPME͝ͱͷ༧ଌ஋ͷ෼෍ͷࠩΛٵऩ • ධՁࢦඪͷ28,͸ᮢ஋ͷӨڹΛड͚΍͘͢ɺ༧ଌ஋ͷ෼෍ ͷӨڹ͸େ͖͍

1PTU1SPDFTTJOH • 5SBJOͱ5FTUʹॏෳ͢Δը૾͕ଘࡏ • TU TUBHFͰ͸ɺຕҎ্ • OE TUBHFͰ͸ɺຕҎ্ •
3FTDVFS*%͸ผ • ࠨͷྫͰ͸ɺ౤ߘ೔ʹ೔ͷ։͖ Train Test

1PTU1SPDFTTJOH • JNBHFIBTIͱ͍͏ϥΠϒϥϦΛ࢖͍ɺॏෳը૾Λ୳ࡧ • ,FSOFMͷ࣮ߦ࣌ؒͱͷ݉Ͷ߹͍ͰɺαϜωΛؚΊຕ໨·Ͱͷը૾ Λ࢖༻ • ॏෳ͕͋Δ৔߹ɺ5FTUσʔλͷ༧ଌ஋ΛରԠ͢Δ5SBJOσʔλͷ໨ తม਺ͷ஋ʹஔ͖׵͑Δ QVCMJD
QSJWBUF

΍ͬͨ͜ͱ • టष͍͜ͱॾʑ୲౰ • աڈίϯϖؚΊͯEJTDVTTJPOΛ௥͏ • FYUFSOBM EBUBͷ࡞੒ • σʔλΫϨϯδϯά
• ݱঢ়੔ཧɺλεΫ؅ཧ • PETBJΛݟΔ • UXJUUFS • ೣΧϑΣʹߦ͘ IUUQXXXOFDPSFQVCMJDKQBDDFTTIUNMUPLZP

σʔλΫϨϯδϯά • UZQPࣙॻΛਓྗͰ࡞ΔͨΊͷ5JQT IUUQTVQVSBIBUFOBCMPHDPNFOUSZ • σʔληοτͷ௲Γϛε͸ඞͣ௚͢΂͖͔ʁ IUUQTVQVSBIBUFOBCMPHDPNFOUSZ

ೖ৆ޙͷల։

ೖ৆ޙͷల։ ࠶ݱίʔυɾղ๏ࢿྉͷఏग़ 8JOOFS`T$BMM 1FUGJOEFSͰͷ׆༻

ऴྃ௚ޙʹ࿈བྷ͕དྷΔ

࠶ݱίʔυɾղ๏ࢿྉͷఏग़ • ࠶ݱίʔυ • ,FSOFM0OMZίϯϖͳͷͰɺఏग़ແ͠Ͱ0,ͩͬͨ • ʢී௨ͷίϯϖͰ΍Δ࡞ۀΛίϯϖதʹڧ͍ΒΕ͍ͯΔʁʣ • ղ๏ࢿྉ •
1PXFSQPJOUͰϖʔδऑ • #BDLHSPVOE4VNNBSZ'FBUVSFTFMFDUJPOFOHJOFFSJOH 5SBJOJOHNFUIPET*NQPSUBOUGJOEJOHT4JNQMFNPEFM

8JOOFS`TDBMM • ෼ϓϨθϯ • ෼࣭ٙ

1FUGJOEFSͰͷ׆༻ • ೖ৆ऀͷϞσϧΛ౷߹͠ lVMUJNBUFNPEFMz Λ࣮૷த • ϖοτͷૣظҾ͖औΓʹܨ͛Δ໨త ϖοτͷϓϩϑΟʔϧΛείΞ෇͚
ԿΛमਖ਼͢΂͖͔ఏҊʢը૾ɾςΩετͳͲʣ • 1FUGJOEFSͷػցֶशϓϩδΣΫτϖʔδʹࣸਅ͕ࡌͬͨ

Thank you for all!

Kaggle PetFinder 2nd Place Solution

Kaggle PetFinder 2nd Place Solution

More Decks by u++

Other Decks in Technology

Featured

Transcript