Kaggle Tokyo Meetup #6 の発表資料 https://connpass.com/event/132935/
- https://www.kaggle.com/c/petfinder-adoption-prediction/discussion/88773 - https://kaeru-nantoka.hatenablog.com/entry/2019/04/11/031254
Adoption Prediction2nd Place SolutionKaggle Tokyo Meetup #6July 13th, 2019[kaggler-ja] Wodori
View Slide
ຊൃදʹ͍ͭͯ• ,BHHMF%JTDVTTJPOIUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPOQSFEJDUJPOEJTDVTTJPO• ϙΤϜύʔτIUUQTLBFSVOBOUPLBIBUFOBCMPHDPNFOUSZ
࣍ ίϯϖ֓ཁͱ݁Ռ νʔϜղ๏ͷհ ೖޙͷల։
ίϯϖ֓ཁͱ݁Ռ
ίϯϖ֓ཁ• ݄ʙ݄ʹΘͨͬͯ։࠵• ʮϚϨʔγΞͷϖοτγϣοϓͰͷݘɾೣ͕Ҿ͖औΒΕΔ͞Λ༧ଌ͢Δʯͱ͍͏͓IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPOQSFEJDUJPO
ར༻Ͱ͖Δσʔλ• ϖοτͷը૾ܗࣜσʔλ• આ໌จϖοτͷ໊લͳͲͷςΩετܗࣜσʔλ• ɾମॏɾଐੑͳͲͷςʔϒϧܗࣜσʔλ• ඞཁʹԠͯ͡֎෦σʔλར༻Մೳ
UBSHFUͷৄࡉʢྨλεΫʣ• ݘɾೣ͕Ҿ͖औΒΕΔ͞• ϦετΞοϓͷʢʣ• ϦετΞοϓͷʙ• ϦετΞοϓͷʙ• ϦετΞοϓͷʙ• Ҿ͖औΒΕͳ͍
ධՁࢦඪ• RVBESBUJDXFJHIUFELBQQBʢॏΈ͖Χούʣ• ͍ۙྨͰ֎ͨ࣌͠ͷଛࣦখ͍͕͞ɺԕ͍ྨͰ֎͢ͱଛࣦ͕େ͖͍IUUQTCFMMDVSWFKQTUBUJTUJDTCMPHIUNM
TUBHF੍ ,FSOFM0OMZ• TUBHF੍• UFTUEBUB͕શʹ৽͍͠σʔλʹࠩ͠ସΘΔ• ,FSOFM0OMZ• Ұ࿈ͷॲཧΛ,BHHMFͷϓϩάϥϜ࣮ߦڥʹهड़• (16࣮ߦ࣌ؒI
ࢀՃऀશମͷେ·͔ͳํ• ྨ͕ͩɺճؼͰղ͍ͯɺ$7࠷దͳᮢΛܾΊΔ• QVCMJDLFSOFMͰ(#%5͕ओྲྀ• ը૾ςΩετ͔Βಛྔநग़͠ɺςʔϒϧͱ݁߹• Ұ෦//ܥ͕͋ͬͨɺείΞ͔ͬͨ• ࠩผԽཁҼͷҰͭͩͬͨ
݁ՌʢQVCMJDҐQSJWBUFҐʣ
νʔϜղ๏ͷհ
ղ๏ͷ֓ཁ• Ϟσϧͷ3JEHFTUBDLJOH• /FVSBM/FUXPSL• -JHIU(#.• -JHIU(#.• 9(#PPTU• ͭͷޙॲཧ• 3BOLBWFSBHJOHͰᮢ࠷దԽ• ը૾ॏෳͷൃݟ
νʔϜϝϯόʔ• ຊਓਓνʔϜʢLBHHMFSKBʣ• LBFSVSVSV ZOLUL V UBLVPLP HFHF• ຊޠ 4MBDLͰΓऔΓ• εϓϨουγʔτͰλεΫͳͲΛཧ• ίʔυ,BHHMF ,FSOFMΛ'PSL
LFSVSVSVιϩࢀՃ• QVCMJDLFSOFMϕʔεʹಔϝμϧݍʢऴྃΧ݄લʣ• ػցֶशΤϯδχΞͷస৬ʹޭ• స৬ॕ͍Ͱ V ZOLUL ͱҿΈձʢਓͷग़ձ͍લճͷNFFUVQʣIUUQTUBCFMPHDPNUPLZP""
ਓ͕αϒϚʔδ
ਓମ੍• LFSVSVSVϞσϧΛ৳͢ʢਓ͕͖উखίϝϯτʣ• ZOLUL͔ΒCBTFMJOFϞσϧΛߏங• VաڈίϯϖؚΊEJTDVTTJPOͳͲΛ୳ࡧ• ۜϝμϧݍʹ౸ୡ
UBLVPLP• ং൫ʹࢀՃ͍ͯͨ͠• ਓମ੍ͷޙ
ਓମ੍• LFSVSVSVϞσϧΛ৳͢• ZOLULϞσϧੜʢZOZULϞσϧ͔Βੜ͠UBLVPLPϞσϧ͕ੜʣ• ۚϝμϧ·Ͱɺ͏ͻͱԡ͠ίʔυϚʔδ CZUBLVPLPߴԽ CZZOLUL
ҐҐͰνʔϜϚʔδ • ڧLFSOFMެ։Ͱएׯ-#͕ߥΕΔதɺHFHF͞Μ͕αϒఔͰҐ·Ͱ౸ୡ• UXJUUFSͰ%.ૹ৴ʢ໘ࣝͳ͔ͬͨʣ
ਓମ੍• LFSVSVSVϞσϧ• UBLVPLPϞσϧ• HFHFϞσϧ• ۚϝμϧݍͰऴྃɺ࠷ऴ݁ՌҐ
Ҏޙɺ֤ਓʹΑΔղઆ• UZV@GFBUVSFT // -(#.• L@GFBUVSFT -(#.• H@GFBUVSFT9(#• 3*%(&TUBDLJOHCZZOLUL• ͦͷଞ CZV
UBLVPLP
ྲྀΕ• ,FSOFMͷίʔυ͕ղಡ• ZOLUL͞Μ͕શ෦ϦϑΝΫλϦϯάɻͦΕΛஔ͖࢈ʹதࠃʹཱཱྀͬͨ• UBLVPLP͕ϝΠϯͰҾ͖ܧ͗ɺਫ਼্
ಛ ςΩετ• %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOUBOOPUT@EFTD CSFFE• 1VODUTͷॲཧ FYXIBU`T XIBU`T• .JTTQFMMͷॲཧ FYXIBUTXIBUJT• 4UFNNJOHFYHPJOHHP• /VNCFSͷॲཧ FYDBUTDBUT• #08UGJEG#.47%/.'
ಛ ςΩετ• %FTDSJQUJPOPOMZ• 1VODUTͷॲཧ• .JTTQFMMͷॲཧ• /VNCFSͷॲཧ• &NCFEEJOHXJUIHMPWFBOEGBTUUFYU2VPSBSE NFUIPE• "WFSBHJOHFNCFEEJOHWFDUPSQFSSPXT
ಛ ςΩετ• 2VPSBSE NFUIPE• FNCFEEJOHͰׂ͖ͨ߹• • ଞͷલॲཧ֤ϓϩηεͰFNCFEEJOHׂ߹Λ֬ೝ͠ɺৗʹ্͕Δํ๏Λબͨ͠
ಛ ςΩετ• ਫ਼্͕͕Βͳ͔ͬͨͷ• #FSU &MNP• HMPWFUXJUUFSWFDUPSͳͲଞͷFNCFEEJOHT QZNBHOJUVEF• 4DSBUDIͰֶशͨ͠XW GBTUUFYU• .FUBUFYUGFBUVSFTͷFNCFEEJOHT• -FOHUIͳͲͷUFYU౷ܭྔ• தࠃޠͷલॲཧ
ಛ ը૾• $//GFBUVSFT• ,FSBT EFOTFOFU *ODFQUJPO3FTOFU7• ԼهϕϯνϚʔΫΛݩʹɺਫ਼ͷߴ͍$//Ϟσϧ͔Βॱʹݕ౼• IUUQTHJUIVCDPNDHOPSUIDVUUCFODINBSLJOHLFSBTQZUPSDI
ಛ ը૾• ਫ਼্͕͕Βͳ͔ͬͨͷ• ଞͷ$//NPEFM LFSBTQZUPSDI• /*."• "WJUPͰΘΕ͍ͯͨը૾ͷ౷ܭྔ XIJUFOFTT EVMMOFTT FUDʜ
ಛ ςʔϒϧ• "HHSFHBUJPO• 4JNQMFNFUIPE• 3BUJPNFUIPE• Ձ֨ CSFFEࣲݘ ࣲݘͷฏۉֹͷ߹ SBUJP• %JGGNFUIPE• Ձ֨ CSFFEࣲݘ ࣲݘͷฏۉֹͷ߹ EJGG
ಛ ςʔϒϧ• ΧςΰϦͷڞى -%"47%• શΧςΰϦͷΈ߹Θͤʢ࣍·Ͱʣ• JOUFSBDUJPOTGFBUVSFT• FY"HF2VBOUJUZ• FYUFSOBMEBUB• ֆจࣈ• 4UBUF #SFFE #SFFESBOLJOH
ಛ ͦͷଞ• ,FSOFMͰσʔλʹରͯ͠ɺը૾ຕɺNFUBKTPOGJMFݸ͔͠༻͍ͯ͠ͳ͔ͬͨ• ࣮ࡍʹෳ͋ΓɺͦΕΒશ෦༻ͯ͠ฏۉΛऔͬͨ• ͋Δσʔλશ෦͏͜ͱ͕ॏཁʂ• ಛੜʹUZV QBSU͚ͩͰTFD͔͚ͨͷͰɺଞΛ͍ͩͿѹഭͨ͠PS[
ಛ ಛબ• શ෦Ͱdͷಛ• -(#.ͷJNQPSUBODFUPQΛબ• BMM ͳͲͰ୳ࡧ͠ɺ$7ͱ-#ΛݟܾͯΊͨ• ࣮ͬͱগͳ͍ํ͕$7ͱQSJWBUFΑ͔ͬͨ
Ϟσϧ• -(#.ͱ//• #FTUTJOHMF-(#.ͰUI //ͰUI• ΞϯαϯϒϧͰ༻ͯͨ͠ͷ-(#.ͰUI //Ґ͘Β͍ͷͭ ಛʹUI //$7ߴ͔͕ͬͨ1VCMJD͕গ͠Լ͕ͬͨ• ֶश࣌ؒ//ͰTFD -(#.ͰTFD• -(#.ͷύϥϝʔλ• ,FSOFMͷؙύΫϦ
Ϟσϧ // • 0QUJNJ[FS"EBN• -PTT3.4&• 4DIFEVMFS$-3
7JTVBMJ[F 'FBUVSFJNQPSUBODF 1%1
LBFSVSVSV
ํ• $//GFBUVSFT ࣍ݩ EPDWFD ࣍ݩ UGJEG ͨ͘͞Μ• ߴ࣍ݩϕΫτϧΛ࣍ݩʹ௵ͯ͠ར༻͢Δ͜ͱʹࣥ৺• NFBO WBSͳͲͷ౷ܭྔ 47%ͳͲΛଟ༻͠ɺTDPSF͕৳ͼͨͷ͚ͩར༻ ݁Ռతʹ࣌ؒઅʹ• "HHSFHBUJPOͱͯޮ͖ͦ͏• UFYU౷ܭྔಛͳͲ "HHSFHBUJPOͨ͠• $7ͪΐͬͱ্͕ͬͨ
ಛ ը૾• $//GFBUVSFT• ,FSBTEFOTFOFU• 47%Ͱ ࣍ݩ ࣍ݩʹѹॖ
ಛ ը૾• "WJUPίϯϖͷΧʔωϧͰݟ͚ͭͨը૾ͷαΠζಛ• TJ[F XJEUI IFJHIUΛΧʔωϧͰެ։• LBFSVSVSV NPEFMʹޮ͔ͣɺ࠷ޙʹՃೖͨ͠ HFHFࢯ͕ϕʔεʹ͍ͯͨ͠ΧʔωϧͰΘΕ͍ͯͨ
ಛ ςΩετ• γϯϓϧͳલॲཧ• TUPQ@XPSET จࣈ ͷআ֎• খจࣈԽɺվߦ λϒ ࣈͷআ• ه߸ͷஔ
ಛ ςΩετ• %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOUBOOPUT@EFTD• -FOHUIͳͲͷ UFYU౷ܭྔ• -FOHUIͳͲͷ UFYU౷ܭྔΛ 3FTDVFS*% Ͱ HSPVQCZ• UGJEG47%Ͱѹॖ
ಛ ςΩετ• %FTDSJQUJPOPOMZ• %PDWFD࣍ݩͷࢄදݱΛ NFBO WBSͳͲͰڧҾʹ࣍ݩΛམͱͯ͠ར༻
ಛ ςʔϒϧ• "HHSFHBUJPO• 3FTDVFS*% Λ LFZʹ• ໟͷ͞ɺେ͖͞ɺखྉɺ݈߁ঢ়ଶͳͲઆ໌ޮ͖ͦ͏ͳͷʹద༻ ৭ɺੑผͳͲ º • μϝԡ͠Ͱઆ໌จͷ-FOHUIͳͲʹద༻
8IBUEJEO`UXPSL• ը૾• 1Z5PSDI൛ EFOTFOFU• EBUB@MPBEFSͰͷฒྻॲཧ OVN@XPSLFS૿ͰΤϥʔ• ςΩετ• 4$%7 87 TDSBUDI
8IBUEJEO`UXPSL• &YUFSOBMEBUB• DVUFͰݕࡧͯ͠ग़͖ͯͨݘೣը૾σʔληοτͰֶशͨ͠NPEFMͰ QSFEJDUͯ͠ ಛΛಘΔ• EBUBTFUͷDMFBOJOH͕Γͳ͔ͬͨ
'FBUVSFJNQPSUBODF
͓·͚• 4FFEΨνϟͰۚݍʢ΄΅ೖݍʣtrial seed kaeru_seed Private Ranking6 31416 58 0.44945 3rd7 310416 1033 0.44659 6th8 310416 1324 0.44955 3rd9 310416 1604 0.45096 2nd10 310416 1757 0.45491 2ndtrial seed kaeru_seed Private Ranking1 77777 133777 0.45358 2nd2 31414 1625 0.45072 2nd3 31415 1433 0.44914 3rd4 31415 1631 0.45271 2nd5 31415 1850 0.44837 5th
HFHF
શମ૾• UFTUͬΆ͘ͳ͍USBJOσʔλΛͬͨ *NBHF5BCMF5FYU%FOTFOFU*NBHFTJ[F OVN47%&YUFSOBM"SFB 1PQVMBUJPO#SFFEEFUBJMJOGP.FUBEBUB4FOUJNFOU'SFRVFODZFODPEJOH5'*%' 47% MJHIUHCN YHCPPTUUSBJOEBUBʢUFTUʹࣅ͍ͯͳ͍σʔλͷΛআʣUFTUUSBJOUFTUΛྨ͢ΔΑ͏ʹֶश
ޮ͍ͨͷʁ• 1VCMJDͰେ͖͘ޮ͍͕ͨɺ1SJWBUFͰޮ͔ͳ͔ͬͨ• -BUFTVCNJTTJPOͯ͠Έ͕ͨɺ݁Ռ΄΅มΘΒͣʢҙຯͳ͠ʣ• ͨΒͱอޢ͕ଟ͍3FTDVFS*%Λ͍ͬͯΔΑ͏ʹݟ͑ͨ• ݁ہɺγϯάϧͩͱಔϝμϧݍ
্ख͍͔͘ͳ͔ͬͨू• جຊ͏·͍͔͘ͳ͔ͬͨ• &YUFSOBMEBUBΛ໊دͤͯ͠Βͬͨͭ• 4FFEBWFSBHJOHʢ28,͕ෆ҆ఆͩͬͨͷͰͬͯΈ͕ͨʣ• ݘೣͷ৭Λॱংईʹ͢Δʢྫɿനˠփ৭ˠࠇʣ• قઅੑʢςΩετ͔Β+BOVBSZͱ͔ൈ͖औΔʣ• ͳͲ
ZOLUL
3JEHF4UBDLJOH• 4JOHMFNPEFM͍͍ͤͥNPEFM• $7είΞΛݟͯɺ4UBDLFSΛݕ౼• -(#."WFSBHF3JEHF• #FTU4JOHMF -(#.$7 1SJWBUF UI• #FTU4UBDLJOH$7 1SJWBUF OE
3BOLBWFSBHJOH• 3JEHFͷೖྗͷલॲཧͱͯ͠ɺਖ਼نԽΛߦͬͨ• TDJQZTUBUTSBOLEBUBΛ࠾༻• SBOLԽ͢Δ͜ͱͰɺGPME͝ͱͷ༧ଌͷͷࠩΛٵऩ• ධՁࢦඪͷ28,ᮢͷӨڹΛड͚͘͢ɺ༧ଌͷͷӨڹେ͖͍
1PTU1SPDFTTJOH• 5SBJOͱ5FTUʹॏෳ͢Δը૾͕ଘࡏ• TU TUBHFͰɺຕҎ্• OE TUBHFͰɺຕҎ্• 3FTDVFS*%ผ• ࠨͷྫͰɺߘʹͷ։͖TrainTest
1PTU1SPDFTTJOH• JNBHFIBTIͱ͍͏ϥΠϒϥϦΛ͍ɺॏෳը૾Λ୳ࡧ• ,FSOFMͷ࣮ߦ࣌ؒͱͷ݉Ͷ߹͍ͰɺαϜωΛؚΊຕ·Ͱͷը૾Λ༻• ॏෳ͕͋Δ߹ɺ5FTUσʔλͷ༧ଌΛରԠ͢Δ5SBJOσʔλͷతมͷʹஔ͖͑ΔQVCMJD QSJWBUF
V
ͬͨ͜ͱ• టष͍͜ͱॾʑ୲• աڈίϯϖؚΊͯEJTDVTTJPOΛ͏• FYUFSOBM EBUBͷ࡞• σʔλΫϨϯδϯά• ݱঢ়ཧɺλεΫཧ• PETBJΛݟΔ• UXJUUFS• ೣΧϑΣʹߦ͘IUUQXXXOFDPSFQVCMJDKQBDDFTTIUNMUPLZP
σʔλΫϨϯδϯά • UZQPࣙॻΛਓྗͰ࡞ΔͨΊͷ5JQT IUUQTVQVSBIBUFOBCMPHDPNFOUSZ• σʔληοτͷ௲Γϛεඞ͖͔ͣ͢ʁ IUUQTVQVSBIBUFOBCMPHDPNFOUSZ
ೖޙͷల։
ೖޙͷల։ ࠶ݱίʔυɾղ๏ࢿྉͷఏग़ 8JOOFS`T$BMM 1FUGJOEFSͰͷ׆༻
ऴྃޙʹ࿈བྷ͕དྷΔ
࠶ݱίʔυɾղ๏ࢿྉͷఏग़• ࠶ݱίʔυ• ,FSOFM0OMZίϯϖͳͷͰɺఏग़ແ͠Ͱ0,ͩͬͨ• ʢී௨ͷίϯϖͰΔ࡞ۀΛίϯϖதʹڧ͍ΒΕ͍ͯΔʁʣ• ղ๏ࢿྉ• 1PXFSQPJOUͰϖʔδऑ• #BDLHSPVOE4VNNBSZ'FBUVSFTFMFDUJPOFOHJOFFSJOH5SBJOJOHNFUIPET*NQPSUBOUGJOEJOHT4JNQMFNPEFM
8JOOFS`TDBMM • ϓϨθϯ• ࣭ٙ
1FUGJOEFSͰͷ׆༻• ೖऀͷϞσϧΛ౷߹͠ lVMUJNBUFNPEFMz Λ࣮த• ϖοτͷૣظҾ͖औΓʹܨ͛Δత ϖοτͷϓϩϑΟʔϧΛείΞ͚ ԿΛमਖ਼͖͔͢ఏҊʢը૾ɾςΩετͳͲʣ• 1FUGJOEFSͷػցֶशϓϩδΣΫτϖʔδʹࣸਅ͕ࡌͬͨ
Thank you for all!
upura
radkmb
coconala_engineer
keiko713
oracle4engineer
hokaccha
murachiakira
tomoaki25
peisuke
utam0k
andpad
cyberagentdevelopers
yuhattor
cromwellryan
palkan
andyhume
vanstee
brettharned
skipperchong
bcantrill
tanoku
stephaniewalter
bkeepers
kevinliebholz
zenorocha