Kaggle PetFinder 2nd Place Solution

Slide 1

Slide 1 text

Adoption Prediction 2nd Place Solution Kaggle Tokyo Meetup #6 July 13th, 2019 [kaggler-ja] Wodori

Slide 2

Slide 2 text

ຊൃදʹ͍ͭͯ • ,BHHMF%JTDVTTJPO IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPO QSFEJDUJPOEJTDVTTJPO • ϙΤϜύʔτ IUUQTLBFSV OBOUPLBIBUFOBCMPHDPNFOUSZ

Slide 3

Slide 3 text

໨࣍ ίϯϖ֓ཁͱ݁Ռ νʔϜղ๏ͷ঺հ ೖ৆ޙͷల։

Slide 4

Slide 4 text

ίϯϖ֓ཁͱ݁Ռ

Slide 5

Slide 5 text

ίϯϖ֓ཁ • ೥݄຤ʙ೥݄຤ʹΘͨͬͯ։࠵ • ʮϚϨʔγΞͷϖοτγϣοϓͰͷݘɾೣ͕Ҿ͖औΒΕΔ ଎͞Λ༧ଌ͢Δʯͱ͍͏͓୊ IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPOQSFEJDUJPO

Slide 6

Slide 6 text

ར༻Ͱ͖Δσʔλ • ϖοτͷը૾ܗࣜσʔλ • આ໌จ΍ϖοτͷ໊લͳͲͷςΩετܗࣜσʔλ • ਎௕ɾମॏɾଐੑͳͲͷςʔϒϧܗࣜσʔλ • ඞཁʹԠͯ͡֎෦σʔλ΋ར༻Մೳ

Slide 7

Slide 7 text

UBSHFUͷৄࡉʢ෼ྨλεΫʣ • ݘɾೣ͕Ҿ͖औΒΕΔ଎͞ • ϦετΞοϓͷ౰೔ʢ೔໨ʣ • ϦετΞοϓͷʙ೔໨ • ϦετΞοϓͷʙ೔໨ • ϦετΞοϓͷʙ೔໨ • Ҿ͖औΒΕͳ͍

Slide 8

Slide 8 text

ධՁࢦඪ • RVBESBUJDXFJHIUFELBQQBʢॏΈ෇͖Χού܎਺ʣ • ͍ۙ෼ྨͰ֎ͨ࣌͠ͷଛࣦ͸খ͍͕͞ɺԕ͍෼ྨͰ֎͢ͱ ଛࣦ͕େ͖͍ IUUQTCFMMDVSWFKQTUBUJTUJDT CMPHIUNM

Slide 9

Slide 9 text

TUBHF੍ ,FSOFM0OMZ • TUBHF੍ • UFTUEBUB͕׬શʹ৽͍͠σʔλʹࠩ͠ସΘΔ • ,FSOFM0OMZ • Ұ࿈ͷॲཧΛ,BHHMF಺ͷϓϩάϥϜ࣮ߦ؀ڥʹهड़ • (16࣮ߦ࣌ؒ͸I

Slide 10

Slide 10 text

ࢀՃऀશମͷେ·͔ͳํ਑ • ෼ྨ໰୊͕ͩɺճؼͰղ͍ͯɺ$7࠷దͳᮢ஋ΛܾΊΔ • QVCMJDLFSOFMͰ͸(#%5͕ओྲྀ • ը૾΍ςΩετ͔Βಛ௃ྔநग़͠ɺςʔϒϧͱ݁߹ • Ұ෦//ܥ΋͕͋ͬͨɺείΞ͸௿͔ͬͨ • ࠩผԽཁҼͷҰͭͩͬͨ

Slide 11

Slide 11 text

݁ՌʢQVCMJDҐQSJWBUFҐʣ

Slide 12

Slide 12 text

νʔϜղ๏ͷ঺հ

Slide 13

Slide 13 text

ղ๏ͷ֓ཁ • Ϟσϧͷ3JEHFTUBDLJOH • /FVSBM/FUXPSL • -JHIU(#. • -JHIU(#. • 9(#PPTU • ͭͷޙॲཧ • 3BOLBWFSBHJOHͰᮢ஋࠷దԽ • ը૾ॏෳͷൃݟ

Slide 14

Slide 14 text

νʔϜϝϯόʔ • ೔ຊਓਓνʔϜʢLBHHMFSKBʣ • LBFSVSVSV ZOLUL V UBLVPLP HFHF • ೔ຊޠ 4MBDLͰ΍ΓऔΓ • εϓϨουγʔτͰλεΫͳͲΛ؅ཧ • ίʔυ͸,BHHMF ,FSOFMΛ'PSL

Slide 15

Slide 15 text

LFSVSVSVιϩࢀՃ • QVCMJDLFSOFMϕʔεʹಔϝμϧݍ಺ ʢऴྃΧ݄൒લʣ • ػցֶशΤϯδχΞ΁ͷస৬ʹ΋੒ޭ • స৬ॕ͍Ͱ V ZOLUL ͱҿΈձ ʢਓͷग़ձ͍͸લճͷNFFUVQʣ IUUQTUBCFMPHDPNUPLZP""

Slide 16

Slide 16 text

ਓ͕αϒϚʔδ

Slide 17

Slide 17 text

ਓମ੍ • LFSVSVSVϞσϧΛ৳͹͢ʢਓ͕޷͖উखίϝϯτʣ • ZOLUL͸͔ΒCBTFMJOFϞσϧΛߏங • V͸աڈίϯϖؚΊEJTDVTTJPOͳͲΛ୳ࡧ • ۜϝμϧݍ಺ʹ౸ୡ

Slide 18

Slide 18 text

UBLVPLP • ং൫ʹࢀՃ͍ͯͨ͠ • ਓମ੍ͷ೔ޙ

Slide 19

Slide 19 text

ਓମ੍ • LFSVSVSVϞσϧΛ৳͹͢ • ZOLULϞσϧ΋஀ੜ ʢZOZULϞσϧ͔Β೿ੜ͠UBLVPLPϞσϧ͕஀ੜʣ • ۚϝμϧ·Ͱɺ΋͏ͻͱԡ͠ ίʔυϚʔδ CZUBLVPLP ߴ଎Խ CZZOLUL

Slide 20

Slide 20 text

ҐҐͰνʔϜϚʔδ • ڧLFSOFMެ։Ͱएׯ-#͕ ߥΕΔதɺHFHF͞Μ͕ αϒఔ౓ͰҐ·Ͱ౸ୡ • UXJUUFSͰ%.ૹ৴ ʢ໘ࣝ͸ͳ͔ͬͨʣ

Slide 21

Slide 21 text

ਓମ੍ • LFSVSVSVϞσϧ • UBLVPLPϞσϧ • HFHFϞσϧ • ۚϝμϧݍ಺Ͱऴྃɺ࠷ऴ݁Ռ͸Ґ

Slide 22

Slide 22 text

Ҏޙɺ֤ਓʹΑΔղઆ • UZV@GFBUVSFT // -(#. • L@GFBUVSFT -(#. • H@GFBUVSFT9(# • 3*%(&TUBDLJOHCZZOLUL • ͦͷଞ CZV

Slide 23

Slide 23 text

UBLVPLP

Slide 24

Slide 24 text

ྲྀΕ • ,FSOFMͷίʔυ͕ղಡ೉ • ZOLUL͞Μ͕શ෦ϦϑΝΫλϦϯάɻͦΕΛஔ͖౔࢈ʹதࠃ ʹཱཱྀͬͨ • UBLVPLP͕ϝΠϯͰҾ͖ܧ͗ɺਫ਼౓޲্

Slide 25

Slide 25 text

ಛ௃ ςΩετ • %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOU BOOPUT@EFTD CSFFE • 1VODUTͷॲཧ FYXIBU`T XIBU`T • .JTTQFMMͷॲཧ FYXIBUTXIBUJT • 4UFNNJOHFYHPJOHHP • /VNCFSͷॲཧ FYDBUTDBUT • #08UGJEG#.47%/.'

Slide 26

Slide 26 text

ಛ௃ ςΩετ • %FTDSJQUJPOPOMZ • 1VODUTͷॲཧ • .JTTQFMMͷॲཧ • /VNCFSͷॲཧ • &NCFEEJOHXJUIHMPWFBOEGBTUUFYU2VPSBSE NFUIPE • "WFSBHJOHFNCFEEJOHWFDUPSQFSSPXT

Slide 27

Slide 27 text

ಛ௃ ςΩετ • 2VPSBSE NFUIPE • FNCFEEJOHͰׂ͖ͨ߹ • • ଞͷલॲཧ΋֤ϓϩηεͰ FNCFEEJOHׂ߹Λ֬ೝ͠ɺ ৗʹ্͕Δํ๏Λબ୒ͨ͠

Slide 28

Slide 28 text

ಛ௃ ςΩετ • ਫ਼౓্͕͕Βͳ͔ͬͨ΋ͷ • #FSU &MNP • HMPWFUXJUUFSWFDUPSͳͲଞͷFNCFEEJOHT QZNBHOJUVEF • 4DSBUDIͰֶशͨ͠XW GBTUUFYU • .FUBUFYUGFBUVSFTͷFNCFEEJOHT • -FOHUIͳͲͷUFYU౷ܭྔ • தࠃޠͷલॲཧ

Slide 29

Slide 29 text

ಛ௃ ը૾ • $//GFBUVSFT • ,FSBT EFOTFOFU *ODFQUJPO3FTOFU7 • ԼهϕϯνϚʔΫΛݩʹɺਫ਼౓ͷߴ͍$//Ϟσϧ͔Βॱʹݕ౼ • IUUQTHJUIVCDPNDHOPSUIDVUUCFODINBSLJOHLFSBTQZUPSDI

Slide 30

Slide 30 text

ಛ௃ ը૾ • ਫ਼౓্͕͕Βͳ͔ͬͨ΋ͷ • ଞͷ$//NPEFM LFSBTQZUPSDI • /*." • "WJUPͰ࢖ΘΕ͍ͯͨը૾ͷ౷ܭྔ XIJUFOFTT EVMMOFTT FUDʜ

Slide 31

Slide 31 text

ಛ௃ ςʔϒϧ • "HHSFHBUJPO • 4JNQMFNFUIPE • 3BUJPNFUIPE • Ձ֨ CSFFEࣲݘ ࣲݘͷฏۉֹͷ৔߹ SBUJP • %JGGNFUIPE • Ձ֨ CSFFEࣲݘ ࣲݘͷฏۉֹͷ৔߹ EJGG

Slide 32

Slide 32 text

ಛ௃ ςʔϒϧ • ΧςΰϦͷڞى -%"47% • શΧςΰϦͷ૊Έ߹Θͤʢ࣍·Ͱʣ • JOUFSBDUJPOTGFBUVSFT • FY"HF2VBOUJUZ • FYUFSOBMEBUB • ֆจࣈ • 4UBUF #SFFE #SFFESBOLJOH

Slide 33

Slide 33 text

ಛ௃ ͦͷଞ • ,FSOFMͰ͸σʔλʹରͯ͠ɺը૾ຕɺNFUBKTPOGJMF΋ ݸ͔͠࢖༻͍ͯ͠ͳ͔ͬͨ • ࣮ࡍʹ͸ෳ਺͋ΓɺͦΕΒ͸શ෦࢖༻ͯ͠ฏۉΛऔͬͨ • ͋Δσʔλ͸શ෦࢖͏͜ͱ͕ॏཁʂ • ಛ௃ੜ੒ʹUZV QBSU͚ͩͰTFD͔͚ͨͷͰɺଞΛ͍ͩ Ϳѹഭͨ͠PS[

Slide 34

Slide 34 text

ಛ௃ ಛ௃બ୒ • શ෦Ͱdͷಛ௃ • -(#.ͷJNQPSUBODFUPQΛબ୒ • BMM ͳͲͰ୳ࡧ͠ɺ$7ͱ-#ΛݟܾͯΊͨ • ࣮͸΋ͬͱগͳ͍ํ͕$7ͱQSJWBUF͸Α͔ͬͨ

Slide 35

Slide 35 text

Ϟσϧ • -(#.ͱ// • #FTUTJOHMF͸-(#.ͰUI //ͰUI • ΞϯαϯϒϧͰ࢖༻ͯͨ͠ͷ͸-(#.ͰUI //͸Ґ͘Β͍ ͷ΍ͭ ಛʹUI //͸$7ߴ͔͕ͬͨ1VCMJD͕গ͠Լ͕ͬͨ • ֶश࣌ؒ͸//ͰTFD -(#.ͰTFD • -(#.ͷύϥϝʔλ • ,FSOFMͷؙύΫϦ

Slide 36

Slide 36 text

Ϟσϧ // • 0QUJNJ[FS"EBN • -PTT3.4& • 4DIFEVMFS$-3

Slide 37

Slide 37 text

7JTVBMJ[F 'FBUVSFJNQPSUBODF 1%1

Slide 38

Slide 38 text

LBFSVSVSV

Slide 39

Slide 39 text

ํ਑ • $//GFBUVSFT ࣍ݩ EPDWFD ࣍ݩ UGJEG ͨ͘͞Μ • ߴ࣍ݩϕΫτϧΛ௿࣍ݩʹ௵ͯ͠ར༻͢Δ͜ͱʹࣥ৺ • NFBO WBSͳͲͷ౷ܭྔ΍ 47%ͳͲΛଟ༻͠ɺTDPSF͕৳ͼͨ΋ͷ ͚ͩར༻ ݁Ռతʹ࣌ؒઅ໿ʹ • "HHSFHBUJPOͱͯ΋ޮ͖ͦ͏ • UFYU౷ܭྔಛ௃ͳͲ΋ "HHSFHBUJPOͨ͠ • $7ͪΐͬͱ্͕ͬͨ

Slide 40

Slide 40 text

ಛ௃ ը૾ • $//GFBUVSFT • ,FSBTEFOTFOFU • 47%Ͱ ࣍ݩ ࣍ݩʹѹॖ

Slide 41

Slide 41 text

ಛ௃ ը૾ • "WJUPίϯϖͷΧʔωϧͰݟ͚ͭͨը૾ͷαΠζಛ௃ • TJ[F XJEUI IFJHIU ΛΧʔωϧͰެ։ • LBFSVSVSV NPEFMʹ͸ޮ͔ͣ΋ɺ࠷ޙʹՃೖͨ͠ HFHFࢯ ͕ϕʔεʹ͍ͯͨ͠ΧʔωϧͰ࢖ΘΕ͍ͯͨ

Slide 42

Slide 42 text

ಛ௃ ςΩετ • γϯϓϧͳલॲཧ • TUPQ@XPSET จࣈ ͷআ֎ • খจࣈԽɺվߦ λϒ ਺ࣈͷ࡟আ • ه߸ͷஔ׵

Slide 43

Slide 43 text

ಛ௃ ςΩετ • %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOU BOOPUT@EFTD • -FOHUIͳͲͷ UFYU౷ܭྔ • -FOHUIͳͲͷ UFYU౷ܭྔΛ 3FTDVFS*% Ͱ HSPVQCZ • UGJEG47%Ͱѹॖ

Slide 44

Slide 44 text

ಛ௃ ςΩετ • %FTDSJQUJPOPOMZ • %PDWFD࣍ݩͷ෼ࢄදݱΛ NFBO WBSͳͲͰڧҾ ʹ࣍ݩΛམͱͯ͠ར༻

Slide 45

Slide 45 text

ಛ௃ ςʔϒϧ • "HHSFHBUJPO • 3FTDVFS*% Λ LFZʹ • ໟͷ௕͞ɺେ͖͞ɺख਺ྉɺ݈߁ঢ়ଶͳͲઆ໌ޮ͖ͦ͏ͳ΋ͷ ʹద༻ ৭ɺੑผͳͲ͸ º • μϝԡ͠Ͱઆ໌จͷ-FOHUIͳͲʹ΋ద༻

Slide 46

Slide 46 text

8IBUEJEO`UXPSL • ը૾ • 1Z5PSDI൛ EFOTFOFU • EBUB@MPBEFSͰͷฒྻॲཧ OVN@XPSLFS૿ͰΤϥʔ • ςΩετ • 4$%7 87 TDSBUDI

Slide 47

Slide 47 text

8IBUEJEO`UXPSL • &YUFSOBMEBUB • DVUFͰݕࡧͯ͠ग़͖ͯͨݘೣը૾σʔληοτͰֶशͨ͠ NPEFMͰ QSFEJDUͯ͠ ಛ௃ΛಘΔ • EBUBTFUͷDMFBOJOH͕଍Γͳ͔ͬͨ

Slide 48

Slide 48 text

'FBUVSFJNQPSUBODF

Slide 49

Slide 49 text

͓·͚ • 4FFEΨνϟͰ΋ۚݍʢ΄΅ೖ৆ݍʣ trial seed kaeru_seed Private Ranking 6 31416 58 0.44945 3rd 7 310416 1033 0.44659 6th 8 310416 1324 0.44955 3rd 9 310416 1604 0.45096 2nd 10 310416 1757 0.45491 2nd trial seed kaeru_seed Private Ranking 1 77777 133777 0.45358 2nd 2 31414 1625 0.45072 2nd 3 31415 1433 0.44914 3rd 4 31415 1631 0.45271 2nd 5 31415 1850 0.44837 5th

Slide 50

Slide 50 text

HFHF

Slide 51

Slide 51 text

શମ૾ • UFTUͬΆ͘ͳ͍USBJOσʔλΛ࡟ͬͨ *NBHF 5BCMF 5FYU %FOTFOFU *NBHFTJ[F OVN 47% &YUFSOBM "SFB 1PQVMBUJPO #SFFEEFUBJMJOGP .FUBEBUB 4FOUJNFOU 'SFRVFODZFODPEJOH 5'*%' 47% MJHIUHCN YHCPPTU USBJO EBUB ʢUFTUʹࣅ͍ͯͳ͍σʔλͷΛ࡟আʣ UFTU USBJOUFTUΛ ෼ྨ͢ΔΑ͏ʹֶश

Slide 52

Slide 52 text

ޮ͍ͨͷʁ • 1VCMJDͰ͸େ͖͘ޮ͍͕ͨɺ1SJWBUFͰ͸ޮ͔ͳ͔ͬͨ • -BUFTVCNJTTJPOͯ͠Έ͕ͨɺ݁Ռ͸΄΅มΘΒͣʢҙຯͳ͠ʣ • ΍ͨΒͱอޢ਺͕ଟ͍3FTDVFS*%Λ࡟͍ͬͯΔΑ͏ʹݟ͑ͨ • ݁ہɺγϯάϧͩͱಔϝμϧݍ

Slide 53

Slide 53 text

্ख͍͔͘ͳ͔ͬͨू • جຊ͏·͍͔͘ͳ͔ͬͨ • &YUFSOBMEBUBΛ໊دͤͯ͠΋Βͬͨ΍ͭ • 4FFEBWFSBHJOHʢ28,͕ෆ҆ఆͩͬͨͷͰ΍ͬͯΈ͕ͨʣ • ݘೣͷ৭Λॱংई౓ʹ͢Δʢྫɿനˠփ৭ˠࠇʣ • قઅੑʢςΩετ͔Β+BOVBSZͱ͔ൈ͖औΔʣ • ͳͲ

Slide 54

Slide 54 text

ZOLUL

Slide 55

Slide 55 text

3JEHF4UBDLJOH • 4JOHMFNPEFM͸͍͍ͤͥNPEFM • $7είΞΛݟͯɺ4UBDLFSΛݕ౼ • -(#."WFSBHF3JEHF • #FTU4JOHMF -(#. $7 1SJWBUF UI • #FTU4UBDLJOH$7 1SJWBUF OE

Slide 56

Slide 56 text

3BOLBWFSBHJOH • 3JEHF΁ͷೖྗͷલॲཧͱͯ͠ɺਖ਼نԽΛߦͬͨ • TDJQZTUBUTSBOLEBUBΛ࠾༻ • SBOLԽ͢Δ͜ͱͰɺGPME͝ͱͷ༧ଌ஋ͷ෼෍ͷࠩΛٵऩ • ධՁࢦඪͷ28,͸ᮢ஋ͷӨڹΛड͚΍͘͢ɺ༧ଌ஋ͷ෼෍ ͷӨڹ͸େ͖͍

Slide 57

Slide 57 text

1PTU1SPDFTTJOH • 5SBJOͱ5FTUʹॏෳ͢Δը૾͕ଘࡏ • TU TUBHFͰ͸ɺຕҎ্ • OE TUBHFͰ͸ɺຕҎ্ • 3FTDVFS*%͸ผ • ࠨͷྫͰ͸ɺ౤ߘ೔ʹ೔ͷ։͖ Train Test

Slide 58

Slide 58 text

1PTU1SPDFTTJOH • JNBHFIBTIͱ͍͏ϥΠϒϥϦΛ࢖͍ɺॏෳը૾Λ୳ࡧ • ,FSOFMͷ࣮ߦ࣌ؒͱͷ݉Ͷ߹͍ͰɺαϜωΛؚΊຕ໨·Ͱͷը૾ Λ࢖༻ • ॏෳ͕͋Δ৔߹ɺ5FTUσʔλͷ༧ଌ஋ΛରԠ͢Δ5SBJOσʔλͷ໨ తม਺ͷ஋ʹஔ͖׵͑Δ QVCMJD QSJWBUF

Slide 59

Slide 59 text

Slide 60

Slide 60 text

΍ͬͨ͜ͱ • టष͍͜ͱॾʑ୲౰ • աڈίϯϖؚΊͯEJTDVTTJPOΛ௥͏ • FYUFSOBM EBUBͷ࡞੒ • σʔλΫϨϯδϯά • ݱঢ়੔ཧɺλεΫ؅ཧ • PETBJΛݟΔ • UXJUUFS • ೣΧϑΣʹߦ͘ IUUQXXXOFDPSFQVCMJDKQBDDFTTIUNMUPLZP

Slide 61

Slide 61 text

σʔλΫϨϯδϯά • UZQPࣙॻΛਓྗͰ࡞ΔͨΊͷ5JQT IUUQTVQVSBIBUFOBCMPHDPNFOUSZ • σʔληοτͷ௲Γϛε͸ඞͣ௚͢΂͖͔ʁ IUUQTVQVSBIBUFOBCMPHDPNFOUSZ

Slide 62

Slide 62 text

ೖ৆ޙͷల։

Slide 63

Slide 63 text

ೖ৆ޙͷల։ ࠶ݱίʔυɾղ๏ࢿྉͷఏग़ 8JOOFS`T$BMM 1FUGJOEFSͰͷ׆༻

Slide 64

Slide 64 text

ऴྃ௚ޙʹ࿈བྷ͕དྷΔ

Slide 65

Slide 65 text

࠶ݱίʔυɾղ๏ࢿྉͷఏग़ • ࠶ݱίʔυ • ,FSOFM0OMZίϯϖͳͷͰɺఏग़ແ͠Ͱ0,ͩͬͨ • ʢී௨ͷίϯϖͰ΍Δ࡞ۀΛίϯϖதʹڧ͍ΒΕ͍ͯΔʁʣ • ղ๏ࢿྉ • 1PXFSQPJOUͰϖʔδऑ • #BDLHSPVOE4VNNBSZ'FBUVSFTFMFDUJPOFOHJOFFSJOH 5SBJOJOHNFUIPET*NQPSUBOUGJOEJOHT4JNQMFNPEFM

Slide 66

Slide 66 text

8JOOFS`TDBMM • ෼ϓϨθϯ • ෼࣭ٙ

Slide 67

Slide 67 text

1FUGJOEFSͰͷ׆༻ • ೖ৆ऀͷϞσϧΛ౷߹͠ lVMUJNBUFNPEFMz Λ࣮૷த • ϖοτͷૣظҾ͖औΓʹܨ͛Δ໨త ϖοτͷϓϩϑΟʔϧΛείΞ෇͚ ԿΛमਖ਼͢΂͖͔ఏҊʢը૾ɾςΩετͳͲʣ • 1FUGJOEFSͷػցֶशϓϩδΣΫτϖʔδʹࣸਅ͕ࡌͬͨ

Slide 68

Slide 68 text

Thank you for all!