Kaggle PetFinder 2nd Place Solution

B1cc148711c6a37a5c922b6e72a4ad52?s=47 u++
July 12, 2019

Kaggle PetFinder 2nd Place Solution

B1cc148711c6a37a5c922b6e72a4ad52?s=128

u++

July 12, 2019
Tweet

Transcript

  1. Adoption Prediction 2nd Place Solution Kaggle Tokyo Meetup #6 July

    13th, 2019 [kaggler-ja] Wodori
  2. ຊൃදʹ͍ͭͯ • ,BHHMF%JTDVTTJPO IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPO QSFEJDUJPOEJTDVTTJPO • ϙΤϜύʔτ IUUQTLBFSV OBOUPLBIBUFOBCMPHDPNFOUSZ 

  3. ໨࣍  ίϯϖ֓ཁͱ݁Ռ  νʔϜղ๏ͷ঺հ  ೖ৆ޙͷల։ 

  4. ίϯϖ֓ཁͱ݁Ռ

  5. ίϯϖ֓ཁ • ೥݄຤ʙ೥݄຤ʹΘͨͬͯ։࠵ • ʮϚϨʔγΞͷϖοτγϣοϓͰͷݘɾೣ͕Ҿ͖औΒΕΔ ଎͞Λ༧ଌ͢Δʯͱ͍͏͓୊ IUUQTXXXLBHHMFDPNDQFUGJOEFSBEPQUJPOQSFEJDUJPO 

  6. ར༻Ͱ͖Δσʔλ • ϖοτͷը૾ܗࣜσʔλ • આ໌จ΍ϖοτͷ໊લͳͲͷςΩετܗࣜσʔλ • ਎௕ɾମॏɾଐੑͳͲͷςʔϒϧܗࣜσʔλ • ඞཁʹԠͯ͡֎෦σʔλ΋ར༻Մೳ 

  7. UBSHFUͷৄࡉʢ෼ྨλεΫʣ • ݘɾೣ͕Ҿ͖औΒΕΔ଎͞ •  ϦετΞοϓͷ౰೔ʢ೔໨ʣ •  ϦετΞοϓͷʙ೔໨ •

     ϦετΞοϓͷʙ೔໨ •  ϦετΞοϓͷʙ೔໨ •  Ҿ͖औΒΕͳ͍ 
  8. ධՁࢦඪ • RVBESBUJDXFJHIUFELBQQBʢॏΈ෇͖Χού܎਺ʣ • ͍ۙ෼ྨͰ֎ͨ࣌͠ͷଛࣦ͸খ͍͕͞ɺԕ͍෼ྨͰ֎͢ͱ ଛࣦ͕େ͖͍  IUUQTCFMMDVSWFKQTUBUJTUJDT CMPHIUNM

  9. TUBHF੍ ,FSOFM0OMZ • TUBHF੍ • UFTUEBUB͕׬શʹ৽͍͠σʔλʹࠩ͠ସΘΔ • ,FSOFM0OMZ • Ұ࿈ͷॲཧΛ,BHHMF಺ͷϓϩάϥϜ࣮ߦ؀ڥʹهड़

    • (16࣮ߦ࣌ؒ͸I 
  10. ࢀՃऀશମͷେ·͔ͳํ਑ • ෼ྨ໰୊͕ͩɺճؼͰղ͍ͯɺ$7࠷దͳᮢ஋ΛܾΊΔ • QVCMJDLFSOFMͰ͸(#%5͕ओྲྀ • ը૾΍ςΩετ͔Βಛ௃ྔநग़͠ɺςʔϒϧͱ݁߹ • Ұ෦//ܥ΋͕͋ͬͨɺείΞ͸௿͔ͬͨ •

    ࠩผԽཁҼͷҰͭͩͬͨ 
  11. ݁ՌʢQVCMJDҐQSJWBUFҐʣ 

  12. νʔϜղ๏ͷ঺հ

  13. ղ๏ͷ֓ཁ • Ϟσϧͷ3JEHFTUBDLJOH • /FVSBM/FUXPSL • -JHIU(#. • -JHIU(#. •

    9(#PPTU • ͭͷޙॲཧ • 3BOLBWFSBHJOHͰᮢ஋࠷దԽ • ը૾ॏෳͷൃݟ 
  14. νʔϜϝϯόʔ • ೔ຊਓਓνʔϜʢLBHHMFSKBʣ • LBFSVSVSV ZOLUL V UBLVPLP HFHF •

    ೔ຊޠ 4MBDLͰ΍ΓऔΓ • εϓϨουγʔτͰλεΫͳͲΛ؅ཧ • ίʔυ͸,BHHMF ,FSOFMΛ'PSL 
  15. LFSVSVSVιϩࢀՃ • QVCMJDLFSOFMϕʔεʹಔϝμϧݍ಺ ʢऴྃΧ݄൒લʣ • ػցֶशΤϯδχΞ΁ͷస৬ʹ΋੒ޭ • స৬ॕ͍Ͱ V 

    ZOLUL ͱҿΈձ ʢਓͷग़ձ͍͸લճͷNFFUVQʣ  IUUQTUBCFMPHDPNUPLZP""
  16. ਓ͕αϒϚʔδ 

  17. ਓମ੍ • LFSVSVSVϞσϧΛ৳͹͢ʢਓ͕޷͖উखίϝϯτʣ • ZOLUL͸͔ΒCBTFMJOFϞσϧΛߏங • V ͸աڈίϯϖؚΊEJTDVTTJPOͳͲΛ୳ࡧ • ۜϝμϧݍ಺ʹ౸ୡ

    
  18. UBLVPLP • ং൫ʹࢀՃ͍ͯͨ͠ • ਓମ੍ͷ೔ޙ 

  19. ਓମ੍ • LFSVSVSVϞσϧΛ৳͹͢ • ZOLULϞσϧ΋஀ੜ ʢZOZULϞσϧ͔Β೿ੜ͠UBLVPLPϞσϧ͕஀ੜʣ • ۚϝμϧ·Ͱɺ΋͏ͻͱԡ͠  ίʔυϚʔδ

    CZUBLVPLP ߴ଎Խ CZZOLUL
  20. Ґ ҐͰνʔϜϚʔδ  • ڧLFSOFMެ։Ͱएׯ-#͕ ߥΕΔதɺHFHF͞Μ͕ αϒఔ౓ͰҐ·Ͱ౸ୡ • UXJUUFSͰ%.ૹ৴ ʢ໘ࣝ͸ͳ͔ͬͨʣ

  21. ਓମ੍ • LFSVSVSVϞσϧ • UBLVPLPϞσϧ • HFHFϞσϧ • ۚϝμϧݍ಺Ͱऴྃɺ࠷ऴ݁Ռ͸Ґ 

  22. Ҏޙɺ֤ਓʹΑΔղઆ • UZV@GFBUVSFT // -(#. • L@GFBUVSFT -(#. • H@GFBUVSFT9(#

    • 3*%(&TUBDLJOHCZZOLUL • ͦͷଞ CZV 
  23. UBLVPLP

  24. ྲྀΕ • ,FSOFMͷίʔυ͕ղಡ೉ • ZOLUL͞Μ͕શ෦ϦϑΝΫλϦϯάɻͦΕΛஔ͖౔࢈ʹதࠃ ʹཱཱྀͬͨ • UBLVPLP͕ϝΠϯͰҾ͖ܧ͗ɺਫ਼౓޲্ 

  25. ಛ௃ ςΩετ • %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOU  BOOPUT@EFTD CSFFE • 1VODUTͷॲཧ

    FYXIBU`T XIBU`T • .JTTQFMMͷॲཧ FYXIBUTXIBUJT • 4UFNNJOHFYHPJOHHP • /VNCFSͷॲཧ FYDBUTDBUT • #08UGJEG#.47%/.' 
  26. ಛ௃ ςΩετ • %FTDSJQUJPOPOMZ • 1VODUTͷॲཧ • .JTTQFMMͷॲཧ • /VNCFSͷॲཧ

    • &NCFEEJOHXJUIHMPWFBOEGBTUUFYU2VPSBSE NFUIPE • "WFSBHJOHFNCFEEJOHWFDUPSQFSSPXT 
  27. ಛ௃ ςΩετ • 2VPSBSE NFUIPE • FNCFEEJOHͰׂ͖ͨ߹ •  •

    ଞͷલॲཧ΋֤ϓϩηεͰ FNCFEEJOHׂ߹Λ֬ೝ͠ɺ ৗʹ্͕Δํ๏Λબ୒ͨ͠ 
  28. ಛ௃ ςΩετ • ਫ਼౓্͕͕Βͳ͔ͬͨ΋ͷ • #FSU &MNP • HMPWFUXJUUFSWFDUPSͳͲଞͷFNCFEEJOHT QZNBHOJUVEF

    • 4DSBUDIͰֶशͨ͠XW GBTUUFYU • .FUBUFYUGFBUVSFTͷFNCFEEJOHT • -FOHUIͳͲͷUFYU౷ܭྔ • தࠃޠͷલॲཧ 
  29. ಛ௃ ը૾ • $//GFBUVSFT • ,FSBT EFOTFOFU *ODFQUJPO3FTOFU7 • ԼهϕϯνϚʔΫΛݩʹɺਫ਼౓ͷߴ͍$//Ϟσϧ͔Βॱʹݕ౼

    • IUUQTHJUIVCDPNDHOPSUIDVUUCFODINBSLJOHLFSBTQZUPSDI 
  30. ಛ௃ ը૾ • ਫ਼౓্͕͕Βͳ͔ͬͨ΋ͷ • ଞͷ$//NPEFM LFSBTQZUPSDI • /*." •

    "WJUPͰ࢖ΘΕ͍ͯͨը૾ͷ౷ܭྔ XIJUFOFTT EVMMOFTT FUDʜ 
  31. ಛ௃ ςʔϒϧ • "HHSFHBUJPO • 4JNQMFNFUIPE • 3BUJPNFUIPE • Ձ֨

    CSFFEࣲݘ ࣲݘͷฏۉֹͷ৔߹ SBUJP • %JGGNFUIPE • Ձ֨ CSFFEࣲݘ ࣲݘͷฏۉֹͷ৔߹ EJGG 
  32. ಛ௃ ςʔϒϧ • ΧςΰϦͷڞى  -%"47% • શΧςΰϦͷ૊Έ߹Θͤʢ࣍·Ͱʣ • JOUFSBDUJPOTGFBUVSFT

    • FY"HF 2VBOUJUZ • FYUFSOBMEBUB • ֆจࣈ • 4UBUF #SFFE #SFFESBOLJOH 
  33. ಛ௃ ͦͷଞ • ,FSOFMͰ͸σʔλʹରͯ͠ɺը૾ຕɺNFUBKTPOGJMF΋ ݸ͔͠࢖༻͍ͯ͠ͳ͔ͬͨ • ࣮ࡍʹ͸ෳ਺͋ΓɺͦΕΒ͸શ෦࢖༻ͯ͠ฏۉΛऔͬͨ • ͋Δσʔλ͸શ෦࢖͏͜ͱ͕ॏཁʂ •

    ಛ௃ੜ੒ʹUZV QBSU͚ͩͰTFD͔͚ͨͷͰɺଞΛ͍ͩ Ϳѹഭͨ͠PS[ 
  34. ಛ௃ ಛ௃બ୒ • શ෦Ͱdͷಛ௃ • -(#.ͷJNQPSUBODFUPQΛબ୒ • BMM  

    ͳͲͰ୳ࡧ͠ɺ$7ͱ-#ΛݟܾͯΊͨ • ࣮͸΋ͬͱগͳ͍ํ͕$7ͱQSJWBUF͸Α͔ͬͨ 
  35. Ϟσϧ • -(#.ͱ// • #FTUTJOHMF͸-(#.ͰUI //ͰUI • ΞϯαϯϒϧͰ࢖༻ͯͨ͠ͷ͸-(#.ͰUI //͸Ґ͘Β͍ ͷ΍ͭ

    ಛʹUI //͸$7ߴ͔͕ͬͨ1VCMJD͕গ͠Լ͕ͬͨ • ֶश࣌ؒ͸//ͰTFD -(#.ͰTFD • -(#.ͷύϥϝʔλ • ,FSOFMͷؙύΫϦ 
  36. Ϟσϧ //  • 0QUJNJ[FS"EBN • -PTT3.4& • 4DIFEVMFS$-3

  37. 7JTVBMJ[F  'FBUVSFJNQPSUBODF 1%1

  38. LBFSVSVSV

  39. ํ਑ • $//GFBUVSFT ࣍ݩ  EPDWFD ࣍ݩ UGJEG ͨ͘͞Μ 

    • ߴ࣍ݩϕΫτϧΛ௿࣍ݩʹ௵ͯ͠ར༻͢Δ͜ͱʹࣥ৺ • NFBO WBSͳͲͷ౷ܭྔ΍ 47%ͳͲΛଟ༻͠ɺTDPSF͕৳ͼͨ΋ͷ ͚ͩར༻ ݁Ռతʹ࣌ؒઅ໿ʹ • "HHSFHBUJPOͱͯ΋ޮ͖ͦ͏ • UFYU౷ܭྔಛ௃ͳͲ΋ "HHSFHBUJPOͨ͠ • $7ͪΐͬͱ্͕ͬͨ 
  40. ಛ௃ ը૾ • $//GFBUVSFT • ,FSBTEFOTFOFU • 47%Ͱ ࣍ݩ ࣍ݩʹѹॖ

    
  41. ಛ௃ ը૾ • "WJUPίϯϖͷΧʔωϧͰݟ͚ͭͨը૾ͷαΠζಛ௃ • TJ[F XJEUI IFJHIU ΛΧʔωϧͰެ։ •

    LBFSVSVSV NPEFMʹ͸ޮ͔ͣ΋ɺ࠷ޙʹՃೖͨ͠ HFHFࢯ ͕ϕʔεʹ͍ͯͨ͠ΧʔωϧͰ࢖ΘΕ͍ͯͨ 
  42. ಛ௃ ςΩετ • γϯϓϧͳલॲཧ • TUPQ@XPSET  จࣈ ͷআ֎ •

    খจࣈԽɺվߦ λϒ ਺ࣈͷ࡟আ • ه߸ͷஔ׵ 
  43. ಛ௃ ςΩετ • %FTDSJQUJPONFUBUFYUGFBUVSFT TFOUJNFOU BOOPUT@EFTD • -FOHUIͳͲͷ UFYU౷ܭྔ •

    -FOHUIͳͲͷ UFYU౷ܭྔΛ 3FTDVFS*% Ͱ HSPVQCZ • UGJEG47%Ͱѹॖ 
  44. ಛ௃ ςΩετ • %FTDSJQUJPOPOMZ • %PDWFD࣍ݩͷ෼ࢄදݱΛ NFBO WBSͳͲͰڧҾ ʹ࣍ݩΛམͱͯ͠ར༻ 

  45. ಛ௃ ςʔϒϧ • "HHSFHBUJPO • 3FTDVFS*% Λ LFZʹ • ໟͷ௕͞ɺେ͖͞ɺख਺ྉɺ݈߁ঢ়ଶͳͲઆ໌ޮ͖ͦ͏ͳ΋ͷ

    ʹద༻ ৭ɺੑผͳͲ͸ º • μϝԡ͠Ͱઆ໌จͷ-FOHUIͳͲʹ΋ద༻ 
  46. 8IBUEJEO`UXPSL • ը૾ • 1Z5PSDI൛ EFOTFOFU • EBUB@MPBEFSͰͷฒྻॲཧ OVN@XPSLFS૿ͰΤϥʔ 

    • ςΩετ • 4$%7 87 TDSBUDI 
  47. 8IBUEJEO`UXPSL • &YUFSOBMEBUB • DVUFͰݕࡧͯ͠ग़͖ͯͨݘೣը૾σʔληοτͰֶशͨ͠ NPEFMͰ QSFEJDUͯ͠ <bDVUF`>ಛ௃ΛಘΔ • EBUBTFUͷDMFBOJOH͕଍Γͳ͔ͬͨ

    
  48. 'FBUVSFJNQPSUBODF 

  49. ͓·͚ • 4FFEΨνϟͰ΋ۚݍʢ΄΅ೖ৆ݍʣ  trial seed kaeru_seed Private Ranking 6

    31416 58 0.44945 3rd 7 310416 1033 0.44659 6th 8 310416 1324 0.44955 3rd 9 310416 1604 0.45096 2nd 10 310416 1757 0.45491 2nd trial seed kaeru_seed Private Ranking 1 77777 133777 0.45358 2nd 2 31414 1625 0.45072 2nd 3 31415 1433 0.44914 3rd 4 31415 1631 0.45271 2nd 5 31415 1850 0.44837 5th
  50. HFHF

  51. શମ૾ • UFTUͬΆ͘ͳ͍USBJOσʔλΛ࡟ͬͨ   *NBHF 5BCMF 5FYU %FOTFOFU *NBHFTJ[F

    OVN 47% &YUFSOBM "SFB 1PQVMBUJPO #SFFEEFUBJMJOGP .FUBEBUB 4FOUJNFOU 'SFRVFODZFODPEJOH 5'*%' 47% MJHIUHCN YHCPPTU USBJO EBUB ʢUFTUʹࣅ͍ͯͳ͍σʔλͷΛ࡟আʣ UFTU USBJOUFTUΛ ෼ྨ͢ΔΑ͏ʹֶश
  52. ޮ͍ͨͷʁ • 1VCMJDͰ͸େ͖͘ޮ͍͕ͨɺ1SJWBUFͰ͸ޮ͔ͳ͔ͬͨ • -BUFTVCNJTTJPOͯ͠Έ͕ͨɺ݁Ռ͸΄΅มΘΒͣʢҙຯͳ͠ʣ • ΍ͨΒͱอޢ਺͕ଟ͍3FTDVFS*%Λ࡟͍ͬͯΔΑ͏ʹݟ͑ͨ • ݁ہɺγϯάϧͩͱಔϝμϧݍ 

  53. ্ख͍͔͘ͳ͔ͬͨू • جຊ͏·͍͔͘ͳ͔ͬͨ • &YUFSOBMEBUBΛ໊دͤͯ͠΋Βͬͨ΍ͭ • 4FFEBWFSBHJOHʢ28,͕ෆ҆ఆͩͬͨͷͰ΍ͬͯΈ͕ͨʣ • ݘೣͷ৭Λॱংई౓ʹ͢Δʢྫɿനˠփ৭ˠࠇʣ •

    قઅੑʢςΩετ͔Β+BOVBSZͱ͔ൈ͖औΔʣ • ͳͲ 
  54. ZOLUL

  55. 3JEHF4UBDLJOH • 4JOHMFNPEFM͸͍͍ͤͥNPEFM • $7είΞΛݟͯɺ4UBDLFSΛݕ౼ • -(#."WFSBHF3JEHF • #FTU4JOHMF -(#.

    $7 1SJWBUF UI • #FTU4UBDLJOH$7 1SJWBUF OE 
  56. 3BOLBWFSBHJOH • 3JEHF΁ͷೖྗͷલॲཧͱͯ͠ɺਖ਼نԽΛߦͬͨ • TDJQZTUBUTSBOLEBUBΛ࠾༻ • SBOLԽ͢Δ͜ͱͰɺGPME͝ͱͷ༧ଌ஋ͷ෼෍ͷࠩΛٵऩ • ධՁࢦඪͷ28,͸ᮢ஋ͷӨڹΛड͚΍͘͢ɺ༧ଌ஋ͷ෼෍ ͷӨڹ͸େ͖͍

    
  57. 1PTU1SPDFTTJOH • 5SBJOͱ5FTUʹॏෳ͢Δը૾͕ଘࡏ • TU TUBHFͰ͸ɺຕҎ্ • OE TUBHFͰ͸ɺຕҎ্ •

    3FTDVFS*%͸ผ • ࠨͷྫͰ͸ɺ౤ߘ೔ʹ೔ͷ։͖  Train Test
  58. 1PTU1SPDFTTJOH • JNBHFIBTIͱ͍͏ϥΠϒϥϦΛ࢖͍ɺॏෳը૾Λ୳ࡧ • ,FSOFMͷ࣮ߦ࣌ؒͱͷ݉Ͷ߹͍ͰɺαϜωΛؚΊຕ໨·Ͱͷը૾ Λ࢖༻ • ॏෳ͕͋Δ৔߹ɺ5FTUσʔλͷ༧ଌ஋ΛରԠ͢Δ5SBJOσʔλͷ໨ తม਺ͷ஋ʹஔ͖׵͑Δ QVCMJD

     QSJWBUF 
  59. V

  60. ΍ͬͨ͜ͱ • టष͍͜ͱॾʑ୲౰ • աڈίϯϖؚΊͯEJTDVTTJPOΛ௥͏ • FYUFSOBM EBUBͷ࡞੒ • σʔλΫϨϯδϯά

    • ݱঢ়੔ཧɺλεΫ؅ཧ • PETBJΛݟΔ • UXJUUFS • ೣΧϑΣʹߦ͘  IUUQXXXOFDPSFQVCMJDKQBDDFTTIUNMUPLZP
  61. σʔλΫϨϯδϯά  • UZQPࣙॻΛਓྗͰ࡞ΔͨΊͷ5JQT IUUQTVQVSBIBUFOBCMPHDPNFOUSZ • σʔληοτͷ௲Γϛε͸ඞͣ௚͢΂͖͔ʁ IUUQTVQVSBIBUFOBCMPHDPNFOUSZ

  62. ೖ৆ޙͷల։

  63. ೖ৆ޙͷల։  ࠶ݱίʔυɾղ๏ࢿྉͷఏग़  8JOOFS`T$BMM  1FUGJOEFSͰͷ׆༻ 

  64. ऴྃ௚ޙʹ࿈བྷ͕དྷΔ 

  65. ࠶ݱίʔυɾղ๏ࢿྉͷఏग़ • ࠶ݱίʔυ • ,FSOFM0OMZίϯϖͳͷͰɺఏग़ແ͠Ͱ0,ͩͬͨ • ʢී௨ͷίϯϖͰ΍Δ࡞ۀΛίϯϖதʹڧ͍ΒΕ͍ͯΔʁʣ • ղ๏ࢿྉ •

    1PXFSQPJOUͰϖʔδऑ • #BDLHSPVOE4VNNBSZ'FBUVSFTFMFDUJPOFOHJOFFSJOH 5SBJOJOHNFUIPET*NQPSUBOUGJOEJOHT4JNQMFNPEFM 
  66. 8JOOFS`TDBMM  • ෼ϓϨθϯ • ෼࣭ٙ

  67. 1FUGJOEFSͰͷ׆༻ • ೖ৆ऀͷϞσϧΛ౷߹͠ lVMUJNBUFNPEFMz Λ࣮૷த • ϖοτͷૣظҾ͖औΓʹܨ͛Δ໨త  ϖοτͷϓϩϑΟʔϧΛείΞ෇͚ 

    ԿΛमਖ਼͢΂͖͔ఏҊʢը૾ɾςΩετͳͲʣ • 1FUGJOEFSͷػցֶशϓϩδΣΫτϖʔδʹࣸਅ͕ࡌͬͨ 
  68. Thank you for all!