コンペに公開されているデータを使っての機械学習事始め / pyml-niigata-20180818

Transcript

1. ίϯϖͰެ։͞Ε͍ͯ ΔσʔλΛ࢖ͬͯػց ֶशࣄ࢝Ί Pythonػցֶशษڧձ in ৽ׁ Restart 2018/08/18 @kasacchiful

2. ࣗݾ঺հ • ּݪ ޺ (@kasacchiful) • ৽ׁࢢࡏॅ • Ruby͕͓ؾʹೖΓ •

   JaSST৽ׁ࣮ߦҕһ

3. ຊ೔ͷ಺༰ 1. “ίϯϖ”ͬͯԿʁ 2. ίϯϖͰԿ͕਎ʹͭ͘ͷʁ 3. KaggleͱSIGNATEͷ࿅श໰୊ʹఏग़ͯ͠ΈΑ͏ 4. ʢ͕࣌ؒ͋Ε͹ʣखॻ͖จࣈը૾ͷೝࣝ໰୊ʹఏग़͠ ͯΈΑ͏

   5. ·ͱΊ

4. 1. ίϯϖͬͯԿʁ

5. ίϯϖͬͯԿʁ • σʔλ෼ੳʹ͓͚Δίϯϖͱ͸ɺࢀՃऀಉ࢜ ͕ެ։͞Ε͍ͯΔσʔλΛ࢖ͬͯ՝୊Λղ͖ɺ ͦͷ༏ྼΛڝ͏৔ɻ ίϯϖςΟγϣϯ
   $PNQFUJUJPO
   ͱ͸ɺڝ૪ɺڝٕɺڝٕձͷҙɻ
   ڝ͍߹Θͤɺ༏ྼΛ͚ͭΔ͜ͱɻུͯ͠ίϯϖͱ΋͍͏ɻ Ҿ༻:

   ίϯϖςΟγϣϯ - Wikipedia https://ja.wikipedia.org/wiki/ίϯϖςΟγϣϯ

6. ίϯϖͷ࢓૊Έ اۀ ίϯϖ
   ओ࠵ ࢀՃऀ ᶃ ᶃ: ՝୊ɾσʔλɾ৆ۚΛఏڙ

7. ίϯϖͷ࢓૊Έ اۀ ίϯϖ
   ओ࠵ ࢀՃऀ ᶄ: σʔλΛѻ͍΍͍͢Α͏ʹલॲཧ͠ɺ ධՁํ๏Λܾఆͯ͠ίϯϖΛެ։ ᶄ

8. ίϯϖͷ࢓૊Έ اۀ ίϯϖ
   ओ࠵ ࢀՃऀ ᶅ: ఏڙ͞ΕͨσʔλΛ࢖ͬͯɺ ࣗΒͷ஌ݟ΍ख๏Λۦ࢖ͯ͠՝୊Λղ͘ ᶅ ※ίϯϖ։࠵ظؒத͸Կճ΋ఏग़Ͱ͖Δʢ※1೔5ճ·Ͱ౳੍ݶ͸͋Δ͚Ͳʣ

9. ίϯϖͷ࢓૊Έ اۀ ίϯϖ
   ओ࠵ ࢀՃऀ ᶆ: ίϯϖظؒऴྃޙɺ ্Ґೖ৆ऀʹ৆͕ۚत༩͞ΕΔ ※kaggleͰ͸ɺ্Ґೖ৆ऀʹϝμϧ͕त༩͞ΕɺϝμϧʹԠͯ͡kaggleϥϯΫ্͕͕Δ ᶆ

10. ίϯϖͷ࢓૊Έ اۀ ίϯϖ
    ओ࠵ ࢀՃऀ ᶇ: ্Ґͷղ๏ʹ͍ͭͯ͸ɺ ࣮ࡍͷاۀ՝୊ʹରͯ͠ద༻͞Ε ۀ຿վળʹ࢖ΘΕΔ ᶇ

11. ୅දతͳίϯϖ • Kaggle • SIGNATE

12. IUUQTXXXLBHHMFDPN

13. Kaggleͱ͸ • ੈքதͷσʔλαΠΤϯςΟετ͕ू·Γɺ͠ͷ͗Λ࡟ΔϓϥοτϑΥʔϜ • ୅දతͳίϯϖͷ՝୊ • ϩʔϯͷσϑΥϧτϦεΫ൑ఆ • Ϣʔβͷෆਖ਼ΞΫηεݕ஌ •

    ը૾಺ͷΦϒδΣΫτݕग़ • Φεεϝग़඼Ձ֨ͷఏࣔ • etc. • KaggleʹࢀՃ͍ͯ͠Δਓͷ͜ͱΛʮKagglerʯͱݺΜͩΓ͢Δ

14. KaggleΛͷ͍ͧͯΈΑ͏ • Competition • ίϯϖͷҰཡ • Datasets • Kaggle͕ެ։͍ͯ͠ΔσʔληοτͷҰཡ •

    Kernels • kaggler͕֤໰୊ͷղ͖ํΛNotebookܗࣜͰ͍ࣔͯ͠Δ • ·ͣ͸໰୊ͷղ͖ํΛࣸܦͯ͠ɺྲྀΕΛݟΔͷ͕Φεεϝ • Discussion • kagglerಉ͕࢜ٞ࿦͠߹͏ܝࣔ൘ • ࠷৽ΞϧΰϦζϜ౳ͷτϐοΫε͕ग़͖ͯͨΓ͢Δ • learn • Python/R΍ɺPandasɺػցֶशɺσΟʔϓϥʔχϯάɺՄࢹԽͳͲͷجຊΛNotebookܗࣜͰެ։

15. IUUQTJHOBUFKQ

16. SIGNATEͱ͸ • גࣜձࣾSIGNATE͕ӡӦ͢Δɺ೔ຊͷίϯϖςΟγϣϯ • 2018೥4݄ΑΓɺDeepAnalytics͔ΒSIGNATEʹαʔϏε໊มߋ • ਓ޻஌ೳֶձ͕ओ࠵͢ΔίϯϖͷϓϥοτϑΥʔϜ౳ʹ࢖ΘΕͨ Γ͢Δ • ࠓ೥͸ɺΫοΫύουఏڙͷը૾͔Β৯ࡐ55඼໨ͷ෼ྨΛ͢

    Δίϯϖ͕։࠵͞Εͨ • ࿅श໰୊͕ଟ͍ͷͰɺػցֶशͷษڧͱͯ͠΋࢖͍΍͍͢

17. SIGNATEΛͷ͍ͧͯΈΑ͏ • Competitions • ίϯϖͷҰཡ • Learning • ࿅श໰୊ͷίϯϖςΟγϣϯ΍Udemyͷಈըߨ࠲ͷ঺հ •

    Carrer • SIGNATEʹΑΔస৬αϙʔτ

18. 2. ίϯϖͰԿ͕਎ʹͭ͘ͷʁ

19. σʔλαΠΤϯςΟετʹͳΔ ͨΊʹඞཁͳ8ͭͷεΩϧ 1. Programming Skills (Python / R / SQL)

    2. Statistics (౷ܭͷ஌ࣝʣ 3. Machine Learning (ػցֶशͷཧ࿦) 4. Multivariable Calculus & Linear Algebra (ଟม਺ඍ෼ੵ෼ͱઢܗ୅਺) 5. Data Wrangling (σʔλͷૢ࡞(Pandas / dplyr)) 6. Data Visualization & Communication (σʔλͷՄࢹԽ) 7. Software Engineering (σʔλΛूΊͨΓσʔλۦಈܕ੡඼։ൃ͢ΔͨΊͷΤϯδχΞϦϯάྗ) 8. Data Intuition (σʔλʹର͢Δʮ௚ײྗʯ) Ҿ༻: 8 Skills You Need to Be a Data Scientist | Udacity https://blog.udacity.com/2014/11/data-science-job-skills.html

20. ίϯϖͰ਎ʹͭ͘ͱࢥ͏εΩϧ 1. Programming Skills (Python / R / SQL) 2.

    Statistics (౷ܭͷ஌ࣝʣ 3. Machine Learning (ػցֶशͷཧ࿦) 4. Multivariable Calculus & Linear Algebra (ଟม਺ඍ෼ੵ෼ͱઢܗ୅਺) 5. Data Wrangling (σʔλͷૢ࡞(Pandas / dplyr)) 6. Data Visualization & Communication (σʔλͷՄࢹԽ) 7. Software Engineering (σʔλΛूΊͨΓσʔλۦಈܕ੡඼։ൃ͢ΔͨΊͷΤϯδχΞϦϯάྗ) 8. Data Intuition (σʔλʹର͢Δʮ௚ײྗʯ) ˕ ̋: ࠷ॳ͸͍Βͳ͍͚Ͳɺ্Ґૂ͏ͳΒඞཁ ̋: ࠷ॳ͸͍Βͳ͍͚Ͳɺ্Ґૂ͏ͳΒඞཁ ̋: ࠷ॳ͸͍Βͳ͍͚Ͳɺ্Ґૂ͏ͳΒඞཁ ˕ ˚: ίϯϖ͚ͩͰ͸਎ʹ͔ͭͳ͍͔΋ ˚: ίϯϖ͚ͩͰ͸਎ʹ͔ͭͳ͍͔΋ ˕

21. ෳࡶͳཧ࿦͕Θ͔Βͳͯ͘΋ ࢝ΊΔ͜ͱ͕Ͱ͖·͢

22. ࢝ΊͯΈͳ͚Ε͹ Α͘Θ͔Βͳ͍··

23. 3. kaggleͱSIGNATEͷ࿅श ໰୊ʹఏग़ͯ͠ΈΑ͏

24. ࿅श໰୊ʹఏग़ • ͲͪΒͷίϯϖʹ΋͋Δ࿅श໰୊ʮTitanicʯΛ΍ͬͯΈ·͢ • Kaggle: “Titanic: Machine Learning from Disaster”

    • https://www.kaggle.com/c/titanic • SIGNATE: “ʲ࿅श໰୊ʳλΠλχοΫͷੜଘ༧ଌ” • https://signate.jp/competitions/102 λΠλχοΫ߸௜຅ࣄނ
    λΠλχοΫ͝͏ͪΜ΅ͭ͜͡
    ͸ɺ೥݄೔ͷ໷͔Β೔ͷேʹ͔͚ͯɺ ΠΪϦεɾα΢αϯϓτϯ͔ΒΞϝϦΧ߹ऺࠃɾχϡʔϤʔΫߦ͖ͷॲঁߤւதͷ೔໨ʹ๺େ੢༸Ͱى͖ͨɻ ౰࣌࠷େͷ٬ધͰ͋ͬͨλΠλχοΫ߸͸ɺ೥݄೔ͷ࣌෼
    ࣄނݱ৔࣌ؒ
    ʹණࢁʹিಥͨ࣌͠ ʹ͸ ਓΛ৐͍ͤͯͨɻࣄނ͕ى͖͔ͯΒ࣌ؒ෼ޙͷཌ݄೔ͷ࣌෼ʹ௜຅͠ɺ ਓҎ্͕ ๢͘ͳͬͨɻ͜Ε͸೥౰࣌ɺւ೉ࣄނͷ࠷େࢮऀ਺Ͱ͋ͬͨɻ Ҿ༻: λΠλχοΫ߸௜຅ࣄނ - Wikipedia https://ja.wikipedia.org/wiki/λΠλχοΫ߸௜຅ࣄނ

25. 4. ʢ͕࣌ؒ͋Ε͹ʣ खॻ͖จࣈը૾ೝࣝʹఏग़͠ ͯΈΑ͏

26. 5. ·ͱΊ

27. ·ͱΊ • ػցֶशʹڵຯ͋Δ͚Ͳɺσʔλ࣋ͬͯͳ͍ 㱺 kaggle΍SIGNATEʹ͋ΔσʔλΛ࢖ͬͯ࿅शʂ • ػցֶशʹڵຯ͋Δ͚ͲɺͲ͏΍ͬͯεΩϧ্͛Ε͹͍͍ͷʁ 㱺 KaggleͷKernels͸ͱͯ΋ࢀߟʹͳΔʂͦͷதͰΘ͔Βͳ͍୯ޠ΍ख๏͸ผ్ௐ ΂ͯΈΑ͏ʂ

    • Kaggle͸ӳޠͰ͠ΐ… 㱺 จ໌ͷརث (Google຋༁) ͕͋ΔͷͰɺͳΜͱ͔ͳΔɻ·ͨɺӳޠͷઐ໳༻ޠ͸ CourseraͷʮMachine LearningʯίʔεΛऔΓ૊ΜͰ͍͘ͱɺࣗવͱΘ͔ͬͯ͘Δ

28. ͓ΘΓ