pandasでのOSS活動事例と最初の一歩

22f56e55955b9aa693081ed5dc6400ae?s=47 Sinhrks
September 09, 2017

 pandasでのOSS活動事例と最初の一歩

@ PyConJP 2017 Keynote

22f56e55955b9aa693081ed5dc6400ae?s=128

Sinhrks

September 09, 2017
Tweet

Transcript

  1. pandasͰͷOSS׆ಈ ࣄྫͱ࠷ॳͷҰา Masaaki Horikoshi @ ARISE analytics

  2. • ງӽ ਅө @ גࣜձࣾ ARISE analytics • OSS׆ಈ •

    https://github.com/sinhrks ࣗݾ঺հ 867 2017/9/3ݱࡏ 11 106 15 9 2 2 3 77 286 1,879 10,699 ggfortify(R) pandas-ml Github Stars Org. Members (Committers) Contributors
  3. ձࣾ঺հ • גࣜձࣾ ARISE analytics • 2017೥4݄ઃཱ • ۀ຿ •

    ௨৴αʔϏεɺEίϚʔεͳͲʹ͓͚Δσʔλ෼ੳ • ϨίϝϯυΤϯδϯ΍৽نιϦϡʔγϣϯͷ։ൃ
  4. ໨࣍ • PythonͱՊֶܭࢉ • pandasͰͷOSS׆ಈࣄྫ • OSS׆ಈ ࠷ॳͷҰา

  5. PythonͱՊֶܭࢉ

  6. Pythonͱ͸ • ൚༻ϓϩάϥϛϯάݴޠ • ۙ೥ɺՊֶܭࢉͰ΋޿͘࢖ΘΕ͍ͯΔ • Google Trends (Python /

    ೔ຊ)
  7. IEEE Spectrum The 2017 Top Programming Language http://spectrum.ieee.org/computing/software/the-2017-top-prog

  8. Pythonͱ͸ • Guido van Rossum: • … The first sound

    bite I had for Python was, "Bridge the gap between the shell and C.” • So I never intended Python to be the primary language for programmers … It was intended to be a second language for people who were already experienced programmers… • We now have a large community of people using Python as an educational language … These people aren't and may never be professional programmers, but they still find some programming skills useful. • Python's Design Goals (A Conversation with Guido van Rossum, Part II) by Bill Venners 2003/01/20
  9. ՊֶܭࢉͱPython • Python͕ՊֶܭࢉͰ࢖ΘΕ͍ͯΔཧ༝(ࢲݟ) • ڭҭػؔͰ࢖ΘΕ͍ͯΔ • C/Fortranࢿ࢈ͷར༻͕༰қ • ՊֶܭࢉܥOSS͕ൃలɾ੒ख़

  10. Պֶܭࢉύοέʔδ

  11. Պֶܭࢉύοέʔδ

  12. ՊֶܭࢉܥίϛϡχςΟ • ࠃ಺֎Ͱ༷ʑͳίϛϡχςΟΠϕϯτ͕੝ΜʹߦΘ Ε͍ͯΔ • PyCon • PyData • SciPy

    • …
  13. PyCon JP 2016 ࢀՃऀΞϯέʔτ • Q3. ීஈɺͲͷ෼໺ͰPythonΛ࢖͍ͬͯ·͔͢ʁ 168 127 111

    94 55 51 14 12 12 2 0 20 40 60 80 100 120 140 160 180 &! / Private tools development Web+% / Web development #" */machine learning '( / Research & Development )$ / System management Big Data    +% / Desktop Application   / Computer graphics +% / Game programming https://pycon.jp/2016/ja/files/32/pyconjp2016survey.htm
  14. ւ֎ͷՊֶܭࢉܥΠϕϯτ • 2017೥7݄ SciPy 2017ʹࢀՃ • PythonΛ༻͍ͨσʔλ෼ੳɺՊֶݚڀʹؔ͢Δੈք࠷େ ن໛ͷΠϕϯτ • ࢀՃऀ:

    ໿700໊ • ظؒ: 9೔ؒ (νϡʔτϦΞϧɺεϓϦϯτؚ) • ։࠵৔ॴ: ςΩαεେֶ (ΦʔεςΟϯ)
  15. SciPy2017ͷ༷ࢠ • Keynote: Drilling the Chicxulub Impact Structure / Sean

    Gulick, University of Texas • ڪཽઈ໓͸୯ҰͷᯁੴͷিಥͰͰ͖ͨՄೳੑ͕ߴ͍ͱ݁࿦ • σʔλղੳʹ͸PythonΛར༻
  16. pandasͰͷOSS׆ಈࣄྫ

  17. ͦ΋ͦ΋ pandas ͬͯԿʁ

  18. pandasͱ͸ • σʔλ෼ੳͷͨΊͷߴ଎Ͱ࢖͍΍͍͢σʔλߏ଄Λఏڙ • 2࣍ݩදܗࣜ • RͰ͍͏“data.frame” • Author: Wes

    McKinney • License: BSD • ޠݯ: PANel DAta System
  19. pandasͱ͸ import pandas as pd df = pd.read_csv(‘adult.csv’) df "EVMU%BUBTFUUBLFOGSPN6$*.-3FQPTJUPSZ

    -JDINBO .  6$*.BDIJOF-FBSOJOH3FQPTJUPSZ<IUUQBSDIJWFJDTVDJFEVNM>*SWJOF $"6OJWFSTJUZPG$BMJGPSOJB 4DIPPMPG*OGPSNBUJPOBOE$PNQVUFS4DJFODF 3FBEDTWpMF $PMVNOT *OEFY .JYFEEBUBUZQFT
  20. pandasͱ͸ df[['age', 'marital-status']] df.groupby('income')['hours-per-week'].mean() (SPVQCZ 4FMFDU "HHSFHBUF 4FMFDU

  21. ͳͥ pandas ͕ඞཁͳͷʁ SQLͰΑ͘ͳ͍ʁ

  22. ͳͥ pandas ͔ • ݱ࣮ͷ(Ԛ͍)σʔλΛɺ௚ײతʹૢ࡞Ͱ͖Δ

  23. ͳͥ pandas ͔ • ྫ: খചళͰΩϟϯϖʔϯɾఱީ͕ചΓ্͛ʹ༩͑ΔӨڹΛ஌Γ͍ͨ • σʔλιʔε͕όϥόϥ… ೔࣌ ৔ॴ

    ఱީ  ౦ژ౎໨ࠇ۠ ੖Ε Ωϟϯϖʔϯ໊শ ։࢝೔ ऴྃ೔ 999   :::   Ωϟϯϖʔϯ͸ళฮ͝ͱʹ &YDFMͰه࿥ʜ ఱީσʔλ͸8FC͔Β$47Ͱ μ΢ϯϩʔυ ళฮ*% Ϣʔβ*% Ϩγʔτ*% ߪങ೔࣌ "" BBB   ## CCC   104σʔλΛ42-Ͱऔಘʜ
  24. ͳͥ pandas ͔ • ྫ: খചళͰΩϟϯϖʔϯɾఱީ͕ചΓ্͛ʹ༩͑ΔӨڹΛ஌Γ͍ͨ • ༷ʑͳߟྀࣄ߲… ళฮ*% Ϣʔβ*%

    Ϩγʔτ*% ߪങ೔࣌ "" BBB   ## CCC   ೔࣌ ৔ॴ ఱީ  ౦ژ౎໨ࠇ۠ ੖Ε Ωϟϯϖʔϯ໊শ ։࢝೔ ऴྃ೔ 999   :::   ෼୯ҐͰूܭ͍ͨ͠ ৽ฉ޿ࠂͱళ಄νϥγͰ͸ޮ Ռ͕ҧ͏ʁ Ұఆ࣌ؒ͝ͱͷ؍ଌ஋Λ ద౰ͳϧʔϧͰඥ෇͚
  25. pandas ͷҐஔ෇͚ CRISP-DM: Cross Industry Standard Process for Data Mining

    Ϗδωεͷཧղ σʔλͷཧղ σʔλ४උ ϞσϦϯά ධՁ ల։ ݱ࣮ͷσʔλΛཧղ͠ɺ ෼ੳͰ͖Δܗʹམͱ͢
  26. pandas ΤίγεςϜ • I/O • pandas-datareader • pandas-msgpack • pandas-gbq

    • υϝΠϯ • geopandas • xarray • ػցֶश • sklearn-pandas • pandas-ml • ฒྻॲཧ • Dask • ։ൃऀ޲͚ • pandas-compat Լઢ͸pandas/pydataϦϙδτϦʹ͋Δ΋ͷ
  27. ίϛολͬͯ ͲΜͳ͜ͱΛͯ͠Δʁ

  28. pandas ͰͷOSS׆ಈܦҢ 1ϓϧϦΫΤετ=1ίϛοτӡ༻ Number of Commits ॳΊͯͷissue ·ͱ΋ʹ(?)׆ಈ։࢝ ίΞνʔϜՃೖ

  29. ίϛολʹͳΔʹ͸ • ϓϩδΣΫτʹΑͬͯҧ͏͕ɺpandasͷ৔߹: • ࣭ɾྔͱ΋े෼ͳߩݙΛ͍ͯ͠Δ͜ͱ • Ұ೥Ҏ্׆ಈ͍ͯ͠Δ͜ͱ →ݱࡏͷίΞνʔϜϝϯόͷਪનɺ౤ථͰܾఆ

  30. ίϛολͷ໾ׂ • Make decisions about: • The overall scope, vision

    and direction of the project. • Strategic collaborations with other organizations or individuals. • Specific technical issues, features, bugs and pull requests. • The services that are run by the project. • Regular community discussion doesn’t produce consensus.
  31. ίϛολͷۀ຿ • Issue΁ͷճ౴ • ίʔυϨϏϡʔ • ϓϧϦΫΤετͷϚʔδ • पลΠϯϑϥ(CI౳)ͷ੔උɺӡ༻ •

    ϦϦʔε (ϦϦʔεϚωʔδϟ͕࣮ࢪ) • ςϨίϯ
  32. ॏཁͳ͜ͱ • ϓϩμΫτͷ඼࣭ • ίϛϡχςΟͷ඼࣭ • ͜ΕΒΛҡ࣋͢ΔͨΊͷ࢓૊Έͮ͘Γ

  33. ϓϩμΫτͷ඼࣭ • ࢖͍΍͍͢ • Πϯετʔϧ • ϢʔβϏϦςΟ • υΩϡϝϯτ •

    ඼࣭͕୲อ͞Ε͍ͯΔ • ܧଓతΠϯςάϨʔγϣϯ • ύϑΥʔϚϯεςετ
  34. Πϯετʔϧ • Anaconda • ֤छ wheel ͷఏڙ

  35. ϢʔβϏϦςΟ • API • ύοέʔδ಺֎Ͱͷ౷Ұੑ • ໊લۭؒΛద੾ʹ෼ׂ(ΞΫηα) • ޙํޓ׵ੑ •

    ௚ͪʹมߋͤͣඇਪ঑Խˠ2ϝδϟʔόʔδϣϯޙʹมߋ • աڈʹγϦΞϥΠζ͞ΕͨΦϒδΣΫτͷಡΈࠐΈΛ୲อ
  36. υΩϡϝϯτ • ެࣜυΩϡϝϯτ • http://pandas.pydata.org/pandas-docs/stable/ • ॻ੶ • PythonʹΑΔσʔλ෼ੳೖ໳ /

    Wes McKinney • ༗ࢤυΩϡϝϯτ • Modern pandas / Tom Augspurger • https://tomaugspurger.github.io/modern-1.html
  37. ܧଓతΠϯςάϨʔγϣϯ • ੩తνΣοΫ (flake8) • ࣗಈςετ (Travis-CI, AppVeyor, Circle-CI) •

    Python όʔδϣϯ x पล؀ڥͷ૊Έ߹Θͤ • ΧόϨοδνΣοΫ (Codecov)
  38. ύϑΥʔϚϯεςετ • Airspeed Velocity • ࢦఆίϛοτͰςετˍίϛοτؒൺֱ • 1000↑ͷϕϯνϚʔΫΛ࣮ߦ http://pandas.pydata.org/speed/pandas

  39. ॏཁͳ͜ͱ • ϓϩμΫτͷ඼࣭ • ίϛϡχςΟͷ඼࣭ • ͜ΕΒΛҡ࣋͢ΔͨΊͷ࢓૊Έͮ͘Γ

  40. ίϛϡχςΟͷ඼࣭ • ΦʔϓϯͰ͋Δ͜ͱ • ࢀՃ͠΍͍͢ • αϙʔςΟϒͰ͋Δ

  41. ࢀՃ͠΍͘͢͢ΔͨΊʹ • Φʔϓϯͳίϛϡχέʔγϣϯ (GitHub, Gitter) • ߦಈنൣ (Code Of Conduct)

    • ։ൃऀ޲͚υΩϡϝϯτ(GitHub Wiki)
  42. GitHub • దٓλά෇͚͠ɺٞ࿦ʹࢀՃ͠΍͘͢ • IssueςϯϓϨʔτΛར༻͠ɺॳ৺ऀͰ΋Ϩ ϙʔτ͠΍͘͢

  43. ߦಈنൣ (Code of Conduct) • ΨόφϯεͷͨΊͷGitHubϦϙδτϦΛ੔උ • https://github.com/pandas-dev/pandas- governance

  44. ։ൃऀ޲͚υΩϡϝϯτ • ։ൃऀ޲͚ • ίϯτϦϏϡʔγϣϯΨΠυ • Gitͷ࢖͍ํ • ςετํ๏ •

    ίʔσΟϯάελΠϧ • ϦϦʔεϊʔτͷॻ͖ํ • Specialities • … • ϝϯςφ޲͚ • ϦϦʔε࣌ͷνΣοΫϦετ • υΩϡϝϯτެ։ํ๏ • …
  45. ։ൃऀ޲͚υΩϡϝϯτ • Specialities • ػೳผͷ༗ࣝऀҰཡ(ίΞνʔϜҎ֎΋ؚ) • ؔ࿈͢Δ Issue / ϓϧϦΫΤετ͔Β௨஌Λ

    ඈ͹͢ • https://github.com/pandas-dev/pandas/ wiki/Specialities
  46. ஂମɾاۀ͔Βͷࢧԉ • • Institutional Partners • Continuum Analytics • Two

    Sigma
  47. OSS׆ಈ ࠷ॳͷҰา

  48. OSS׆ಈͷϝϦοτ • ঝೝཉٻ͕ຬͨͤΔ • ༗ࣝऀͷίʔυϨϏϡʔʹΑΓεΩϧΞοϓͰ ͖Δ • ಺෦࣮૷͕ཧղͰ͖ɺޮ཰తͳίʔυ͕ॻ͚Δ • मਖ਼ΛϚελʹऔΓࠐΜͰ΋Β͏͜ͱͰɺݸਓ

    ͷϝϯςෛՙΛܰݮͰ͖Δ
  49. OSS׆ಈΛ࢝ΊΔʹ͸ • ίʔυमਖ਼͚͕ͩOSS׆ಈͰ͸ͳ͍ • એ఻͢Δ (ϒϩάΛॻ͘ɺൃද͢Δ) • σʔλܥͷ৔߹ɺOSSΛ࢖ͬͨ෼ੳϊ΢ϋ΢Ͱ΋ • ࣭໰

    (StackOverflow, GitHub Issue)ʹ౴͑Δ • IssueΛॻ͘ • ϓϧϦΫΤετΛૹΔ
  50. OSS׆ಈΛ࢝ΊΔʹ͸ • Կʹߩݙ͢Δ͔ΛܾΊΔ • ݸਓϓϩδΣΫτͱͯ͠ • ࣗࣾϓϩμΫτΛOSSͱͯ͠ • ۀ຿Ͱ࢖͏OSSʹରͯ͠ •

    ༗໊ͳOSSʹରͯ͠
  51. ϓϧϦΫΤετ͕ ૹΕΔΑ͏ʹͳΓ͍ͨʂ

  52. ॳΊͯͷϓϧϦΫΤετ • ·ͣ͸ҰͭͷϓϩδΣΫτ/ػೳ͔Β • ϓϩηε/πʔϧͷशख़ʹ͸ίετ͕͔͔Δ • ίʔυͷؔ܎ΛಡΈղ͘ͷ͸೉͍͠

  53. ॳΊͯͷϓϧϦΫΤετ • ؆୯ͳ΋ͷ͔Β • υΩϡϝϯτͷվగ • Τϥʔϝοηʔδͷվળ • қ͍͠όάमਖ਼

  54. ϓϧϦΫΤετͷྲྀΕ • ௚͍ͨ͠ issue Λ୳͢ • issueʹͳ͍৔߹ɺ·ͣissueΛॻ͍ͨํ͕ྑ͍ • ίʔυΛमਖ਼͢Δ •

    ϓϧϦΫΤετΛૹΔ
  55. ϓϧϦΫΤετର৅ͷ୳͠ํ • େ͖ΊͷϓϩμΫτͷ৔߹ɺλάͰ੔ཧ͞Ε͍ͯ Δ • ෼໺΍೉қ౓Ͱର৅Λ୳͢ • ଞͷϢʔβ͕ணख͍ͯ͠Δ৔߹͸ආ͚ͨํ͕ྑ͍

  56. ϓϧϦΫΤετͷૹΓํ • ࠷௿ݶͷϧʔϧΛकΔ • ໌ࣔ͞Ε͍ͯΔ΋ͷ͚ͩͰྑ͍ • ςετΛ௥Ճ͢Δ • ίʔσΟϯάن໿ΛकΔ •

  57. ϓϧϦΫΤετΛૹͬͨޙ • ։ൃऀ΍ଞϢʔβʹΑΔϨϏϡʔ • (ϓϩδΣΫτʹΑΓ)υΩϡϝϯτɺϦϦʔεϊʔτͷ੔උ • Ϛʔδ

  58. ϋʔυϧʹײ͡ΔཁҼ • ӳޠ • ίϛϡχςΟϩʔΧϧϧʔϧ • Gitͷ࢖͍ํ • ٕज़ྗ •

    Կ͔໰୊͕͋Ε͹ڭ͑ͯ͘ΕΔͷͰɺա౓ʹؾΛ࢖͏ඞཁ͸ͳ͍
  59. ϓϧϦΫΤετʹ͸ ͲΜͳ΋ͷ͕͋Δʁ

  60. ϓϧϦΫΤετͷछྨ • υΩϡϝϯτमਖ਼ • ෆ۩߹मਖ਼ • ػೳ௥ՃɺAPIมߋ • ϦϑΝΫλϦϯά •

    ςετ௥Ճ • …
  61. ϓϧϦΫΤετࣄྫ (1/3) υΩϡϝϯτमਖ਼ • मਖ਼಺༰Λ؆ܿʹهࡌ https://github.com/pandas-dev/pandas/pull/13312 ͜ͷυΩϡϝϯτɺͳΜ͔յΕͯΔͬΆ͍

  62. ϓϧϦΫΤετࣄྫ (1/3) υΩϡϝϯτमਖ਼ ۭനߦΛೖΕ͚ͨͩ

  63. • ෳࡶͳมߋͷ৔߹͸मਖ਼લޙͷڍಈΛ੔ཧ͢ Δ ࠓͷڍಈ͸ʜ͕ͩ मਖ਼ޙͷڍಈ͸ʜʹͳΔɻ ͜Ε͸ʜͱ͍͏ϧʔϧʹج͍͍ͮͯͯʜ https://github.com/pandas-dev/pandas/pull/13849 ϓϧϦΫΤετࣄྫ (2/3) ࢓༷มߋ

  64. • ϨϏϡʔࢦఠΛड͚ͭͭɺ࢓༷ΛݻΊΔ 999ͷ࣌ʹΤϥʔग़ͨ͠ํ͕Α͘ͳ͍ʁ ͦ͏ͳΜ͚ͩͲɺ999Λېࢭ͢Δͱผͷػೳ͕ಈ͔ͳ͘ͳΔ ͡Ό͋࢓ํͳ͍ͶɻଞͷΈΜͳ͸Ͳ͏ࢥ͏ʁ Ҏ߱ٞ࿦ ϓϧϦΫΤετࣄྫ (2/3) ࢓༷มߋ

  65. • ϦϦʔεϊʔτ΍υΩϡϝϯτΛमਖ਼͢Δ ϓϧϦΫΤετࣄྫ (2/3) ࢓༷มߋ

  66. ϓϧϦΫΤετࣄྫ (3/3) όάमਖ਼ • ҙਤ͕఻ΘΒͳ͚Ε͹ίʔυΛࣔ͢ https://github.com/pandas-dev/pandas/pull/13312 ͍΍ɺผͷͱ͜௚ͨ͠ํ͕͍͍Μ͡Όͳ͍ʁ ӳޠ͕͓͔͍͔͠΋ʜɻ मਖ਼ՕॴΛϓϧϦΫͰૹͬͨͷͰ֬ೝͯ͠ ࠓͷڍಈ͸ʜͰ

    मਖ਼ޙͷڍಈ͸ʜʹͳΔɻ ͨͿΜʜͷॲཧΛमਖ਼͢Ε͹͍͍
  67. ϓϧϦΫΤετࣄྫ (3/3) όάमਖ਼ • (Өڹൣғ͕େ͖͍৔߹)ͻͨ͢ΒςετΛ଍͢ ҟͳΔΫϥε͕ಉ͡ڍಈʹͳ Δ͔νΣοΫ

  68. ϓϧϦΫΤετࣄྫ (3/3) όάमਖ਼ • (Өڹൣғ͕େ͖͍৔߹)ͻͨ͢ΒςετΛ଍͢ ܕͷ૊Έ߹ΘͤΛ໢ཏతʹ νΣοΫ ͦΕͧΕͷܕͷ૊Έ߹Θ͕ͤ /VN1Zͱಉ͡ڍಈʹͳΔ͔ νΣοΫ

  69. ؾΛ͚͍ͭͯΔ͜ͱ • ίϛϡχέʔγϣϯ • ӳޠ͸௨͡ͳ͍͔΋͠Εͳ͍͕ɺίʔυͳΒ఻ΘΔ • ίʔυमਖ਼ • ہॴతͳमਖ਼Λආ͚Δ •

    ςετΛͪΌΜͱॻ͘ • ࣗ෼Ҏ֎͕ͦͷίʔυΛ৮ͬͨࡍ΋όάͷຒΊࠐΈΛ༧๷Ͱ͖Δ͔ʁ • υΩϡϝϯτ΋Ͱ͖Δ͚ͩॻ͘ • (࢓༷͕͔ͬ͠Γ఻ΘΕ͹)୭͔௚ͯ͘͠ΕΔ
  70. ΍ͬͺΓͪΐͬͱ େมͦ͏…ʁ

  71. ࠓ೔͔ΒͰ͖Δ͜ͱ • ؾʹͳΔϓϩδΣΫτΛWatchͯ͠ΈΔ • ίϛϡχςΟͷงғؾ͕Θ͔Δ • ໘നͦ͏ͳ Issue / ϓϧϦΫΤετ͕͋Δ͔΋ʁ

  72. ·ͱΊ

  73. ·ͱΊ • PythonͱՊֶܭࢉ • pandasͰͷOSS׆ಈࣄྫ • OSS׆ಈ ࠷ॳͷҰา • PythonՊֶܭࢉίϛϡχςΟʹࢀՃͯ͠ΈΑ͏

    • ؾʹͳΔϓϩδΣΫτΛWatchͯ͠ΈΑ͏