pandasでのOSS活動事例

Stapy x 理研AIP オープンソース研究会
https://startpython.connpass.com/event/55579/

Transcript

1. pandasͰͷ
   OSS׆ಈࣄྫ
   Masaaki Horikoshi @ ARISE analytics
   

   View Slide

2. OSS׆ಈ঺հ
   767
   2017/5/17ݱࡏ
   10
   94
   14
   9
   2
   2 3
   65
   265
   1,535
   9,608
   ggfortify(R)
   pandas-ml
   Github Stars
   Org. Members
   (Committers)
   Contributors
   

   View Slide

3. ໨࣍
   • pandasͰͷ׆ಈ
   • ϓϧϦΫΤετࣄྫ
   • OSS׆ಈ͸͡ΊͷҰา
   

   View Slide

4. pandasͱ͸
   • σʔλ෼ੳͷͨΊͷߴ଎Ͱ࢖͍΍͍͢σʔλߏ଄Λఏڙ
   • 2࣍ݩදܗࣜ
   • RͰ͍͏“data.frame”
   • Author: Wes McKinney
   • License: BSD
   • ޠݯ: PANel DAta System
   

   View Slide

5. pandasͱ͸
   import pandas as pd
   df = pd.read_csv(‘adult.csv’)
   df
   "EVMU%BUBTFUUBLFOGSPN6$*.-3FQPTJUPSZ
   -JDINBO .
   6$*.BDIJOF-FBSOJOH3FQPTJUPSZ*SWJOF $"6OJWFSTJUZPG$BMJGPSOJB 4DIPPMPG*OGPSNBUJPOBOE$PNQVUFS4DJFODF
   3FBEDTWpMF
   $PMVNOT
   *OEFY
   .JYFEEBUBUZQFT
   

   View Slide

6. pandasͱ͸
   df[['age', 'marital-status']]
   df.groupby('income')['hours-per-week'].mean()
   (SPVQCZ 4FMFDU "HHSFHBUF
   4FMFDU
   

   View Slide

7. PyData Ecosystem (Scipy
   stack)
   #PLFI NBUQMPUMJC
   4DJLJUMFBSO 4UBUTNPEFM
   /VN1Z
   1Z5BCMFT 42-"MDIFNZ
   *CJT
   4DJ1Z
   1Z4QBSL
   #MB[F%BTL
   +VQZUFS
   QBOEBT
   6TFS*OUFSGBDF
   7JTVBMJ[BUJPO
   #JH%BUB
   *0
   $PNQVUBUJPO
   .BDIJOF-FBSOJOH
   4UBUJTUJDT SQZ
   0UIFS1SPHSBNNJOH
   -BOHVBHFT
   

   View Slide

8. pandasͰͷ׆ಈ
   

   View Slide

9. ܦҢ
   1ϓϧϦΫΤετ=1ίϛοτӡ༻
   Number of Commits
   ॳΊͯͷissue
   ·ͱ΋ʹ(?)׆ಈ։࢝
   ίΞνʔϜՃೖ
   

   View Slide

10. ίϛολʹͳΔʹ͸
    • ϓϩδΣΫτʹΑͬͯҧ͏͕ɺpandasͷ৔߹:
    • ࣭ྔͱ΋े෼ͳߩݙΛ͍ͯ͠Δ͜ͱ
    • Ұ೥Ҏ্׆ಈ͍ͯ͠Δ͜ͱ
    →ݱࡏͷίΞνʔϜϝϯόͷਪનɺ౤ථͰܾఆ
    

    View Slide

11. ίϛολͷ໾ׂ
    • Make decisions about:
    • The overall scope, vision and direction of the project.
    • Strategic collaborations with other organizations or
    individuals.
    • Specific technical issues, features, bugs and pull requests.
    • The services that are run by the project.
    • Regular community discussion doesn’t produce consensus.
    

    View Slide

12. ίϛολͷۀ຿
    • Issue΁ͷճ౴
    • ίʔυϨϏϡʔ (౰વɺίϛολҎ֎ͷίʔυϨϏϡʔ΋׻ܴ)
    • ϓϧϦΫΤετͷϚʔδ
    • पลΠϯϑϥ(CI౳)ͷ੔උɺӡ༻
    • ϦϦʔε (ϦϦʔεϚωʔδϟ͕࣮ࢪ)
    • ςϨίϯ
    

    View Slide

13. ॏཁͳ͜ͱ
    • ϓϩμΫτͷ඼࣭
    • ίϛϡχςΟͷ࣭
    →ͷͨΊͷ࢓૊Έͮ͘Γ
    

    View Slide

14. ॏཁͳ͜ͱ
    ˒ ϓϩμΫτͷ඼࣭
    • ίϛϡχςΟͷ࣭
    →ͷͨΊͷ࢓૊Έͮ͘Γ
    

    View Slide

15. ϓϩμΫτͷ඼࣭
    • ࢖͍΍͍͢͜ͱ
    • Πϯετʔϧ͠΍͍͢
    • API͕Θ͔Γ΍͍͢
    • υΩϡϝϯτ͕Θ͔Γ΍͍͢
    

    View Slide

16. ϓϩμΫτͷ඼࣭
    • ඼࣭͕୲อ͞Ε͍ͯΔ͜ͱ
    • ෆ۩߹͕গͳ͍
    • ύϑΥʔϚϯε͕ྑ͍
    

    View Slide

17. ࢖͍΍͘͢͢ΔͨΊʹ
    • Πϯετʔϧ͠΍͘͢
    • ɹ
    • ֤छwheelఏڙ
    • APIΛΘ͔Γ΍͘͢
    • issue্ͰͷσΟεΧογϣϯ
    • ίʔυϨϏϡʔ
    • υΩϡϝϯτΛΘ͔Γ΍͘͢
    • υΩϡϝϯτϨϏϡʔ
    • Travis-CI Ͱͷdev൛υΩϡϝϯτϏϧυ & νΣοΫ
    

    View Slide

18. ֤छwheelఏڙ
    

    View Slide

19. υΩϡϝϯτ
    • ެࣜυΩϡϝϯτ
    • http://pandas.pydata.org/pandas-docs/stable/
    • ॻ੶
    • PythonʹΑΔσʔλ෼ੳೖ໳ / Wes McKinney
    • ༗ࢤυΩϡϝϯτ
    • Modern pandas / Tom Augspurger
    • https://tomaugspurger.github.io/modern-1.html
    

    View Slide

20. ඼࣭୲อͷͨΊʹ
    • ෆ۩߹Λগͳ͘
    • ੩తνΣοΫ (flake8)
    • ࣗಈςετ (Travis-CI, Appveyor, Circle-CI)
    • ΧόϨοδνΣοΫ (Codecov)
    • ύϑΥʔϚϯεΛྑ͘
    • ύϑΥʔϚϯεςετ (Airspeed Velocity)
    

    View Slide

21. ࣗಈςετ
    • Travis-CI
    • Python όʔδϣϯ x पล؀ڥͷ૊Έ߹Θͤ
    

    View Slide

22. ࣗಈςετ
    • AppVeyor
    • Windows؀ڥͰͷςετʹར༻
    • Circle-CI
    • Travis-CIͷิ׬తʹར༻
    

    View Slide

23. ύϑΥʔϚϯεςετ
    • Airspeed Velocity
    • ࢦఆίϛοτͰςετˍίϛοτؒൺֱ
    • https://github.com/spacetelescope/asv
    All benchmarks:
    before after ratio
    [5049b5 ] [53ac28 ]
    293.20ns 290.10ns 0.99 attrs_caching.getattr_dataframe_index.time_getattr_dataframe_index
    3.13μs 3.08μs 0.98 attrs_caching.setattr_dataframe_index.time_setattr_dataframe_index
    7.45ms 7.23ms 0.97 binary_ops.frame_add.time_frame_add
    4.14ms 4.09ms 0.99 binary_ops.frame_add_no_ne.time_frame_add_no_ne
    4.28ms 4.40ms 1.03 binary_ops.frame_add_st.time_frame_add_st
    21.67ms 21.58ms 1.00 binary_ops.frame_float_div.time_frame_float_div
    5.74ms 5.84ms 1.02 binary_ops.frame_float_div_by_zero.time_frame_float_div_by_zero
    17.90ms 17.81ms 0.99 binary_ops.frame_float_floor_by_zero.time_frame_float_floor_by_zero
    10.49ms 9.97ms 0.95 binary_ops.frame_float_mod.time_frame_float_mod
    5.95ms 6.14ms 1.03 binary_ops.frame_int_div_by_zero.time_frame_int_div_by_zero
    10.64ms 10.64ms 1.00 binary_ops.frame_int_mod.time_frame_int_mod
    7.26ms 7.31ms 1.01 binary_ops.frame_mult.time_frame_mult
    4.14ms 4.10ms 0.99 binary_ops.frame_mult_no_ne.time_frame_mult_no_ne
    

    View Slide

24. ॏཁͳ͜ͱ
    • ϓϩμΫτͷ඼࣭
    ˒ ίϛϡχςΟͷ࣭
    →ͷͨΊͷ࢓૊Έͮ͘Γ
    

    View Slide

25. ίϛϡχςΟͷ࣭
    • ΦʔϓϯͰ͋Δ͜ͱ
    • ࢀՃ͠΍͍͢
    • αϙʔςΟϒͰ͋Δ
    

    View Slide

26. ࢀՃ͠΍͘͢͢ΔͨΊʹ
    • ίϥϘϨʔγϣϯπʔϧ
    • GitHub
    • Gitter
    • υΩϡϝϯςʔγϣϯ
    • Code of Conduct
    • ։ൃऀ޲͚υΩϡϝϯτ(Wiki)
    

    View Slide

27. GitHub
    • దٓλά෇͚͠ɺٞ࿦ʹࢀՃ͠΍͘͢
    • IssueςϯϓϨʔτΛར༻͠ɺॳ৺ऀͰ΋Ϩ
    ϙʔτ͠΍͘͢
    

    View Slide

28. Code of Conduct
    • ΨόφϯεͷͨΊͷGitHubϦϙδτϦΛ੔උ
    • https://github.com/pandas-dev/pandas-
    governance
    

    View Slide

29. ։ൃऀ޲͚υΩϡϝϯτ
    • ։ൃऀ޲͚
    • ίϯτϦϏϡʔγϣϯΨΠυ
    • Gitͷ࢖͍ํ
    • ςετํ๏
    • ίʔυελΠϧ
    • ϦϦʔεϊʔτͷॻ͖ํ
    • Specialities
    • …
    • ϝϯςφ޲͚
    • ϦϦʔε࣌ͷνΣοΫϦετ
    • υΩϡϝϯτެ։ํ๏
    • …
    

    View Slide

30. νʔϜϫʔΫ
    • Specialities
    • ػೳผͷ༗ࣝऀҰཡ(ίΞνʔϜҎ֎΋ؚ)
    • ؔ࿈͢Δ Issue / ϓϧϦΫΤετ͔Β௨஌Λ
    ඈ͹͢
    • https://github.com/pandas-dev/pandas/
    wiki/Specialities
    

    View Slide

31. ஂମɾاۀ͔Βͷࢧԉ
    •
    • Institutional Partners
    • Continuum Analytics
    • Two Sigma
    

    View Slide

32. ϓϧϦΫΤετࣄྫ
    

    View Slide

33. ϓϧϦΫΤετͷछྨ
    • υΩϡϝϯτमਖ਼
    • ෆ۩߹मਖ਼
    • ػೳ௥ՃɺAPIมߋ
    • ϦϑΝΫλϦϯά
    • ςετ௥Ճ
    • …
    

    View Slide

34. ϓϧϦΫΤετࣄྫ
    • υΩϡϝϯτमਖ਼
    • https://github.com/pandas-dev/pandas/pull/13312
    • APIมߋ
    • https://github.com/pandas-dev/pandas/issues/6511
    • ςετվળ
    • https://github.com/pandas-dev/pandas/issues/10373
    

    View Slide

35. OSS׆ಈ͸͡ΊͷҰา
    

    View Slide

36. OSS׆ಈͷϝϦοτ
    • ঝೝཉٻ͕ຬͨͤΔ
    • ༗ࣝऀͷίʔυϨϏϡʔʹΑΓεΩϧΞοϓͰ
    ͖Δ
    • ಺෦࣮૷͕ཧղͰ͖ɺޮ཰తͳίʔυ͕ॻ͚Δ
    • मਖ਼ΛϚελʹऔΓࠐΜͰ΋Β͏͜ͱͰɺݸਓ
    ͰͷϝϯςෛՙΛܰݮͰ͖Δ
    

    View Slide

37. OSS׆ಈΛ࢝ΊΔʹ͸
    • Կʹߩݙ͢Δ͔ΛܾΊΔ
    • ݸਓͰ࢝ΊΔ
    • ࣗࣾϓϩμΫτΛOSSͱͯ͠ެ։͢Δ
    • ۀ຿Ͱ࢖͏OSSʹߩݙ͢Δ
    • ༗໊ͳOSSʹߩݙ͢Δ
    

    View Slide

38. OSS׆ಈΛ࢝ΊΔʹ͸
    • ͲͷΑ͏ʹߩݙ͢Δ͔ΛܾΊΔ
    • એ఻͢Δ (ϒϩάΛॻ͘ɺൃද͢Δ)
    • σʔλܥͷ৔߹ɺOSSΛ࢖ͬͨ෼ੳϊ΢ϋ΢Ͱ΋
    • ࣭໰ (StackOverflow, GitHub Issue)ʹ౴͑Δ
    • IssueΛॻ͘
    • ϓϧϦΫΤετΛૹΔ
    

    View Slide

39. ॳΊͯͷϓϧϦΫΤετ
    • ·ͣ͸ҰͭͷϓϩδΣΫτ/ػೳ͔Β
    • ϓϩηε/πʔϧͷशख़ʹ͸ίετ͕͔͔Δ
    • ίʔυͷؔ܎ΛಡΈղ͘ͷ͸೉͍͠
    

    View Slide

40. ॳΊͯͷϓϧϦΫΤετ
    • ؆୯ͳ΋ͷ͔Β
    • υΩϡϝϯτͷվగ
    • Τϥʔϝοηʔδͷվળ
    • қ͍͠Issueͷमਖ਼
    

    View Slide

41. ࠓ೔͔ΒͰ͖Δ͜ͱ
    • ؾʹͳΔϓϩδΣΫτΛWatchͯ͠ΈΔ
    • ໘നͦ͏ͳ Issue / ϓϧϦΫΤετ͕͋Δ͔
    ΋ʁ
    

    View Slide