JupyterとPythonのしらべ

 JupyterとPythonのしらべ

Jupyter、およびPythonの入門スライドです。

B168d7837ed7b41e4f0c1d952823628f?s=128

WATANABE Takuma

February 19, 2016
Tweet

Transcript

  1. JUPYTERͱPYTHONͷ͠Β΂ WATANABE Takuma (takumaw@sfo.kuramane.ne.jp) 1 2016/02/19

  2. Table of Contents 2 ¨  Before geKng started… ¨  IntroducPon

    ¤  What is Jupyter ¤  History of Jupyter ¤  Jupyter in detail ¨  Jupyter with Python in use
  3. Before geKng started… 3

  4. Before geKng started… QuesPon No. 1 ೔ৗͷࡉʑͨ͠࡞ۀΛࣗಈԽ͍ͨ͠ͱ͖ɺ Ͳ͏ͯ͠·͔͢ʁ 4 Excel

    Macro Heavy languages (e.g. C, Java, …) Shell Script (e.g. Bash, BATCH, WSH, …)
  5. Before geKng started… QuesPon No. 2 ͦΕ͸ͲͷΑ͏ʹॻ͍͍ͯ·͔͢ʁ 5 With full-featured

    IDE? Write one and Run a big-bang? Manually save one liners each by each?
  6. Before geKng started… QuesPon No. 3 Ͳ͏΍ͬͯอଘɾ؅ཧͯ͠·͔͢ʁ 6 Oh! I

    think it must be in that folder … hey? Write once, Run once … 3 Manually save into some folder
  7. Before geKng started… QuesPon No. 4 ͦΕΛ֦ு͍ͨ͠ͱ͖͸ʁ 7 Use another

    software (e.g. awk + tableau) Just fully-scratch it (e.g. ूܭ͢Δ, άϥϑΛग़͢, …) Write another code (e.g. Bash + Gnuplot)
  8. Well… then? 8 Is there any easier way? That works

    anyway, but…
  9. Well… then? 9 Yes! J Introducing…

  10. 10

  11. IntroducPon 11

  12. What is Jupyter? 12 [3] [1] [1] [2] 0 2

    4 6 ߲໨ 1 ߲໨ 2 ߲໨ 3 ߲໨ 4 ¨  ΢ΣϒγΣϧ(ϒϥ΢β͔Β࢖͑ΔγΣϧ) localhost:8080
  13. What is Jupyter? 13 [3] [1] [1] [2] 0 2

    4 6 ߲໨ 1 ߲໨ 2 ߲໨ 3 ߲໨ 4 ¨  ίʔυΛॻ͍ͯɺ࣮ߦͰ͖Δ localhost:8080
  14. What is Jupyter? 14 [3] [1] [1] [2] 0 2

    4 6 ߲໨ 1 ߲໨ 2 ߲໨ 3 ߲໨ 4 localhost:8080 ¨  ίʔυΛॻ͍ͯɺ࣮ߦͰ͖Δ ¤  ը૾΍೚ҙͷHTMLΛग़ྗͰ͖Δ
  15. [3] [1] [1] [2] localhost:8080 What is Jupyter? 15 0

    2 4 6 ߲໨ 1 ߲໨ 2 ߲໨ 3 ߲໨ 4 ¨  อଘɾ؅ཧ͕Ͱ͖Δ localhost:8080 Folder A Notebook 1 Notebook 2 Notebook 3 Notebook 4
  16. What is Jupyter? 16 localhost:8080 Folder A Notebook 1 Notebook

    2 Notebook 3 Notebook 4 ¨  อଘɾ؅ཧ͕Ͱ͖Δ ¤  ʰϊʔτʱ୯ҐͰηʔϒ
  17. What is Jupyter? 17 localhost:8080 Folder A Notebook 1 Notebook

    2 Notebook 3 Notebook 4 ¨  ΠϯϙʔτɾΤΫεϙʔτ͕Ͱ͖Δ ¤  ʰϊʔτʱʹ͸શͯͷग़ྗ͕อଘ͞Ε͍ͯΔ(άϥϑ΍HTMLͳͲ) Jupyter Notebook Notebook1.ipynb Python Script Notebook1.py
  18. History of Jupyter 18 ¨  ݩʑIPython Notebookͱݺ͹Εͯͨ ¤  IPython: PythonͷڧԽγΣϧ

    ¤  IPython Notebook: IPythonͷ΢Σϒ൛(΢ΣϒγΣϧ) In [1]: print "Hello, world!" Hello, world! In [2]: [1] [1] print "Hello, world!" Hello, world! [2]
  19. History of Jupyter 19 ¨  ΢ΣϒγΣϧ͕ಠཱͯ͠Jupyterʹͳͬͨ ¤  PythonҎ֎ͷݴޠʹରԠ [1] [1]

    print "Hello, world!" Hello, world! [2]
  20. Jupyter in detail: Architecture 20 Jupyter Notebook Web Browser Kernels

    Kernels Kernels Kernels •  UIͱͳΔσʔϞϯ •  ϊʔτϒοΫͷҰཡ •  ϊʔτϒοΫͷىಈ/ऴྃ •  Ϣʔβೝূʹ΋ରԠ •  ϚϧνϢʔβʔ൛΋͋Δ (Jupyterhub) •  ॲཧΤϯδϯ •  ݴޠ͝ͱʹछྨ͕͋Δ
  21. 21 Folder A Notebook 1 Notebook 2 Notebook 3 Notebook

    4 •  ϊʔτϒοΫҰཡը໘ •  ϊʔτϒοΫͷҰཡɺىಈɺ ఀࢭ͕Ͱ͖Δ •  ϑΝΠϧγεςϜͷߏ଄·· •  1ϊʔτϒοΫ1ϑΝΠϧ •  ϑΥϧμͰ֊૚ԽͰ͖Δ •  ϊʔτϒοΫͷ࣮ଶ͸JSONܗࣜ •  ೖग़ྗ͸શͯอଘ͞ΕΔ •  μ΢ϯϩʔυͯ͠ผͷ JupyterʹΞοϓϩʔυ͢Ε͹ ೖग़ྗ͸શͯ࠶ݱ͞ΕΔ
  22. Jupyter in detail: Notebook 22 [3] [1] [1] [2] 0

    2 4 6 ߲໨ 1 ߲໨ 2 ߲໨ 3 ߲໨ 4 •  ʰϊʔτϒοΫʱ •  ෳ਺ͷʰηϧʱ͔Β੒Δ •  ηϧ୯ҐͰίʔυΛ࣮ߦ͢Δ •  ίʔυิ׬ɾϔϧϓ͋Γ •  ͻͱͭͷ࣮ߦ୯Ґ •  1ϊʔτϒοΫ͝ͱʹ 1Χʔωϧ͕ىಈ •  ηϧʹ͸छྨ͕͋Δ •  Heading(ݟग़͠)ηϧ •  Markdownηϧ •  ίʔυηϧ •  ཁ͢ΔʹʮݟͤΔʯϓϩάϥϜ
  23. See it in acPon ¨  See the demo! 23

  24. Jupyter with Python in use 24

  25. Get started 25 ¨  Πϯετʔϧ ¤  $ pip install jupyter

    ¤  $ pip install “ipython[all]” ¨  ىಈ ¤  $ jupyter-notebook n  Screen΍tmux, byobuͷதͰىಈ͢Δͱྑ͍ ¨  ิ଍ ¤  جຊ͸PythonͰ࢖͏ n  Ҏ߱͸Pythonʹ͍ͭͯઆ໌ ¤  ଞݴޠΛ࢖͏ͱ͖͸ɺΧʔωϧΛΠϯετʔϧ͢Δ n  IPython ͷGithub wikiΛࢀর
  26. Jupyter in use: basics 1/3 26 ¨  ࢖͍ํ: جૅฤ 1/3

    ¤  ·ͣ͸׳ΕΑ͏J ¤  ϊʔτϒοΫҰཡը໘ʹ͍ͭͯ n  New͔Β։͘/࡞੒͢Δ n  υϥοά&υϩοϓͰΞοϓϩʔυͰ͖Δ n  ্ͷλϒʹ͍ͭͯ n  Files λϒ: ϊʔτϒοΫͱϑΥϧμҰཡ n  Runningλϒ: ࣮ߦதͷϊʔτϒοΫɾλʔϛφϧҰཡ ¤  Ұཡը໘͔Β։͚Δ/࡞੒Ͱ͖ΔϞϊͷछྨ n  ϊʔτϒοΫ (֤ݴޠ) n  ςΩετϑΝΠϧ n  ϑΥϧμ n  λʔϛφϧ
  27. Jupyter in use: basics 2/3 27 ¨  ࢖͍ํ: جૅฤ 2/3

    ¤  ϊʔτϒοΫը໘ʹ͍ͭͯ n  File > New͔Β΋৽ن࡞੒Ͱ͖Δ n  File > Make a CopyͰίϐʔͰ͖Δ n  File > RevertΑΓݹ͍όʔδϣϯ(ࣗಈอଘ)ʹ໭ͤΔ n  File > Download asͰΤΫεϙʔτͰ͖Δ n  IPython Notebook n  Python n  HTML n  Markdown n  reST n  PDF
  28. Jupyter in use: basics 3/3 28 ¨  ࢖͍ํ: جૅฤ 2/3

    ¤  ηϧͷ࡞੒ n  +Ͱ৽نηϧΛͭ͘ΕΔ n  ηϧ͸લޙͰ͖Δ (↑/↓Ϙλϯ) n  InsertͰηϧΛૠೖͰ͖Δ ¤  ηϧͷ࣮ߦ n  Shin + Enter ͰηϧΛ࣮ߦ n  CellϝχϡʔΑΓҰׅ࣮ߦͰ͖Δ n  Run All, Run Above, Run Below n  Cellϝχϡʔ͔ΒCell typeΛม͑ΒΕΔ n  Code, Markdown, Heading(ݟग़͠) n  Cellϝχϡʔ͔ΒOutputͷදࣔ/ඇදࣔΛtoggleͰ͖Δ
  29. Jupyter in use: config 29 ¨  ࢖͍ํ: ઃఆฤ ¤  Jupyterͷઃఆ

    n  $ jupyter notebook --generate-config Ͱ࡞Δ n  ~/.jupyter/ ʹอଘ͞ΕΔ n  jupyter_notebook_config.py: n  ύεϫʔυͷ༗ແɺHTTPSͰ࢖͏ূ໌ॻͳͲ ¤  IPython (Χʔωϧ)ͷઃఆ n  $ ipython profile create Ͱ࡞੒ n  ~/.ipython/profile_XXX/ ʹอଘ͞ΕΔ n  ipython_config.py: n  IPythonͷઃఆ (λʔϛφϧ/ϊʔτϒοΫڞ௨) n  ىಈ࣌ʹ࣮ߦ͢Δίʔυ΍ɺgreedyͳิ׬Λ༗ޮʹ͢Δ͔ͳͲ n  ipython_kernel_config.py n  IPython Notebookͷઃఆ (ϊʔτϒοΫͷΈద༻) n  ىಈ࣌ʹ࣮ߦ͢Δίʔυɺάϥϑඳࣸʹ΋͍ͪΔόοΫΤϯυ ͳͲɻ
  30. Jupyter in use: config 30 ¨  ࢖͍ํ: ઃఆฤ (άϥϑඳࣸʹ͍ͭͯิ଍) ¤ 

    ipython_kernel_config.py ʹ͍ͭͯ n  άϥϑඳࣸͷઃఆ߲໨͸ҎԼͷΑ͏ʹ͢Δ n  c.InlineBackend.figure_formats = set([‘rePna’]) n  RePnaղ૾౓ͷPNGΛੜ੒͢Δ৔߹͸rePna n  Png: ී௨ͷղ૾౓ͷPNG n  svg΍pdfͳͲ΋࢖͑Δ n  c.InlineBackend.rc = {} n  ͜ͷߦ͕ͳ͍ͱmatplotlibrcͷઃఆͷ͍͔͕ͭ͘ແࢹ͞ΕΔ (IPython಺෦σϑΥϧτʹ͓͖͔͑ΒΕΔ) ¤  Matplotlibͷઃఆ n  hxp://matplotlib.org/users/customizing.html ͔Β matplotlibrcΛμ΢ϯϩʔυ͢Δ n  ~/.config/matplotlib/matplotlibrc (Linux) n  ~/.matplotlib/matplotlibrc (LinuxҎ֎) n  σϑΥϧτͷαΠζɺελΠϧɺTeXΛ༗ޮʹ͢Δ͔ɺͳͲ
  31. Jupyter in use: common libraries 31 ¨  ࢖͍͔ͨ: Α͘࢖͏ϥΠϒϥϦ ¤ 

    Կ͸ͱ΋͋Εnumpy, scipy, matplotlib͸ඞਢ n  ଞͷϥΠϒϥϦͷجૅ ¤  Numpy: CͰॻ͔Εͨߴ଎ͳ਺஋ܭࢉϥΠϒϥϦ ¤  Scipy: ՊֶܭࢉϥΠϒϥϦ ¤  Matplotlib: άϥϑඳࣸϥΠϒϥϦ ¨  Note: ¤  MacͰ͸homebrewͷtap(python)͔ΒೖΕΑ͏
  32. Juputer in use: ploKng 32 ¨  ࢖͍ํ: άϥϑΛඳࣸ͠Α͏ ¤  %pylab

    inline ͱଧͱ͏ n  %Ͱ࢝·Δͷ͸ magic command ͱݺ͹ΕΔ n  Matplotlib͕άϩʔόϧʹΠϯϙʔτ͞ΕΔ n  ओྲྀͰɺศརɻͨͩ͠ΠϚΠνͱݴ͏ਓ΋͍Δɻ n  ඞਢͰ͸ͳ͍ɻ import matplotlib͠ɺmatplotlib.plot()ͯ͠΋ྑ͍ n  MATLAB෩ΠϯλϑΣΠε ¤  plot, grid, legend, xPtle, yPtle, xaxis, yaxis… Ͱ άϥϑΛ࡞Δ n  Gnuplot΍MATLABͱಉ͡ཁྖͰάϥϑΛ࡞Δ ¤  ϥϕϧʹ(La)TeX΋ར༻Ͱ͖Δ
  33. Jupyter in use: ploKng nicely 33 ¨  ࢖͍ํ: ͖Ε͍ͳάϥϑΛඳࣸ͠Α͏ ¤ 

    ग़ྗΛRePnaͳPNG΍ɺ(E)PSɺPDFɺSVGʹͰ͖Δ n  Configʹઃఆ͢Δ͔ɺsave(ϑΝΠϧ໊)ͰมߋՄ ¤  mpld3Λ࢖͏ͱɺग़ྗ͕D3.jsΛ࢖͏Α͏ʹͳΔ n  mpld3.display() Λ࠷ޙʹݺͿ͚ͩ ¤  prexyplotlibΛ࢖͏ͱඳ͕͍͍ࣸײ͡ʹ៉ྷʹͳΔ n  σϑΥϧτͷςʔϚ͸ͪΐͬͱݹ͍͘͞ͷͰ
  34. Jupyter in use: interacPng w/ web 34 ¨  ࢖͍ํ: ΢ΣϒΛऔΓѻ͓͏

    ¤  requests, beauPfulsoup4 (΍pyquery)͋ͨΓ͕ศར ¤  requests: hxpϥΠϒϥϦ n  ඪ४ͷurllib͸෗ͬͯΔͷͰɺ͍ͭ͜Λ࢖͓͏ n  ηογϣϯΦϒδΣΫτΛ࢖͏ͱCookie΋ѻ͑Δ ¤  BeauPfulSoup4: (X)HTMLύʔα n  ෗ͬͨHTMLͰ΋͍͍ײ͡ʹಡΊΔ n  CSSηϨΫλ͕࢖͑Δ ¤  pyquery: ্ಉɻྲྀߦΓͩͯ͠Δʁ
  35. Jupyter in use: storing data 35 ¨  ࢖͍ํ: σʔλΛอଘ͠Α͏ ¤ 

    pymongo͕ศར ¤  pymongo: MongodbΛѻ͏ϥΠϒϥϦ n  Mongodb: JSONΦϒδΣΫτΛόΧεΧͿͪࠐΊΔDB n  Կ΋ߟ͑ͳͯ͘ྑ͍ΦϒδΣΫτετϨʔδ n  εΩʔϚແ͠ɺSQLͳ͠ n  Կ͔ԋࢉ͔ͤͨͬͨ͞ΒJavaScript͕૸ΒͤΒΕΔ
  36. Jupyter in use: data analysis 36 ¨  ࢖͍ํ: σʔλΛ෼ੳ͠Α͏ ¤ 

    pandas, scikit-learn, scikit-image, pillow, nltkͳͲΛ࢖͏ ¤  Pandas: σʔλϑϨʔϜϫʔΫɻ n  σʔλೖग़ྗ͕Ͱ͖Δ (CSV, JSON, Excel, HDF5, …) n  σʔλͷूܭͳͲ͕Ͱ͖Δ ¤  Scikit-learn: ػցֶशϥΠϒϥϦ ¤  Scikit-image: ্ͷը૾൛ ¤  Pillow: ը૾ॲཧϥΠϒϥϦ ¤  NLTK: ࣗવݴޠϑϨʔϜϫʔΫ ¨  E.g. ¤  hxp://ipython-books.github.io/featured-04/
  37. Jupyter in use: symbolic computaPon 37 ¨  ࢖͍ํ: ਺ࣜॲཧΛ͠Α͏ ¤ 

    Sympy͕ྑ͍ ¤  Sympy: ਺ࣜॲཧϥΠϒϥϦ n  ਺ࣜΛදݱ͠ɺҼ਺෼ղ΍ඍੵ෼ͳͲ͕Ͱ͖Δ n  ߦྻɾ૊Έ߹ΘͤɾزԿֶͷॲཧ΋Ͱ͖Δ
  38. And more… 38 ¨  ͓·͚ ¤  ઌ೔LIGOʹΑΔॏྗ೾ݕग़ʹ͍ͭͯɺ ৴߸ॲཧͷҰ୺ͷϊʔτϒοΫ͕ެ։͞Ε͍ͯΔɻ ¤  Ref.

    hxps://losc.ligo.org/s/events/GW150914/ GW150914_tutorial.html
  39. Happy compuPng! 39

  40. Appendix 40

  41. References 41 ¨  Jupyter ¤  hxp://jupyter.org/ ¨  IPython ¤  hxp://ipython.org/

    ¨  Available kernels in Jupyter ¤  hxps://github.com/ipython/ipython/wiki/IPython- kernels-for-other-languages
  42. How to Install 42 ¨  PipΛ࢖͏ ¤  ύοέʔδϚωʔδϟ ¤  c.f.

    Easy_install n  ͍ͭ͜͸ΞϯΠϯετʔϧग़དྷͳ͍ܗͰ Πϯετʔϧ͢ΔͷͰ΍ΊΑ͏ɻ ¤  pip install –user ͱ͢Δͱɺ~/.localҎԼʹΠϯετʔϧ Ͱ͖Δ n  ڞ༻ͷܭࢉ؀ڥ΍globalΛԚͨ͘͠ͳ͍৔߹͸ศར ¨  $ pip install jupyter ¤  JupyterΛΠϯετʔϧ͢Δ ¨  $ pip install “ipython[all]” ¤  IPythonΛΠϯετʔϧ͢Δ
  43. How to install (for Mac) 43 ¨  Macͷ৔߹… ¤  HomebrewʹPython

    tap͕͋ΔͷͰɺ ઌʹpython, numpy, scipy, matplotlibΛΠϯετʔϧ͢Δ ¤  $ brew tap homebrew/python ¤  $ brew install python n  ˞Python3ͷ৔߹͸ brew install python3 ¤  $ brew install numpy scipy n  ˞Python3ͷ৔߹͸ brew install numpy --with-python3 n  ˞scipy΋ಉ༷ʹ ¤  $ brew install matplotlib n  --with-tex Λ͚ͭͨ΄͏͕ྑ͍(TeXαϙʔτ͕༗ޮʹͳΔ)
  44. How to install (for Windows) 44 ¨  Windowsͷ৔߹… ¤  Anaconda΍EPDΛ࢖͏ख͕͋Δ

    n  PythonͷdistribuPon n  ಠࣗͷύοέʔδγεςϜΛ࣋ͭɻ ¤  Python, numpy, scipy, matplotlibͷ ެࣜͷόΠφϦΠϯετʔϥ΋͋Δ͕ɺ ࣗྗϝϯς͢Δͷ͸͠ΜͲ͍
  45. Folder structure (installaPon, profile) 45 ¨  Profile folder structure ¤ 

    ~/.jupyter n  JupyterͷઃఆϓϩϑΝΠϧ͕֨ೲ͞ΕΔ n  IPython Χʔωϧͷઃఆ͸͜͜ʹ͸ແ͍ (4.0ʙ) ¤  ~/.ipython n  IPythonͷઃఆ͕֨ೲ͞ΕΔ ¤  ~/.matplotlib n  Matplotlibrc͕֨ೲ͞ΕΔ ¨  InstallaPon folder structure (Ұ෦) ¤  /usr/local/lib/pythonX.Y/site-packages n  (Mac͸/Library/Python/2.7/site-packages) ¤  ~/.local/lib/python2.7/site-packages n  (Mac͸~/Library/Python/2.7/lib/python/site-packages)