JupyterとPythonのしらべ

Jupyter、およびPythonの入門スライドです。

Transcript

1. JUPYTERͱPYTHONͷ͠Β΂
   WATANABE Takuma (takumaw@sfo.kuramane.ne.jp)
   1 2016/02/19
2. Table of Contents
   2 ¨  Before geKng started… ¨  IntroducPon ¤  What is Jupyter ¤  History of Jupyter ¤  Jupyter in detail ¨  Jupyter with Python in use
3. Before geKng started…
   3
4. Before geKng started…
   QuesPon No. 1 ೔ৗͷࡉʑͨ͠࡞ۀΛࣗಈԽ͍ͨ͠ͱ͖ɺ Ͳ͏ͯ͠·͔͢ʁ
   4 Excel Macro
   Heavy languages (e.g. C, Java, …)
   Shell Script (e.g. Bash, BATCH, WSH, …)
   
5. Before geKng started…
   QuesPon No. 2 ͦΕ͸ͲͷΑ͏ʹॻ͍͍ͯ·͔͢ʁ
   5 With full-featured IDE? Write one and Run a big-bang? Manually save one liners each by each?
6. Before geKng started…
   QuesPon No. 3 Ͳ͏΍ͬͯอଘɾ؅ཧͯ͠·͔͢ʁ
   6 Oh! I think it must be in that folder … hey? Write once, Run once … 3 Manually save into some folder
7. Before geKng started…
   QuesPon No. 4 ͦΕΛ֦ு͍ͨ͠ͱ͖͸ʁ 7 Use another software (e.g. awk + tableau) Just fully-scratch it (e.g. ूܭ͢Δ, άϥϑΛग़͢, …) Write another code (e.g. Bash + Gnuplot)
8. Well… then?
   8 Is there any easier way? That works anyway, but…
9. Well… then?
   9 Yes! J Introducing…
11. IntroducPon
    11
12. What is Jupyter?
    12 [3]
    [1]
    [1]
    [2]
    0 2 4 6 ߲໨
    1 ߲໨
    2 ߲໨
    3 ߲໨
    4 ¨  ΢ΣϒγΣϧ
    (ϒϥ΢β͔Β࢖͑ΔγΣϧ)
    localhost:8080
    
13. What is Jupyter?
    13 [3]
    [1]
    [1]
    [2]
    0 2 4 6 ߲໨
    1 ߲໨
    2 ߲໨
    3 ߲໨
    4 ¨  ίʔυΛॻ͍ͯɺ࣮ߦͰ͖Δ
    localhost:8080
    
14. What is Jupyter?
    14 [3]
    [1]
    [1]
    [2]
    0 2 4 6 ߲໨
    1 ߲໨
    2 ߲໨
    3 ߲໨
    4 localhost:8080
    ¨  ίʔυΛॻ͍ͯɺ࣮ߦͰ͖Δ ¤  ը૾΍೚ҙͷHTMLΛग़ྗͰ͖Δ
    
15. [3]
    [1]
    [1]
    [2]
    localhost:8080
    What is Jupyter?
    15 0 2 4 6 ߲໨
    1 ߲໨
    2 ߲໨
    3 ߲໨
    4 ¨  อଘɾ؅ཧ͕Ͱ͖Δ
    localhost:8080
    Folder A
    Notebook 1
    Notebook 2
    Notebook 3
    Notebook 4
    
16. What is Jupyter?
    16 localhost:8080
    Folder A
    Notebook 1
    Notebook 2
    Notebook 3
    Notebook 4
    ¨  อଘɾ؅ཧ͕Ͱ͖Δ ¤  ʰϊʔτʱ୯ҐͰηʔϒ
    
17. What is Jupyter?
    17 localhost:8080
    Folder A
    Notebook 1
    Notebook 2
    Notebook 3
    Notebook 4
    ¨  ΠϯϙʔτɾΤΫεϙʔτ͕Ͱ͖Δ ¤  ʰϊʔτʱʹ͸શͯͷग़ྗ͕อଘ͞Ε͍ͯΔ(άϥϑ΍HTMLͳͲ)
    Jupyter Notebook
    Notebook1.ipynb
    Python Script
    Notebook1.py
    
18. History of Jupyter
    18 ¨  ݩʑIPython Notebookͱݺ͹Εͯͨ ¤  IPython: PythonͷڧԽγΣϧ ¤  IPython Notebook: IPythonͷ΢Σϒ൛(΢ΣϒγΣϧ)
    In [1]: print "Hello, world!" Hello, world! In [2]: [1]
    [1]
    print "Hello, world!"
    Hello, world!
    [2]
    
19. History of Jupyter
    19 ¨  ΢ΣϒγΣϧ͕ಠཱͯ͠Jupyterʹͳͬͨ ¤  PythonҎ֎ͷݴޠʹରԠ
    [1]
    [1]
    print "Hello, world!"
    Hello, world!
    [2]
    
20. Jupyter in detail: Architecture
    20 Jupyter Notebook
    Web Browser
    Kernels
    Kernels
    Kernels
    Kernels
    •  UIͱͳΔσʔϞϯ •  ϊʔτϒοΫͷҰཡ •  ϊʔτϒοΫͷىಈ/ऴྃ •  Ϣʔβೝূʹ΋ରԠ •  ϚϧνϢʔβʔ൛΋͋Δ (Jupyterhub)
    •  ॲཧΤϯδϯ
    •  ݴޠ͝ͱʹछྨ͕͋Δ
21. 21 Folder A
    Notebook 1
    Notebook 2
    Notebook 3
    Notebook 4
    •  ϊʔτϒοΫҰཡը໘ •  ϊʔτϒοΫͷҰཡɺىಈɺ ఀࢭ͕Ͱ͖Δ •  ϑΝΠϧγεςϜͷߏ଄·· •  1ϊʔτϒοΫ1ϑΝΠϧ •  ϑΥϧμͰ֊૚ԽͰ͖Δ •  ϊʔτϒοΫͷ࣮ଶ͸JSONܗࣜ •  ೖग़ྗ͸શͯอଘ͞ΕΔ •  μ΢ϯϩʔυͯ͠ผͷ JupyterʹΞοϓϩʔυ͢Ε͹ ೖग़ྗ͸શͯ࠶ݱ͞ΕΔ
    
22. Jupyter in detail: Notebook
    22 [3]
    [1]
    [1]
    [2]
    0 2 4 6 ߲໨
    1 ߲໨
    2 ߲໨
    3 ߲໨
    4 •  ʰϊʔτϒοΫʱ •  ෳ਺ͷʰηϧʱ͔Β੒Δ •  ηϧ୯ҐͰίʔυΛ࣮ߦ͢Δ •  ίʔυิ׬ɾϔϧϓ͋Γ •  ͻͱͭͷ࣮ߦ୯Ґ •  1ϊʔτϒοΫ͝ͱʹ 1Χʔωϧ͕ىಈ •  ηϧʹ͸छྨ͕͋Δ •  Heading(ݟग़͠)ηϧ •  Markdownηϧ •  ίʔυηϧ •  ཁ͢ΔʹʮݟͤΔʯϓϩάϥϜ
    
23. See it in acPon
    ¨  See the demo!
    23
24. Jupyter with Python in use 24
25. Get started
    25 ¨  Πϯετʔϧ ¤  $ pip install jupyter ¤  $ pip install “ipython[all]” ¨  ىಈ ¤  $ jupyter-notebook n  Screen΍tmux, byobuͷதͰىಈ͢Δͱྑ͍ ¨  ิ଍ ¤  جຊ͸PythonͰ࢖͏ n  Ҏ߱͸Pythonʹ͍ͭͯઆ໌ ¤  ଞݴޠΛ࢖͏ͱ͖͸ɺΧʔωϧΛΠϯετʔϧ͢Δ n  IPython ͷ
    Github wikiΛࢀর
26. Jupyter in use: basics 1/3
    26 ¨  ࢖͍ํ: جૅฤ 1/3 ¤  ·ͣ͸׳ΕΑ͏
    J ¤  ϊʔτϒοΫҰཡը໘ʹ͍ͭͯ n  New͔Β։͘/࡞੒͢Δ n  υϥοά&υϩοϓͰΞοϓϩʔυͰ͖Δ n  ্ͷλϒʹ͍ͭͯ n  Files λϒ: ϊʔτϒοΫͱϑΥϧμҰཡ n  Runningλϒ: ࣮ߦதͷϊʔτϒοΫɾλʔϛφϧҰཡ ¤  Ұཡը໘͔Β։͚Δ/࡞੒Ͱ͖ΔϞϊͷछྨ n  ϊʔτϒοΫ (֤ݴޠ) n  ςΩετϑΝΠϧ n  ϑΥϧμ n  λʔϛφϧ
27. Jupyter in use: basics 2/3
    27 ¨  ࢖͍ํ: جૅฤ 2/3 ¤  ϊʔτϒοΫը໘ʹ͍ͭͯ n  File > New͔Β΋৽ن࡞੒Ͱ͖Δ n  File > Make a CopyͰίϐʔͰ͖Δ n  File > RevertΑΓݹ͍όʔδϣϯ(ࣗಈอଘ)ʹ໭ͤΔ n  File > Download asͰΤΫεϙʔτͰ͖Δ n  IPython Notebook n  Python n  HTML n  Markdown n  reST n  PDF
28. Jupyter in use: basics 3/3
    28 ¨  ࢖͍ํ: جૅฤ 2/3 ¤  ηϧͷ࡞੒ n  +Ͱ৽نηϧΛͭ͘ΕΔ n  ηϧ͸લޙͰ͖Δ (↑/↓Ϙλϯ) n  InsertͰηϧΛૠೖͰ͖Δ ¤  ηϧͷ࣮ߦ n  Shin + Enter ͰηϧΛ࣮ߦ n  CellϝχϡʔΑΓҰׅ࣮ߦͰ͖Δ n  Run All, Run Above, Run Below n  Cellϝχϡʔ͔ΒCell typeΛม͑ΒΕΔ n  Code, Markdown, Heading(ݟग़͠) n  Cellϝχϡʔ͔ΒOutputͷදࣔ/ඇදࣔΛtoggleͰ͖Δ
    
29. Jupyter in use: config
    29 ¨  ࢖͍ํ: ઃఆฤ ¤  Jupyterͷઃఆ n  $ jupyter notebook --generate-config Ͱ࡞Δ n  ~/.jupyter/ ʹอଘ͞ΕΔ n  jupyter_notebook_config.py: n  ύεϫʔυͷ༗ແɺHTTPSͰ࢖͏ূ໌ॻͳͲ ¤  IPython (Χʔωϧ)ͷઃఆ n  $ ipython profile create Ͱ࡞੒ n  ~/.ipython/profile_XXX/ ʹอଘ͞ΕΔ n  ipython_config.py: n  IPythonͷઃఆ (λʔϛφϧ/ϊʔτϒοΫڞ௨) n  ىಈ࣌ʹ࣮ߦ͢Δίʔυ΍ɺgreedyͳิ׬Λ༗ޮʹ͢Δ͔ͳͲ n  ipython_kernel_config.py n  IPython Notebookͷઃఆ (ϊʔτϒοΫͷΈద༻) n  ىಈ࣌ʹ࣮ߦ͢Δίʔυɺάϥϑඳࣸʹ΋͍ͪΔόοΫΤϯυ ͳͲɻ
30. Jupyter in use: config
    30 ¨  ࢖͍ํ: ઃఆฤ (άϥϑඳࣸʹ͍ͭͯิ଍) ¤  ipython_kernel_config.py ʹ͍ͭͯ n  άϥϑඳࣸͷઃఆ߲໨͸ҎԼͷΑ͏ʹ͢Δ n  c.InlineBackend.figure_formats = set([‘rePna’]) n  RePnaղ૾౓ͷPNGΛੜ੒͢Δ৔߹͸rePna n  Png: ී௨ͷղ૾౓ͷPNG n  svg΍pdfͳͲ΋࢖͑Δ n  c.InlineBackend.rc = {} n  ͜ͷߦ͕ͳ͍ͱmatplotlibrcͷઃఆͷ͍͔͕ͭ͘ແࢹ͞ΕΔ (IPython಺෦σϑΥϧτʹ͓͖͔͑ΒΕΔ) ¤  Matplotlibͷઃఆ n  hxp://matplotlib.org/users/customizing.html ͔Β matplotlibrcΛμ΢ϯϩʔυ͢Δ n  ~/.config/matplotlib/matplotlibrc (Linux) n  ~/.matplotlib/matplotlibrc (LinuxҎ֎) n  σϑΥϧτͷαΠζɺελΠϧɺTeXΛ༗ޮʹ͢Δ͔ɺͳͲ
31. Jupyter in use: common libraries
    31 ¨  ࢖͍͔ͨ: Α͘࢖͏ϥΠϒϥϦ ¤  Կ͸ͱ΋͋Εnumpy, scipy, matplotlib͸ඞਢ n  ଞͷϥΠϒϥϦͷجૅ ¤  Numpy: CͰॻ͔Εͨߴ଎ͳ਺஋ܭࢉϥΠϒϥϦ ¤  Scipy: ՊֶܭࢉϥΠϒϥϦ ¤  Matplotlib: άϥϑඳࣸϥΠϒϥϦ ¨  Note: ¤  MacͰ͸homebrewͷtap(python)͔ΒೖΕΑ͏
32. Juputer in use: ploKng
    32 ¨  ࢖͍ํ: άϥϑΛඳࣸ͠Α͏ ¤  %pylab inline ͱଧͱ͏ n  %Ͱ࢝·Δͷ͸ magic command ͱݺ͹ΕΔ n  Matplotlib͕άϩʔόϧʹΠϯϙʔτ͞ΕΔ n  ओྲྀͰɺศརɻͨͩ͠ΠϚΠνͱݴ͏ਓ΋͍Δɻ n  ඞਢͰ͸ͳ͍ɻ import matplotlib͠ɺmatplotlib.plot()ͯ͠΋ྑ͍ n  MATLAB෩ΠϯλϑΣΠε ¤  plot, grid, legend, xPtle, yPtle, xaxis, yaxis… Ͱ άϥϑΛ࡞Δ n  Gnuplot΍MATLABͱಉ͡ཁྖͰάϥϑΛ࡞Δ ¤  ϥϕϧʹ(La)TeX΋ར༻Ͱ͖Δ
33. Jupyter in use: ploKng nicely
    33 ¨  ࢖͍ํ: ͖Ε͍ͳάϥϑΛඳࣸ͠Α͏ ¤  ग़ྗΛRePnaͳPNG΍ɺ(E)PSɺPDFɺSVGʹͰ͖Δ n  Configʹઃఆ͢Δ͔ɺsave(ϑΝΠϧ໊)ͰมߋՄ ¤  mpld3Λ࢖͏ͱɺग़ྗ͕D3.jsΛ࢖͏Α͏ʹͳΔ n  mpld3.display() Λ࠷ޙʹݺͿ͚ͩ ¤  prexyplotlibΛ࢖͏ͱඳ͕͍͍ࣸײ͡ʹ៉ྷʹͳΔ n  σϑΥϧτͷςʔϚ͸ͪΐͬͱݹ͍͘͞ͷͰ
    
34. Jupyter in use: interacPng w/ web
    34 ¨  ࢖͍ํ: ΢ΣϒΛऔΓѻ͓͏ ¤  requests, beauPfulsoup4 (΍pyquery)͋ͨΓ͕ศར ¤  requests: hxpϥΠϒϥϦ n  ඪ४ͷurllib͸෗ͬͯΔͷͰɺ͍ͭ͜Λ࢖͓͏ n  ηογϣϯΦϒδΣΫτΛ࢖͏ͱCookie΋ѻ͑Δ ¤  BeauPfulSoup4: (X)HTMLύʔα n  ෗ͬͨHTMLͰ΋͍͍ײ͡ʹಡΊΔ n  CSSηϨΫλ͕࢖͑Δ ¤  pyquery: ্ಉɻྲྀߦΓͩͯ͠Δʁ
35. Jupyter in use: storing data
    35 ¨  ࢖͍ํ: σʔλΛอଘ͠Α͏ ¤  pymongo͕ศར ¤  pymongo: MongodbΛѻ͏ϥΠϒϥϦ n  Mongodb: JSONΦϒδΣΫτΛόΧεΧͿͪࠐΊΔDB n  Կ΋ߟ͑ͳͯ͘ྑ͍ΦϒδΣΫτετϨʔδ n  εΩʔϚແ͠ɺSQLͳ͠ n  Կ͔ԋࢉ͔ͤͨͬͨ͞ΒJavaScript͕૸ΒͤΒΕΔ
    
36. Jupyter in use: data analysis
    36 ¨  ࢖͍ํ: σʔλΛ෼ੳ͠Α͏ ¤  pandas, scikit-learn, scikit-image, pillow, nltkͳͲΛ࢖͏ ¤  Pandas: σʔλϑϨʔϜϫʔΫɻ n  σʔλೖग़ྗ͕Ͱ͖Δ (CSV, JSON, Excel, HDF5, …) n  σʔλͷूܭͳͲ͕Ͱ͖Δ ¤  Scikit-learn: ػցֶशϥΠϒϥϦ ¤  Scikit-image: ্ͷը૾൛ ¤  Pillow: ը૾ॲཧϥΠϒϥϦ ¤  NLTK: ࣗવݴޠϑϨʔϜϫʔΫ ¨  E.g. ¤  hxp://ipython-books.github.io/featured-04/
37. Jupyter in use: symbolic computaPon
    37 ¨  ࢖͍ํ: ਺ࣜॲཧΛ͠Α͏ ¤  Sympy͕ྑ͍ ¤  Sympy: ਺ࣜॲཧϥΠϒϥϦ n  ਺ࣜΛදݱ͠ɺҼ਺෼ղ΍ඍੵ෼ͳͲ͕Ͱ͖Δ n  ߦྻɾ૊Έ߹ΘͤɾزԿֶͷॲཧ΋Ͱ͖Δ
    
38. And more…
    38 ¨  ͓·͚ ¤  ઌ೔LIGOʹΑΔॏྗ೾ݕग़ʹ͍ͭͯɺ ৴߸ॲཧͷҰ୺ͷϊʔτϒοΫ͕ެ։͞Ε͍ͯΔɻ ¤  Ref. hxps://losc.ligo.org/s/events/GW150914/ GW150914_tutorial.html
39. Happy compuPng!
    39
40. Appendix
    40
41. References
    41 ¨  Jupyter ¤  hxp://jupyter.org/ ¨  IPython ¤  hxp://ipython.org/ ¨  Available kernels in Jupyter ¤  hxps://github.com/ipython/ipython/wiki/IPython- kernels-for-other-languages
42. How to Install
    42 ¨  PipΛ࢖͏ ¤  ύοέʔδϚωʔδϟ ¤  c.f. Easy_install n  ͍ͭ͜͸ΞϯΠϯετʔϧग़དྷͳ͍ܗͰ Πϯετʔϧ͢ΔͷͰ΍ΊΑ͏ɻ ¤  pip install –user ͱ͢Δͱɺ~/.localҎԼʹΠϯετʔϧ Ͱ͖Δ n  ڞ༻ͷܭࢉ؀ڥ΍globalΛԚͨ͘͠ͳ͍৔߹͸ศར ¨  $ pip install jupyter ¤  JupyterΛΠϯετʔϧ͢Δ ¨  $ pip install “ipython[all]” ¤  IPythonΛΠϯετʔϧ͢Δ
    
43. How to install (for Mac)
    43 ¨  Macͷ৔߹… ¤  HomebrewʹPython tap͕͋ΔͷͰɺ ઌʹpython, numpy, scipy, matplotlibΛΠϯετʔϧ͢Δ ¤  $ brew tap homebrew/python ¤  $ brew install python n  ˞Python3ͷ৔߹͸ brew install python3 ¤  $ brew install numpy scipy n  ˞Python3ͷ৔߹͸ brew install numpy --with-python3 n  ˞scipy΋ಉ༷ʹ ¤  $ brew install matplotlib n  --with-tex Λ͚ͭͨ΄͏͕ྑ͍(TeXαϙʔτ͕༗ޮʹͳΔ)
44. How to install (for Windows)
    44 ¨  Windowsͷ৔߹… ¤  Anaconda΍EPDΛ࢖͏ख͕͋Δ n  PythonͷdistribuPon n  ಠࣗͷύοέʔδγεςϜΛ࣋ͭɻ ¤  Python, numpy, scipy, matplotlibͷ ެࣜͷόΠφϦΠϯετʔϥ΋͋Δ͕ɺ ࣗྗϝϯς͢Δͷ͸͠ΜͲ͍
    
45. Folder structure (installaPon, profile)
    45 ¨  Profile folder structure ¤  ~/.jupyter n  JupyterͷઃఆϓϩϑΝΠϧ͕֨ೲ͞ΕΔ n  IPython Χʔωϧͷઃఆ͸͜͜ʹ͸ແ͍ (4.0ʙ) ¤  ~/.ipython n  IPythonͷઃఆ͕֨ೲ͞ΕΔ ¤  ~/.matplotlib n  Matplotlibrc͕֨ೲ͞ΕΔ ¨  InstallaPon folder structure (Ұ෦) ¤  /usr/local/lib/pythonX.Y/site-packages n  (Mac͸
    /Library/Python/2.7/site-packages) ¤  ~/.local/lib/python2.7/site-packages n  (Mac͸
    ~/Library/Python/2.7/lib/python/site-packages)