Share
Embed
Transcript
45 slides
-
Table of Contents 2 ¨ Before geKng started… ¨ IntroducPon
¤ What is Jupyter ¤ History of Jupyter ¤ Jupyter in detail ¨ Jupyter with Python in use -
Before geKng started… QuesPon No. 1 ৗͷࡉʑͨ͠࡞ۀΛࣗಈԽ͍ͨ͠ͱ͖ɺ Ͳ͏ͯ͠·͔͢ʁ 4 Excel
Macro Heavy languages (e.g. C, Java, …) Shell Script (e.g. Bash, BATCH, WSH, …) -
Before geKng started… QuesPon No. 2 ͦΕͲͷΑ͏ʹॻ͍͍ͯ·͔͢ʁ 5 With full-featured
IDE? Write one and Run a big-bang? Manually save one liners each by each? -
Before geKng started… QuesPon No. 3 Ͳ͏ͬͯอଘɾཧͯ͠·͔͢ʁ 6 Oh! I
think it must be in that folder … hey? Write once, Run once … 3 Manually save into some folder -
Before geKng started… QuesPon No. 4 ͦΕΛ֦ு͍ͨ͠ͱ͖ʁ 7 Use another
software (e.g. awk + tableau) Just fully-scratch it (e.g. ूܭ͢Δ, άϥϑΛग़͢, …) Write another code (e.g. Bash + Gnuplot) -
Well… then? 8 Is there any easier way? That works
anyway, but… -
What is Jupyter? 12 [3] [1] [1] [2] 0 2
4 6 ߲ 1 ߲ 2 ߲ 3 ߲ 4 ¨ ΣϒγΣϧ(ϒϥβ͔Β͑ΔγΣϧ) localhost:8080 -
What is Jupyter? 13 [3] [1] [1] [2] 0 2
4 6 ߲ 1 ߲ 2 ߲ 3 ߲ 4 ¨ ίʔυΛॻ͍ͯɺ࣮ߦͰ͖Δ localhost:8080 -
What is Jupyter? 14 [3] [1] [1] [2] 0 2
4 6 ߲ 1 ߲ 2 ߲ 3 ߲ 4 localhost:8080 ¨ ίʔυΛॻ͍ͯɺ࣮ߦͰ͖Δ ¤ ը૾ҙͷHTMLΛग़ྗͰ͖Δ -
[3] [1] [1] [2] localhost:8080 What is Jupyter? 15 0
2 4 6 ߲ 1 ߲ 2 ߲ 3 ߲ 4 ¨ อଘɾཧ͕Ͱ͖Δ localhost:8080 Folder A Notebook 1 Notebook 2 Notebook 3 Notebook 4 -
What is Jupyter? 16 localhost:8080 Folder A Notebook 1 Notebook
2 Notebook 3 Notebook 4 ¨ อଘɾཧ͕Ͱ͖Δ ¤ ʰϊʔτʱ୯ҐͰηʔϒ -
What is Jupyter? 17 localhost:8080 Folder A Notebook 1 Notebook
2 Notebook 3 Notebook 4 ¨ ΠϯϙʔτɾΤΫεϙʔτ͕Ͱ͖Δ ¤ ʰϊʔτʱʹશͯͷग़ྗ͕อଘ͞Ε͍ͯΔ(άϥϑHTMLͳͲ) Jupyter Notebook Notebook1.ipynb Python Script Notebook1.py -
History of Jupyter 18 ¨ ݩʑIPython NotebookͱݺΕͯͨ ¤ IPython: PythonͷڧԽγΣϧ
¤ IPython Notebook: IPythonͷΣϒ൛(ΣϒγΣϧ) In [1]: print "Hello, world!" Hello, world! In [2]: [1] [1] print "Hello, world!" Hello, world! [2] -
History of Jupyter 19 ¨ ΣϒγΣϧ͕ಠཱͯ͠Jupyterʹͳͬͨ ¤ PythonҎ֎ͷݴޠʹରԠ [1] [1]
print "Hello, world!" Hello, world! [2] -
Jupyter in detail: Architecture 20 Jupyter Notebook Web Browser Kernels
Kernels Kernels Kernels • UIͱͳΔσʔϞϯ • ϊʔτϒοΫͷҰཡ • ϊʔτϒοΫͷىಈ/ऴྃ • ϢʔβೝূʹରԠ • ϚϧνϢʔβʔ൛͋Δ (Jupyterhub) • ॲཧΤϯδϯ • ݴޠ͝ͱʹछྨ͕͋Δ -
21 Folder A Notebook 1 Notebook 2 Notebook 3 Notebook
4 • ϊʔτϒοΫҰཡը໘ • ϊʔτϒοΫͷҰཡɺىಈɺ ఀࢭ͕Ͱ͖Δ • ϑΝΠϧγεςϜͷߏ·· • 1ϊʔτϒοΫ1ϑΝΠϧ • ϑΥϧμͰ֊ԽͰ͖Δ • ϊʔτϒοΫͷ࣮ଶJSONܗࣜ • ೖग़ྗશͯอଘ͞ΕΔ • μϯϩʔυͯ͠ผͷ JupyterʹΞοϓϩʔυ͢Ε ೖग़ྗશͯ࠶ݱ͞ΕΔ -
Jupyter in detail: Notebook 22 [3] [1] [1] [2] 0
2 4 6 ߲ 1 ߲ 2 ߲ 3 ߲ 4 • ʰϊʔτϒοΫʱ • ෳͷʰηϧʱ͔ΒΔ • ηϧ୯ҐͰίʔυΛ࣮ߦ͢Δ • ίʔυิɾϔϧϓ͋Γ • ͻͱͭͷ࣮ߦ୯Ґ • 1ϊʔτϒοΫ͝ͱʹ 1Χʔωϧ͕ىಈ • ηϧʹछྨ͕͋Δ • Heading(ݟग़͠)ηϧ • Markdownηϧ • ίʔυηϧ • ཁ͢ΔʹʮݟͤΔʯϓϩάϥϜ -
Get started 25 ¨ Πϯετʔϧ ¤ $ pip install jupyter
¤ $ pip install “ipython[all]” ¨ ىಈ ¤ $ jupyter-notebook n Screentmux, byobuͷதͰىಈ͢Δͱྑ͍ ¨ ิ ¤ جຊPythonͰ͏ n Ҏ߱Pythonʹ͍ͭͯઆ໌ ¤ ଞݴޠΛ͏ͱ͖ɺΧʔωϧΛΠϯετʔϧ͢Δ n IPython ͷGithub wikiΛࢀর -
Jupyter in use: basics 1/3 26 ¨ ͍ํ: جૅฤ 1/3
¤ ·ͣ׳ΕΑ͏J ¤ ϊʔτϒοΫҰཡը໘ʹ͍ͭͯ n New͔Β։͘/࡞͢Δ n υϥοά&υϩοϓͰΞοϓϩʔυͰ͖Δ n ্ͷλϒʹ͍ͭͯ n Files λϒ: ϊʔτϒοΫͱϑΥϧμҰཡ n Runningλϒ: ࣮ߦதͷϊʔτϒοΫɾλʔϛφϧҰཡ ¤ Ұཡը໘͔Β։͚Δ/࡞Ͱ͖ΔϞϊͷछྨ n ϊʔτϒοΫ (֤ݴޠ) n ςΩετϑΝΠϧ n ϑΥϧμ n λʔϛφϧ -
Jupyter in use: basics 2/3 27 ¨ ͍ํ: جૅฤ 2/3
¤ ϊʔτϒοΫը໘ʹ͍ͭͯ n File > New͔Β৽ن࡞Ͱ͖Δ n File > Make a CopyͰίϐʔͰ͖Δ n File > RevertΑΓݹ͍όʔδϣϯ(ࣗಈอଘ)ʹͤΔ n File > Download asͰΤΫεϙʔτͰ͖Δ n IPython Notebook n Python n HTML n Markdown n reST n PDF -
Jupyter in use: basics 3/3 28 ¨ ͍ํ: جૅฤ 2/3
¤ ηϧͷ࡞ n +Ͱ৽نηϧΛͭ͘ΕΔ n ηϧલޙͰ͖Δ (↑/↓Ϙλϯ) n InsertͰηϧΛૠೖͰ͖Δ ¤ ηϧͷ࣮ߦ n Shin + Enter ͰηϧΛ࣮ߦ n CellϝχϡʔΑΓҰׅ࣮ߦͰ͖Δ n Run All, Run Above, Run Below n Cellϝχϡʔ͔ΒCell typeΛม͑ΒΕΔ n Code, Markdown, Heading(ݟग़͠) n Cellϝχϡʔ͔ΒOutputͷදࣔ/ඇදࣔΛtoggleͰ͖Δ -
Jupyter in use: config 29 ¨ ͍ํ: ઃఆฤ ¤ Jupyterͷઃఆ
n $ jupyter notebook --generate-config Ͱ࡞Δ n ~/.jupyter/ ʹอଘ͞ΕΔ n jupyter_notebook_config.py: n ύεϫʔυͷ༗ແɺHTTPSͰ͏ূ໌ॻͳͲ ¤ IPython (Χʔωϧ)ͷઃఆ n $ ipython profile create Ͱ࡞ n ~/.ipython/profile_XXX/ ʹอଘ͞ΕΔ n ipython_config.py: n IPythonͷઃఆ (λʔϛφϧ/ϊʔτϒοΫڞ௨) n ىಈ࣌ʹ࣮ߦ͢ΔίʔυɺgreedyͳิΛ༗ޮʹ͢Δ͔ͳͲ n ipython_kernel_config.py n IPython Notebookͷઃఆ (ϊʔτϒοΫͷΈద༻) n ىಈ࣌ʹ࣮ߦ͢Δίʔυɺάϥϑඳࣸʹ͍ͪΔόοΫΤϯυ ͳͲɻ -
Jupyter in use: config 30 ¨ ͍ํ: ઃఆฤ (άϥϑඳࣸʹ͍ͭͯิ) ¤
ipython_kernel_config.py ʹ͍ͭͯ n άϥϑඳࣸͷઃఆ߲ҎԼͷΑ͏ʹ͢Δ n c.InlineBackend.figure_formats = set([‘rePna’]) n RePnaղ૾ͷPNGΛੜ͢Δ߹rePna n Png: ී௨ͷղ૾ͷPNG n svgpdfͳͲ͑Δ n c.InlineBackend.rc = {} n ͜ͷߦ͕ͳ͍ͱmatplotlibrcͷઃఆͷ͍͔͕ͭ͘ແࢹ͞ΕΔ (IPython෦σϑΥϧτʹ͓͖͔͑ΒΕΔ) ¤ Matplotlibͷઃఆ n hxp://matplotlib.org/users/customizing.html ͔Β matplotlibrcΛμϯϩʔυ͢Δ n ~/.config/matplotlib/matplotlibrc (Linux) n ~/.matplotlib/matplotlibrc (LinuxҎ֎) n σϑΥϧτͷαΠζɺελΠϧɺTeXΛ༗ޮʹ͢Δ͔ɺͳͲ -
Jupyter in use: common libraries 31 ¨ ͍͔ͨ: Α͘͏ϥΠϒϥϦ ¤
Կͱ͋Εnumpy, scipy, matplotlibඞਢ n ଞͷϥΠϒϥϦͷجૅ ¤ Numpy: CͰॻ͔ΕͨߴͳܭࢉϥΠϒϥϦ ¤ Scipy: ՊֶܭࢉϥΠϒϥϦ ¤ Matplotlib: άϥϑඳࣸϥΠϒϥϦ ¨ Note: ¤ MacͰhomebrewͷtap(python)͔ΒೖΕΑ͏ -
Juputer in use: ploKng 32 ¨ ͍ํ: άϥϑΛඳࣸ͠Α͏ ¤ %pylab
inline ͱଧͱ͏ n %Ͱ࢝·Δͷ magic command ͱݺΕΔ n Matplotlib͕άϩʔόϧʹΠϯϙʔτ͞ΕΔ n ओྲྀͰɺศརɻͨͩ͠ΠϚΠνͱݴ͏ਓ͍Δɻ n ඞਢͰͳ͍ɻ import matplotlib͠ɺmatplotlib.plot()ͯ͠ྑ͍ n MATLAB෩ΠϯλϑΣΠε ¤ plot, grid, legend, xPtle, yPtle, xaxis, yaxis… Ͱ άϥϑΛ࡞Δ n GnuplotMATLABͱಉ͡ཁྖͰάϥϑΛ࡞Δ ¤ ϥϕϧʹ(La)TeXར༻Ͱ͖Δ -
Jupyter in use: ploKng nicely 33 ¨ ͍ํ: ͖Ε͍ͳάϥϑΛඳࣸ͠Α͏ ¤
ग़ྗΛRePnaͳPNGɺ(E)PSɺPDFɺSVGʹͰ͖Δ n Configʹઃఆ͢Δ͔ɺsave(ϑΝΠϧ໊)ͰมߋՄ ¤ mpld3Λ͏ͱɺग़ྗ͕D3.jsΛ͏Α͏ʹͳΔ n mpld3.display() Λ࠷ޙʹݺͿ͚ͩ ¤ prexyplotlibΛ͏ͱඳ͕͍͍ࣸײ͡ʹ៉ྷʹͳΔ n σϑΥϧτͷςʔϚͪΐͬͱݹ͍͘͞ͷͰ -
Jupyter in use: interacPng w/ web 34 ¨ ͍ํ: ΣϒΛऔΓѻ͓͏
¤ requests, beauPfulsoup4 (pyquery)͋ͨΓ͕ศར ¤ requests: hxpϥΠϒϥϦ n ඪ४ͷurllibͬͯΔͷͰɺ͍ͭ͜Λ͓͏ n ηογϣϯΦϒδΣΫτΛ͏ͱCookieѻ͑Δ ¤ BeauPfulSoup4: (X)HTMLύʔα n ͬͨHTMLͰ͍͍ײ͡ʹಡΊΔ n CSSηϨΫλ͕͑Δ ¤ pyquery: ্ಉɻྲྀߦΓͩͯ͠Δʁ -
Jupyter in use: storing data 35 ¨ ͍ํ: σʔλΛอଘ͠Α͏ ¤
pymongo͕ศར ¤ pymongo: MongodbΛѻ͏ϥΠϒϥϦ n Mongodb: JSONΦϒδΣΫτΛόΧεΧͿͪࠐΊΔDB n Կߟ͑ͳͯ͘ྑ͍ΦϒδΣΫτετϨʔδ n εΩʔϚແ͠ɺSQLͳ͠ n Կ͔ԋࢉ͔ͤͨͬͨ͞ΒJavaScript͕ΒͤΒΕΔ -
Jupyter in use: data analysis 36 ¨ ͍ํ: σʔλΛੳ͠Α͏ ¤
pandas, scikit-learn, scikit-image, pillow, nltkͳͲΛ͏ ¤ Pandas: σʔλϑϨʔϜϫʔΫɻ n σʔλೖग़ྗ͕Ͱ͖Δ (CSV, JSON, Excel, HDF5, …) n σʔλͷूܭͳͲ͕Ͱ͖Δ ¤ Scikit-learn: ػցֶशϥΠϒϥϦ ¤ Scikit-image: ্ͷը૾൛ ¤ Pillow: ը૾ॲཧϥΠϒϥϦ ¤ NLTK: ࣗવݴޠϑϨʔϜϫʔΫ ¨ E.g. ¤ hxp://ipython-books.github.io/featured-04/ -
Jupyter in use: symbolic computaPon 37 ¨ ͍ํ: ࣜॲཧΛ͠Α͏ ¤
Sympy͕ྑ͍ ¤ Sympy: ࣜॲཧϥΠϒϥϦ n ࣜΛදݱ͠ɺҼղඍੵͳͲ͕Ͱ͖Δ n ߦྻɾΈ߹ΘͤɾزԿֶͷॲཧͰ͖Δ -
And more… 38 ¨ ͓·͚ ¤ ઌLIGOʹΑΔॏྗݕग़ʹ͍ͭͯɺ ৴߸ॲཧͷҰͷϊʔτϒοΫ͕ެ։͞Ε͍ͯΔɻ ¤ Ref.
hxps://losc.ligo.org/s/events/GW150914/ GW150914_tutorial.html -
References 41 ¨ Jupyter ¤ hxp://jupyter.org/ ¨ IPython ¤ hxp://ipython.org/
¨ Available kernels in Jupyter ¤ hxps://github.com/ipython/ipython/wiki/IPython- kernels-for-other-languages -
How to Install 42 ¨ PipΛ͏ ¤ ύοέʔδϚωʔδϟ ¤ c.f.
Easy_install n ͍ͭ͜ΞϯΠϯετʔϧग़དྷͳ͍ܗͰ Πϯετʔϧ͢ΔͷͰΊΑ͏ɻ ¤ pip install –user ͱ͢Δͱɺ~/.localҎԼʹΠϯετʔϧ Ͱ͖Δ n ڞ༻ͷܭࢉڥglobalΛԚͨ͘͠ͳ͍߹ศར ¨ $ pip install jupyter ¤ JupyterΛΠϯετʔϧ͢Δ ¨ $ pip install “ipython[all]” ¤ IPythonΛΠϯετʔϧ͢Δ -
How to install (for Mac) 43 ¨ Macͷ߹… ¤ HomebrewʹPython
tap͕͋ΔͷͰɺ ઌʹpython, numpy, scipy, matplotlibΛΠϯετʔϧ͢Δ ¤ $ brew tap homebrew/python ¤ $ brew install python n ˞Python3ͷ߹ brew install python3 ¤ $ brew install numpy scipy n ˞Python3ͷ߹ brew install numpy --with-python3 n ˞scipyಉ༷ʹ ¤ $ brew install matplotlib n --with-tex Λ͚ͭͨ΄͏͕ྑ͍(TeXαϙʔτ͕༗ޮʹͳΔ) -
How to install (for Windows) 44 ¨ Windowsͷ߹… ¤ AnacondaEPDΛ͏ख͕͋Δ
n PythonͷdistribuPon n ಠࣗͷύοέʔδγεςϜΛ࣋ͭɻ ¤ Python, numpy, scipy, matplotlibͷ ެࣜͷόΠφϦΠϯετʔϥ͋Δ͕ɺ ࣗྗϝϯς͢Δͷ͠ΜͲ͍ -
Folder structure (installaPon, profile) 45 ¨ Profile folder structure ¤
~/.jupyter n JupyterͷઃఆϓϩϑΝΠϧ͕֨ೲ͞ΕΔ n IPython Χʔωϧͷઃఆ͜͜ʹແ͍ (4.0ʙ) ¤ ~/.ipython n IPythonͷઃఆ͕֨ೲ͞ΕΔ ¤ ~/.matplotlib n Matplotlibrc͕֨ೲ͞ΕΔ ¨ InstallaPon folder structure (Ұ෦) ¤ /usr/local/lib/pythonX.Y/site-packages n (Mac/Library/Python/2.7/site-packages) ¤ ~/.local/lib/python2.7/site-packages n (Mac~/Library/Python/2.7/lib/python/site-packages)