Practical Data Science with Python and Jupyter

Slide 1

Slide 1 text

@WillingCarol Practical Data Science with Python and Jupyter Carol Willing Python Brasil October 25, 2019

Slide 2

Slide 2 text

@WillingCarol Hi! I'm Carol. • Python Steering Council • Core Developer • PSF Fellow • PSF Director (Former) • Frank Willison Award 2019 2

Slide 3

Slide 3 text

@WillingCarol 3 Core maintainer Papermill, Scrapbook, Bookstore, Commuter Steering Council, Core Developer JupyterHub, BinderHub, mybinder.org I love creating tools which educate and empower people.

Slide 4

Slide 4 text

@WillingCarol Practical Data Science 4

Slide 5

Slide 5 text

@WillingCarol 5 Using data responsibly to solve real world issues and improve human lives

Slide 6

Slide 6 text

@WillingCarol 6 A Story Python Jupyter Practical Data Science

Slide 7

Slide 7 text

@WillingCarol 7 San Diego, CA

Slide 8

Slide 8 text

@WillingCarol 8

Slide 9

Slide 9 text

@WillingCarol 9

Slide 10

Slide 10 text

@WillingCarol 10 Tokyo

Slide 11

Slide 11 text

@WillingCarol 11

Slide 12

Slide 12 text

Copyright: 2019 European Union, contains modiﬁed Copernicus Sentinel data 2019, processed by EUMETSAT Super Typhoon Hagibis View of Super Typhoon Hagibis south-west of Japan, as captured by the Copernicus Sentinel-3 satellite on 08 October at 00:16 UTC.

Slide 13

Slide 13 text

Title Typhoon Hagibis Released 10/10/2019 4:45 pm Copyright contains modiﬁed Copernicus Sentinel data (2019), processed by ESA, CC BY-SA 3.0 IGO

Slide 14

Slide 14 text

Source:Twitter

Slide 15

Slide 15 text

A sign is partially submerged as the Tama River ﬂoods during Typhoon Hagibis. Source:Getty Images Source:Japan Times

Slide 16

Slide 16 text

@WillingCarol 16

Slide 17

Slide 17 text

@WillingCarol 17

Slide 18

Slide 18 text

@WillingCarol Preparation Evacuation Safety 18

Slide 19

Slide 19 text

Title: Typhoon Bualoi 10/24/2019 Source: JTWC/SATOPS

Slide 20

Slide 20 text

@WillingCarol 20 Python

Slide 21

Slide 21 text

@WillingCarol Practical data science grew up around

Slide 22

Slide 22 text

@WillingCarol Python - 2019 Governance Python 3.8 Farewell to Python 2 22

Slide 23

Slide 23 text

@WillingCarol 23 Python's Growth in 2019 Web DevOps Systems Testing Science Data Science Embedded systems Education Micropython, CircuitPython, Raspberry Pi

Slide 24

Slide 24 text

@WillingCarol Governance Post-BDFL 24

Slide 25

Slide 25 text

@WillingCarol PEP 8016 The Steering Council Model 25

Slide 26

Slide 26 text

@WillingCarol Steering Council 26 Barry Warsaw Brett Cannon Carol Willing Guido van Rossum Nick Coghlan

Slide 27

Slide 27 text

@WillingCarol PEP 13 Python Language Governance 27 ‣ Quality and Stability ‣ Contributing accessible, inclusive, sustainable ‣ Core team and PSF relationship ‣ Decision making processes for PEPs ‣ Seek consensus

Slide 28

Slide 28 text

@WillingCarol Core Development 28

Slide 29

Slide 29 text

@WillingCarol Python 3.8 29 https://www.python.org/downloads/ https://www.python.org/dev/peps/pep-0569/ Schedule and Content Try it

Slide 30

Slide 30 text

@WillingCarol 30 Positional only parameters Assignment operator (walrus operator) "equals" speciﬁer added to f-string (debugging) PEP 587: Python Initialization Conﬁguration PEP 574: pickle protocol 5 Python 3.8

Slide 31

Slide 31 text

@WillingCarol 31

Slide 32

Slide 32 text

@WillingCarol 32

Slide 33

Slide 33 text

@WillingCarol Farewell to Python 2 33

Slide 34

Slide 34 text

@WillingCarol 34 Jupyter

Slide 35

Slide 35 text

@WillingCarol 35 jupyter.org

Slide 36

Slide 36 text

@WillingCarol 36 2014 Now, 5 years later... 36

Slide 37

Slide 37 text

  Millions of Notebooks https://github.com/trending/jupyter-notebook Over 5 million on GitHub

Slide 38

Slide 38 text

jupyter.org

Slide 39

Slide 39 text

@WillingCarol 39 ‣ Growth ‣ ACM Award ‣ Industry adoption ‣ Creative uses ‣ Open Source Book https://www.youtube.com/watch?v=qbtDVdEr8SY

Slide 40

Slide 40 text

@WillingCarol JupyterLab 40

Slide 41

Slide 41 text

@WillingCarol ecosystem 41

Slide 42

Slide 42 text

@WillingCarol 42 Practical Data Science Source: xkcd

Slide 43

Slide 43 text

@WillingCarol 43 Start Deploy Explore Practical Data Science

Slide 44

Slide 44 text

@WillingCarol Start 44 Step 1

Slide 45

Slide 45 text

@WillingCarol 45 Binder 2.0 blog post elifesciences: Share your interactive research environment Nature article about Binder 45 mybinder.org Try it in the browser

Slide 46

Slide 46 text

46 jupyter.org demo

Slide 47

Slide 47 text

47 jupyter.org demo

Slide 48

Slide 48 text

@WillingCarol Install Python 48 Use the OS Package Manager Use a Distribution - Anaconda, miniconda Download from Python.org

Slide 49

Slide 49 text

@WillingCarol Install Libraries 49 Use a Distribution such as Anaconda Use pip Use miniconda, conda, and conda-forge

Slide 50

Slide 50 text

@WillingCarol 50 python3 -m venv myenv source myenv/bin/activate pip install jupyter jupyter notebook

Slide 51

Slide 51 text

@WillingCarol 51 conda create -n myenv python=3.7 conda activate myenv conda install jupyter -c conda-forge jupyter notebook

Slide 52

Slide 52 text

@WillingCarol Choose your Tools 52

Slide 53

Slide 53 text

@WillingCarol JupyterLab 53

Slide 54

Slide 54 text

@WillingCarol 54 ReactJS front end nteract nteract.io

Slide 55

Slide 55 text

@WillingCarol VS Code 55 PyCharm

Slide 56

Slide 56 text

@WillingCarol Start 56 Try it in the browser Install Python Install Libraries Choose your tools Step 1

Slide 57

Slide 57 text

@WillingCarol Explore 57 Step 2

Slide 58

Slide 58 text

@WillingCarol Start with your interests I usually create music projects when exploring new languages. 58 https://web.mit.edu/music21/

Slide 59

Slide 59 text

@WillingCarol 59 Try a library's tutorial

Slide 60

Slide 60 text

@WillingCarol 60

Slide 61

Slide 61 text

@WillingCarol 61 ipyvolume https://towardsdatascience.com/multivolume- rendering-in-jupyter-with-ipyvolume-cross- language-3d-visualization-64389047634a

Slide 62

Slide 62 text

@WillingCarol 62

Slide 63

Slide 63 text

@WillingCarol 63 github.com/napari/napari napari https://ilovesymposia.com/2019/10/24/introducing-napari- a-fast-n-dimensional-image-viewer-in-python/

Slide 64

Slide 64 text

@WillingCarol Learning resources 64 Online Books with Jupyter Notebooks Conferences, pyvideo.org and YouTube Research papers, blogs Source: O'Reilly

Slide 65

Slide 65 text

@WillingCarol 65 Ten Simple Rules for Reproducible Research in Jupyter Notebooks Adam Rule et al. https://github.com/jupyter-guide/ ten-rules-jupyter https://github.com/jupyter-guide/ jupyter-guide

Slide 66

Slide 66 text

@WillingCarol Learn AI online 66

Slide 67

Slide 67 text

@WillingCarol Community 67 User groups Meetups PyLadies Carpentries Photo: Python Sul

Slide 68

Slide 68 text

@WillingCarol What's new 68 Talk Python to Me Tracking Jupyter Newsletter https://tinyletter.com/TrackingJupyter/archive Open Source Directions GitHub Trending Follow projects on Social Media

Slide 69

Slide 69 text

@WillingCarol Explore 69 Start with your interests Try a tutorial Learning resources Community What's new Step 2

Slide 70

Slide 70 text

@WillingCarol Deploy 70 Step 3

Slide 71

Slide 71 text

@WillingCarol Notebooks to web 71 https://blog.jupyter.org/and- voil%C3%A0-f6a2c08a4a93

Slide 72

Slide 72 text

@WillingCarol 72 nteract Papermill Scrapbook Bookstore Commuter Production data at scale 72 https://medium.com/netﬂix-techblog/notebook-innovation-591ee3221233 Bookstore

Slide 73

Slide 73 text

@WillingCarol 73 Papermill - parameterize / run Scrapbook - recording / reading Bookstore - store notebooks Commuter - share notebooks Production data at scale 73

Slide 74

Slide 74 text

@WillingCarol 74 Enterprise data workﬂows 74

Slide 75

Slide 75 text

@WillingCarol 75 https://medium.com/dagster-io/dagster-0-6-0-impossible-princess-898b459375e0 Pipelines

Slide 76

Slide 76 text

zero-to-jupyterhub.readthedocs.io

Slide 77

Slide 77 text

https://blog.jupyter.org/simpler-authentication-for-small- scale-jupyterhubs-with-nativeauthenticator-999534c77a09 Native Authenticator Thank you Leticia

Slide 78

Slide 78 text

@WillingCarol 78 78 Deploy binders

Slide 79

Slide 79 text

79 Juliette Taka

Slide 80

Slide 80 text

@WillingCarol 80 From a phone in the park! 80

Slide 81

Slide 81 text

@WillingCarol Pangeo 81 https://pangeo.io

Slide 82

Slide 82 text

@WillingCarol 82

Slide 83

Slide 83 text

@WillingCarol 83 https://simexp.github.io/vcog_hps_ad_book/intro.html Jupyter Book Binder Jupyter pandas scipy scikit learn matplotlib numpy seaborn Canadian Open Neuroscience Platform

Slide 84

Slide 84 text

@WillingCarol 84 Practical Data Science How?

Slide 85

Slide 85 text

@WillingCarol Start 85

Slide 86

Slide 86 text

@WillingCarol Ask why 86

Slide 87

Slide 87 text

@WillingCarol Plan Execute Change 87 https://jupyterhub-team-compass.readthedocs.io https://github.com/jupyterhub/team-compass

Slide 88

Slide 88 text

@WillingCarol Share binders. Foster research. 88

Slide 89

Slide 89 text

@WillingCarol Build Community 89 Python Brasil official photo. The sign represents Python in BSL (Brazilian Sign Language), made by Amanda and Sávio at Python Sul 2018 https://pyfound.blogspot.com/2019/02/python-brasil-people-technology.html PyLadiesBRConf official photo PyLadiesBRConf official photo

Slide 90

Slide 90 text

@WillingCarol 90 Python Brasil Practical Data Science

Slide 91

Slide 91 text

@WillingCarol 91 Python Brasil Changing the world together

Slide 92

Slide 92 text

@WillingCarol 92 Thank you https://speakerdeck.com/willingc @WillingCarol

Slide 93

Slide 93 text

@WillingCarol 93 Join the

Slide 94

Slide 94 text

@WillingCarol 94 Questions https://speakerdeck.com/willingc

Slide 95

Slide 95 text

@WillingCarol 95 Karaoke Time Lorena, This is for you. https://www.youtube.com/watch?v=m41cVv3tChE

Slide 96

Slide 96 text

@WillingCarol 96 Thank you Python Brasil Project Jupyter Team Core Python Team PSF

Slide 97

Slide 97 text

@WillingCarol Attributions 97 Attributions on slides. Photos [7-10] Carol Willing and Linnea Willing [38] The Carpentries, Tracy Teal, Bérénice Batut [17] Godzilla By Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha) © 1954 - movie poster made by Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha), Public Domain, https://commons.wikimedia.org/w/index.php?curid=3648684