Practical Data Science with Python and Jupyter

Python Brasil 2019 keynote in Ribeirao Preto. Python and Jupyter provide the tools and libraries for Practical Data Science.

What is Practical Data Science? Practical Data Science uses data responsibly to solve real world issues, such as climate change and disease, and improve human lives.

The keynote gives a state of Python and Jupyter in 2019. It outlines three major steps for Practical Data Science: Start, Explore, Deploy. Weather and climate are used as evidence of the impact and importance of data science predictions.

Transcript

1. @WillingCarol Practical Data Science with Python and Jupyter Carol Willing Python Brasil October 25, 2019
2. @WillingCarol Hi! I'm Carol. • Python Steering Council • Core Developer • PSF Fellow • PSF Director (Former) • Frank Willison Award 2019 2
3. @WillingCarol 3 Core maintainer Papermill, Scrapbook, Bookstore, Commuter Steering Council, Core Developer JupyterHub, BinderHub, mybinder.org I love creating tools which educate and empower people.
4. @WillingCarol Practical Data Science 4
5. @WillingCarol 5 Using data responsibly to solve real world issues and improve human lives
6. @WillingCarol 6 A Story Python Jupyter Practical Data Science
7. @WillingCarol 7 San Diego, CA
8. @WillingCarol 8
9. @WillingCarol 9
10. @WillingCarol 10 Tokyo
11. @WillingCarol 11
12. Copyright: 2019 European Union, contains modified Copernicus Sentinel data 2019, processed by EUMETSAT Super Typhoon Hagibis View of Super Typhoon Hagibis south-west of Japan, as captured by the Copernicus Sentinel-3 satellite on 08 October at 00:16 UTC.
13. Title Typhoon Hagibis Released 10/10/2019 4:45 pm Copyright contains modified Copernicus Sentinel data (2019), processed by ESA, CC BY-SA 3.0 IGO
14. Source:Twitter
15. A sign is partially submerged as the Tama River floods during Typhoon Hagibis. Source:Getty Images Source:Japan Times
16. @WillingCarol 16
17. @WillingCarol 17
18. @WillingCarol Preparation Evacuation Safety 18
19. Title: Typhoon Bualoi 10/24/2019 Source: JTWC/SATOPS
20. @WillingCarol 20 Python
21. @WillingCarol Practical data science grew up around
22. @WillingCarol Python - 2019 Governance Python 3.8 Farewell to Python 2 22
23. @WillingCarol 23 Python's Growth in 2019 Web DevOps Systems Testing Science Data Science Embedded systems Education Micropython, CircuitPython, Raspberry Pi
24. @WillingCarol Governance Post-BDFL 24
25. @WillingCarol PEP 8016 The Steering Council Model 25
26. @WillingCarol Steering Council 26 Barry Warsaw Brett Cannon Carol Willing Guido van Rossum Nick Coghlan
27. @WillingCarol PEP 13 Python Language Governance 27 ‣ Quality and Stability ‣ Contributing accessible, inclusive, sustainable ‣ Core team and PSF relationship ‣ Decision making processes for PEPs ‣ Seek consensus
28. @WillingCarol Core Development 28
29. @WillingCarol Python 3.8 29 https://www.python.org/downloads/ https://www.python.org/dev/peps/pep-0569/ Schedule and Content Try it
30. @WillingCarol 30 Positional only parameters Assignment operator (walrus operator) "equals" specifier added to f-string (debugging) PEP 587: Python Initialization Configuration PEP 574: pickle protocol 5 Python 3.8
31. @WillingCarol 31
32. @WillingCarol 32
33. @WillingCarol Farewell to Python 2 33
34. @WillingCarol 34 Jupyter
35. @WillingCarol 35 jupyter.org
36. @WillingCarol 36 2014 Now, 5 years later... 36
37. 
 Millions of Notebooks https://github.com/trending/jupyter-notebook Over 5 million on GitHub
38. jupyter.org
39. @WillingCarol 39 ‣ Growth ‣ ACM Award ‣ Industry adoption ‣ Creative uses ‣ Open Source Book https://www.youtube.com/watch?v=qbtDVdEr8SY
40. @WillingCarol JupyterLab 40
41. @WillingCarol ecosystem 41
42. @WillingCarol 42 Practical Data Science Source: xkcd
43. @WillingCarol 43 Start Deploy Explore Practical Data Science
44. @WillingCarol Start 44 Step 1
45. @WillingCarol 45 Binder 2.0 blog post elifesciences: Share your interactive research environment Nature article about Binder 45 mybinder.org Try it in the browser
46. 46 jupyter.org demo
47. 47 jupyter.org demo
48. @WillingCarol Install Python 48 Use the OS Package Manager Use a Distribution - Anaconda, miniconda Download from Python.org
49. @WillingCarol Install Libraries 49 Use a Distribution such as Anaconda Use pip Use miniconda, conda, and conda-forge
50. @WillingCarol 50 python3 -m venv myenv source myenv/bin/activate pip install jupyter jupyter notebook
51. @WillingCarol 51 conda create -n myenv python=3.7 conda activate myenv conda install jupyter -c conda-forge jupyter notebook
52. @WillingCarol Choose your Tools 52
53. @WillingCarol JupyterLab 53
54. @WillingCarol 54 ReactJS front end nteract nteract.io
55. @WillingCarol VS Code 55 PyCharm
56. @WillingCarol Start 56 Try it in the browser Install Python Install Libraries Choose your tools Step 1
57. @WillingCarol Explore 57 Step 2
58. @WillingCarol Start with your interests I usually create music projects when exploring new languages. 58 https://web.mit.edu/music21/
59. @WillingCarol 59 Try a library's tutorial
60. @WillingCarol 60
61. @WillingCarol 61 ipyvolume https://towardsdatascience.com/multivolume- rendering-in-jupyter-with-ipyvolume-cross- language-3d-visualization-64389047634a
62. @WillingCarol 62
63. @WillingCarol 63 github.com/napari/napari napari https://ilovesymposia.com/2019/10/24/introducing-napari- a-fast-n-dimensional-image-viewer-in-python/
64. @WillingCarol Learning resources 64 Online Books with Jupyter Notebooks Conferences, pyvideo.org and YouTube Research papers, blogs Source: O'Reilly
65. @WillingCarol 65 Ten Simple Rules for Reproducible Research in Jupyter Notebooks Adam Rule et al. https://github.com/jupyter-guide/ ten-rules-jupyter https://github.com/jupyter-guide/ jupyter-guide
66. @WillingCarol Learn AI online 66
67. @WillingCarol Community 67 User groups Meetups PyLadies Carpentries Photo: Python Sul
68. @WillingCarol What's new 68 Talk Python to Me Tracking Jupyter Newsletter https://tinyletter.com/TrackingJupyter/archive Open Source Directions GitHub Trending Follow projects on Social Media
69. @WillingCarol Explore 69 Start with your interests Try a tutorial Learning resources Community What's new Step 2
70. @WillingCarol Deploy 70 Step 3
71. @WillingCarol Notebooks to web 71 https://blog.jupyter.org/and- voil%C3%A0-f6a2c08a4a93
72. @WillingCarol 72 nteract Papermill Scrapbook Bookstore Commuter Production data at scale 72 https://medium.com/netflix-techblog/notebook-innovation-591ee3221233 Bookstore
73. @WillingCarol 73 Papermill - parameterize / run Scrapbook - recording / reading Bookstore - store notebooks Commuter - share notebooks Production data at scale 73
74. @WillingCarol 74 Enterprise data workflows 74
75. @WillingCarol 75 https://medium.com/dagster-io/dagster-0-6-0-impossible-princess-898b459375e0 Pipelines
76. zero-to-jupyterhub.readthedocs.io
77. https://blog.jupyter.org/simpler-authentication-for-small- scale-jupyterhubs-with-nativeauthenticator-999534c77a09 Native Authenticator Thank you Leticia
78. @WillingCarol 78 78 Deploy binders
79. 79 Juliette Taka
80. @WillingCarol 80 From a phone in the park! 80
81. @WillingCarol Pangeo 81 https://pangeo.io
82. @WillingCarol 82
83. @WillingCarol 83 https://simexp.github.io/vcog_hps_ad_book/intro.html Jupyter Book Binder Jupyter pandas scipy scikit learn matplotlib numpy seaborn Canadian Open Neuroscience Platform
84. @WillingCarol 84 Practical Data Science How?
85. @WillingCarol Start 85
86. @WillingCarol Ask why 86
87. @WillingCarol Plan Execute Change 87 https://jupyterhub-team-compass.readthedocs.io https://github.com/jupyterhub/team-compass
88. @WillingCarol Share binders. Foster research. 88
89. @WillingCarol Build Community 89 Python Brasil official photo. The sign represents Python in BSL (Brazilian Sign Language), made by Amanda and Sávio at Python Sul 2018 https://pyfound.blogspot.com/2019/02/python-brasil-people-technology.html PyLadiesBRConf official photo PyLadiesBRConf official photo
90. @WillingCarol 90 Python Brasil Practical Data Science
91. @WillingCarol 91 Python Brasil Changing the world together
92. @WillingCarol 92 Thank you https://speakerdeck.com/willingc @WillingCarol
93. @WillingCarol 93 Join the
94. @WillingCarol 94 Questions https://speakerdeck.com/willingc
95. @WillingCarol 95 Karaoke Time Lorena, This is for you. https://www.youtube.com/watch?v=m41cVv3tChE
96. @WillingCarol 96 Thank you Python Brasil Project Jupyter Team Core Python Team PSF
97. @WillingCarol Attributions 97 Attributions on slides. Photos [7-10] Carol Willing and Linnea Willing [38] The Carpentries, Tracy Teal, Bérénice Batut [17] Godzilla By Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha) © 1954 - movie poster made by Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha), Public Domain, https://commons.wikimedia.org/w/index.php?curid=3648684