Practical Data Science with Python and Jupyter

Practical Data Science with Python and Jupyter

Python Brasil 2019 keynote in Ribeirao Preto. Python and Jupyter provide the tools and libraries for Practical Data Science.

What is Practical Data Science? Practical Data Science uses data responsibly to solve real world issues, such as climate change and disease, and improve human lives.

The keynote gives a state of Python and Jupyter in 2019. It outlines three major steps for Practical Data Science: Start, Explore, Deploy. Weather and climate are used as evidence of the impact and importance of data science predictions.

C8eedb2bca5728f0f73294b5b5a0222e?s=128

Carol Willing

October 25, 2019
Tweet

Transcript

  1. @WillingCarol Practical Data Science with Python and Jupyter Carol Willing

    Python Brasil October 25, 2019
  2. @WillingCarol Hi! I'm Carol. • Python Steering Council • Core

    Developer • PSF Fellow • PSF Director (Former) • Frank Willison Award 2019 2
  3. @WillingCarol 3 Core maintainer Papermill, Scrapbook, Bookstore, Commuter Steering Council,

    Core Developer JupyterHub, BinderHub, mybinder.org I love creating tools which educate and empower people.
  4. @WillingCarol Practical Data Science 4

  5. @WillingCarol 5 Using data responsibly to solve real world issues

    and improve human lives
  6. @WillingCarol 6 A Story Python Jupyter Practical Data Science

  7. @WillingCarol 7 San Diego, CA

  8. @WillingCarol 8

  9. @WillingCarol 9

  10. @WillingCarol 10 Tokyo

  11. @WillingCarol 11

  12. Copyright: 2019 European Union, contains modified Copernicus Sentinel data 2019,

    processed by EUMETSAT Super Typhoon Hagibis View of Super Typhoon Hagibis south-west of Japan, as captured by the Copernicus Sentinel-3 satellite on 08 October at 00:16 UTC.
  13. Title Typhoon Hagibis Released 10/10/2019 4:45 pm Copyright contains modified

    Copernicus Sentinel data (2019), processed by ESA, CC BY-SA 3.0 IGO
  14. Source:Twitter

  15. A sign is partially submerged as the Tama River floods

    during Typhoon Hagibis. Source:Getty Images Source:Japan Times
  16. @WillingCarol 16

  17. @WillingCarol 17

  18. @WillingCarol Preparation Evacuation Safety 18

  19. Title: Typhoon Bualoi 10/24/2019 Source: JTWC/SATOPS

  20. @WillingCarol 20 Python

  21. @WillingCarol Practical data science grew up around

  22. @WillingCarol Python - 2019 Governance Python 3.8 Farewell to Python

    2 22
  23. @WillingCarol 23 Python's Growth in 2019 Web DevOps Systems Testing

    Science Data Science Embedded systems Education Micropython, CircuitPython, Raspberry Pi
  24. @WillingCarol Governance Post-BDFL 24

  25. @WillingCarol PEP 8016 The Steering Council Model 25

  26. @WillingCarol Steering Council 26 Barry Warsaw Brett Cannon Carol Willing

    Guido van Rossum Nick Coghlan
  27. @WillingCarol PEP 13 Python Language Governance 27 ‣ Quality and

    Stability ‣ Contributing accessible, inclusive, sustainable ‣ Core team and PSF relationship ‣ Decision making processes for PEPs ‣ Seek consensus
  28. @WillingCarol Core Development 28

  29. @WillingCarol Python 3.8 29 https://www.python.org/downloads/ https://www.python.org/dev/peps/pep-0569/ Schedule and Content Try

    it
  30. @WillingCarol 30 Positional only parameters Assignment operator (walrus operator) "equals"

    specifier added to f-string (debugging) PEP 587: Python Initialization Configuration PEP 574: pickle protocol 5 Python 3.8
  31. @WillingCarol 31

  32. @WillingCarol 32

  33. @WillingCarol Farewell to Python 2 33

  34. @WillingCarol 34 Jupyter

  35. @WillingCarol 35 jupyter.org

  36. @WillingCarol 36 2014 Now, 5 years later... 36

  37. 
 Millions of Notebooks https://github.com/trending/jupyter-notebook Over 5 million on GitHub

  38. jupyter.org

  39. @WillingCarol 39 ‣ Growth ‣ ACM Award ‣ Industry adoption

    ‣ Creative uses ‣ Open Source Book https://www.youtube.com/watch?v=qbtDVdEr8SY
  40. @WillingCarol JupyterLab 40

  41. @WillingCarol ecosystem 41

  42. @WillingCarol 42 Practical Data Science Source: xkcd

  43. @WillingCarol 43 Start Deploy Explore Practical Data Science

  44. @WillingCarol Start 44 Step 1

  45. @WillingCarol 45 Binder 2.0 blog post elifesciences: Share your interactive

    research environment Nature article about Binder 45 mybinder.org Try it in the browser
  46. 46 jupyter.org demo

  47. 47 jupyter.org demo

  48. @WillingCarol Install Python 48 Use the OS Package Manager Use

    a Distribution - Anaconda, miniconda Download from Python.org
  49. @WillingCarol Install Libraries 49 Use a Distribution such as Anaconda

    Use pip Use miniconda, conda, and conda-forge
  50. @WillingCarol 50 python3 -m venv myenv source myenv/bin/activate pip install

    jupyter jupyter notebook
  51. @WillingCarol 51 conda create -n myenv python=3.7 conda activate myenv

    conda install jupyter -c conda-forge jupyter notebook
  52. @WillingCarol Choose your Tools 52

  53. @WillingCarol JupyterLab 53

  54. @WillingCarol 54 ReactJS front end nteract nteract.io

  55. @WillingCarol VS Code 55 PyCharm

  56. @WillingCarol Start 56 Try it in the browser Install Python

    Install Libraries Choose your tools Step 1
  57. @WillingCarol Explore 57 Step 2

  58. @WillingCarol Start with your interests I usually create music projects

    when exploring new languages. 58 https://web.mit.edu/music21/
  59. @WillingCarol 59 Try a library's tutorial

  60. @WillingCarol 60

  61. @WillingCarol 61 ipyvolume https://towardsdatascience.com/multivolume- rendering-in-jupyter-with-ipyvolume-cross- language-3d-visualization-64389047634a

  62. @WillingCarol 62

  63. @WillingCarol 63 github.com/napari/napari napari https://ilovesymposia.com/2019/10/24/introducing-napari- a-fast-n-dimensional-image-viewer-in-python/

  64. @WillingCarol Learning resources 64 Online Books with Jupyter Notebooks Conferences,

    pyvideo.org and YouTube Research papers, blogs Source: O'Reilly
  65. @WillingCarol 65 Ten Simple Rules for Reproducible Research in Jupyter

    Notebooks Adam Rule et al. https://github.com/jupyter-guide/ ten-rules-jupyter https://github.com/jupyter-guide/ jupyter-guide
  66. @WillingCarol Learn AI online 66

  67. @WillingCarol Community 67 User groups Meetups PyLadies Carpentries Photo: Python

    Sul
  68. @WillingCarol What's new 68 Talk Python to Me Tracking Jupyter

    Newsletter https://tinyletter.com/TrackingJupyter/archive Open Source Directions GitHub Trending Follow projects on Social Media
  69. @WillingCarol Explore 69 Start with your interests Try a tutorial

    Learning resources Community What's new Step 2
  70. @WillingCarol Deploy 70 Step 3

  71. @WillingCarol Notebooks to web 71 https://blog.jupyter.org/and- voil%C3%A0-f6a2c08a4a93

  72. @WillingCarol 72 nteract Papermill Scrapbook Bookstore Commuter Production data at

    scale 72 https://medium.com/netflix-techblog/notebook-innovation-591ee3221233 Bookstore
  73. @WillingCarol 73 Papermill - parameterize / run Scrapbook - recording

    / reading Bookstore - store notebooks Commuter - share notebooks Production data at scale 73
  74. @WillingCarol 74 Enterprise data workflows 74

  75. @WillingCarol 75 https://medium.com/dagster-io/dagster-0-6-0-impossible-princess-898b459375e0 Pipelines

  76. zero-to-jupyterhub.readthedocs.io

  77. https://blog.jupyter.org/simpler-authentication-for-small- scale-jupyterhubs-with-nativeauthenticator-999534c77a09 Native Authenticator Thank you Leticia

  78. @WillingCarol 78 78 Deploy binders

  79. 79 Juliette Taka

  80. @WillingCarol 80 From a phone in the park! 80

  81. @WillingCarol Pangeo 81 https://pangeo.io

  82. @WillingCarol 82

  83. @WillingCarol 83 https://simexp.github.io/vcog_hps_ad_book/intro.html Jupyter Book Binder Jupyter pandas scipy scikit

    learn matplotlib numpy seaborn Canadian Open Neuroscience Platform
  84. @WillingCarol 84 Practical Data Science How?

  85. @WillingCarol Start 85

  86. @WillingCarol Ask why 86

  87. @WillingCarol Plan Execute Change 87 https://jupyterhub-team-compass.readthedocs.io https://github.com/jupyterhub/team-compass

  88. @WillingCarol Share binders. Foster research. 88

  89. @WillingCarol Build Community 89 Python Brasil official photo. The sign

    represents Python in BSL (Brazilian Sign Language), made by Amanda and Sávio at Python Sul 2018 https://pyfound.blogspot.com/2019/02/python-brasil-people-technology.html PyLadiesBRConf official photo PyLadiesBRConf official photo
  90. @WillingCarol 90 Python Brasil Practical Data Science

  91. @WillingCarol 91 Python Brasil Changing the world together

  92. @WillingCarol 92 Thank you https://speakerdeck.com/willingc @WillingCarol

  93. @WillingCarol 93 Join the

  94. @WillingCarol 94 Questions https://speakerdeck.com/willingc

  95. @WillingCarol 95 Karaoke Time Lorena, This is for you. https://www.youtube.com/watch?v=m41cVv3tChE

  96. @WillingCarol 96 Thank you Python Brasil Project Jupyter Team Core

    Python Team PSF
  97. @WillingCarol Attributions 97 Attributions on slides. Photos [7-10] Carol Willing

    and Linnea Willing [38] The Carpentries, Tracy Teal, Bérénice Batut [17] Godzilla By Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha) © 1954 - movie poster made by Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha), Public Domain, https://commons.wikimedia.org/w/index.php?curid=3648684