Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical Data Science with Python and Jupyter

Practical Data Science with Python and Jupyter

Python Brasil 2019 keynote in Ribeirao Preto. Python and Jupyter provide the tools and libraries for Practical Data Science.

What is Practical Data Science? Practical Data Science uses data responsibly to solve real world issues, such as climate change and disease, and improve human lives.

The keynote gives a state of Python and Jupyter in 2019. It outlines three major steps for Practical Data Science: Start, Explore, Deploy. Weather and climate are used as evidence of the impact and importance of data science predictions.

Carol Willing

October 25, 2019
Tweet

More Decks by Carol Willing

Other Decks in Technology

Transcript

  1. @WillingCarol
    Practical Data Science with
    Python and Jupyter
    Carol Willing
    Python Brasil
    October 25, 2019

    View full-size slide

  2. @WillingCarol
    Hi! I'm Carol.
    • Python Steering Council
    • Core Developer
    • PSF Fellow
    • PSF Director (Former)
    • Frank Willison Award 2019
    2

    View full-size slide

  3. @WillingCarol 3
    Core maintainer
    Papermill, Scrapbook, Bookstore, Commuter
    Steering Council, Core Developer
    JupyterHub, BinderHub, mybinder.org
    I love creating tools which
    educate and empower
    people.

    View full-size slide

  4. @WillingCarol
    Practical
    Data Science
    4

    View full-size slide

  5. @WillingCarol 5
    Using data responsibly to
    solve real world issues and
    improve human lives

    View full-size slide

  6. @WillingCarol 6
    A Story
    Python
    Jupyter
    Practical
    Data Science

    View full-size slide

  7. @WillingCarol 7
    San Diego, CA

    View full-size slide

  8. @WillingCarol 8

    View full-size slide

  9. @WillingCarol 9

    View full-size slide

  10. @WillingCarol 10
    Tokyo

    View full-size slide

  11. @WillingCarol 11

    View full-size slide

  12. Copyright: 2019 European
    Union, contains modified
    Copernicus Sentinel data
    2019, processed by
    EUMETSAT
    Super Typhoon
    Hagibis
    View of Super Typhoon
    Hagibis south-west of
    Japan, as captured by
    the Copernicus
    Sentinel-3 satellite on
    08 October at 00:16
    UTC.

    View full-size slide

  13. Title Typhoon Hagibis
    Released 10/10/2019 4:45 pm
    Copyright contains modified Copernicus
    Sentinel data (2019), processed by ESA,
    CC BY-SA 3.0 IGO

    View full-size slide

  14. Source:Twitter

    View full-size slide

  15. A sign is partially submerged as the Tama River floods during Typhoon Hagibis.
    Source:Getty Images
    Source:Japan Times

    View full-size slide

  16. @WillingCarol 16

    View full-size slide

  17. @WillingCarol 17

    View full-size slide

  18. @WillingCarol
    Preparation
    Evacuation
    Safety
    18

    View full-size slide

  19. Title: Typhoon Bualoi
    10/24/2019
    Source: JTWC/SATOPS

    View full-size slide

  20. @WillingCarol 20
    Python

    View full-size slide

  21. @WillingCarol
    Practical data science
    grew up around

    View full-size slide

  22. @WillingCarol
    Python - 2019
    Governance
    Python 3.8
    Farewell to Python 2
    22

    View full-size slide

  23. @WillingCarol 23
    Python's Growth in 2019
    Web
    DevOps
    Systems
    Testing
    Science
    Data Science
    Embedded systems
    Education
    Micropython,
    CircuitPython,
    Raspberry Pi

    View full-size slide

  24. @WillingCarol
    Governance
    Post-BDFL
    24

    View full-size slide

  25. @WillingCarol
    PEP 8016
    The Steering Council Model
    25

    View full-size slide

  26. @WillingCarol
    Steering Council
    26
    Barry Warsaw
    Brett Cannon
    Carol Willing
    Guido van Rossum
    Nick Coghlan

    View full-size slide

  27. @WillingCarol
    PEP 13 Python Language Governance
    27
    ‣ Quality and Stability
    ‣ Contributing accessible, inclusive, sustainable
    ‣ Core team and PSF relationship
    ‣ Decision making processes for PEPs
    ‣ Seek consensus

    View full-size slide

  28. @WillingCarol
    Core Development
    28

    View full-size slide

  29. @WillingCarol
    Python 3.8
    29
    https://www.python.org/downloads/
    https://www.python.org/dev/peps/pep-0569/
    Schedule and Content
    Try it

    View full-size slide

  30. @WillingCarol 30
    Positional only parameters
    Assignment operator (walrus operator)
    "equals" specifier added to f-string (debugging)
    PEP 587: Python Initialization Configuration
    PEP 574: pickle protocol 5
    Python 3.8

    View full-size slide

  31. @WillingCarol 31

    View full-size slide

  32. @WillingCarol 32

    View full-size slide

  33. @WillingCarol
    Farewell to Python 2
    33

    View full-size slide

  34. @WillingCarol 34
    Jupyter

    View full-size slide

  35. @WillingCarol 35
    jupyter.org

    View full-size slide

  36. @WillingCarol 36
    2014
    Now,
    5 years
    later...
    36

    View full-size slide


  37. Millions of
    Notebooks
    https://github.com/trending/jupyter-notebook
    Over 5 million
    on GitHub

    View full-size slide

  38. @WillingCarol 39
    ‣ Growth
    ‣ ACM Award
    ‣ Industry adoption
    ‣ Creative uses
    ‣ Open Source Book
    https://www.youtube.com/watch?v=qbtDVdEr8SY

    View full-size slide

  39. @WillingCarol
    JupyterLab
    40

    View full-size slide

  40. @WillingCarol
    ecosystem
    41

    View full-size slide

  41. @WillingCarol 42
    Practical Data Science
    Source: xkcd

    View full-size slide

  42. @WillingCarol 43
    Start
    Deploy
    Explore
    Practical Data Science

    View full-size slide

  43. @WillingCarol
    Start
    44
    Step 1

    View full-size slide

  44. @WillingCarol 45
    Binder 2.0 blog post
    elifesciences: Share
    your interactive
    research environment
    Nature article about
    Binder
    45
    mybinder.org
    Try it in the
    browser

    View full-size slide

  45. 46
    jupyter.org demo

    View full-size slide

  46. 47
    jupyter.org demo

    View full-size slide

  47. @WillingCarol
    Install Python
    48
    Use the OS Package Manager
    Use a Distribution - Anaconda, miniconda
    Download from Python.org

    View full-size slide

  48. @WillingCarol
    Install Libraries
    49
    Use a Distribution
    such as Anaconda
    Use pip
    Use miniconda, conda,
    and conda-forge

    View full-size slide

  49. @WillingCarol 50
    python3 -m venv myenv
    source myenv/bin/activate
    pip install jupyter
    jupyter notebook

    View full-size slide

  50. @WillingCarol 51
    conda create -n myenv python=3.7
    conda activate myenv
    conda install jupyter -c conda-forge
    jupyter notebook

    View full-size slide

  51. @WillingCarol
    Choose your Tools
    52

    View full-size slide

  52. @WillingCarol
    JupyterLab
    53

    View full-size slide

  53. @WillingCarol 54
    ReactJS front end
    nteract
    nteract.io

    View full-size slide

  54. @WillingCarol
    VS Code
    55
    PyCharm

    View full-size slide

  55. @WillingCarol
    Start
    56
    Try it in the browser
    Install Python
    Install Libraries
    Choose your tools
    Step 1

    View full-size slide

  56. @WillingCarol
    Explore
    57
    Step 2

    View full-size slide

  57. @WillingCarol
    Start with your interests
    I usually create music
    projects when exploring new
    languages.
    58
    https://web.mit.edu/music21/

    View full-size slide

  58. @WillingCarol 59
    Try a library's tutorial

    View full-size slide

  59. @WillingCarol 60

    View full-size slide

  60. @WillingCarol 61
    ipyvolume
    https://towardsdatascience.com/multivolume-
    rendering-in-jupyter-with-ipyvolume-cross-
    language-3d-visualization-64389047634a

    View full-size slide

  61. @WillingCarol 62

    View full-size slide

  62. @WillingCarol 63
    github.com/napari/napari
    napari
    https://ilovesymposia.com/2019/10/24/introducing-napari-
    a-fast-n-dimensional-image-viewer-in-python/

    View full-size slide

  63. @WillingCarol
    Learning resources
    64
    Online Books with
    Jupyter Notebooks
    Conferences, pyvideo.org
    and YouTube
    Research papers, blogs
    Source: O'Reilly

    View full-size slide

  64. @WillingCarol 65
    Ten Simple Rules
    for Reproducible
    Research in
    Jupyter Notebooks
    Adam Rule et al.
    https://github.com/jupyter-guide/
    ten-rules-jupyter
    https://github.com/jupyter-guide/
    jupyter-guide

    View full-size slide

  65. @WillingCarol
    Learn AI online
    66

    View full-size slide

  66. @WillingCarol
    Community
    67
    User groups
    Meetups
    PyLadies
    Carpentries
    Photo: Python Sul

    View full-size slide

  67. @WillingCarol
    What's new
    68
    Talk Python to Me
    Tracking Jupyter Newsletter
    https://tinyletter.com/TrackingJupyter/archive
    Open Source Directions
    GitHub Trending
    Follow projects on Social Media

    View full-size slide

  68. @WillingCarol
    Explore
    69
    Start with your interests
    Try a tutorial
    Learning resources
    Community
    What's new
    Step 2

    View full-size slide

  69. @WillingCarol
    Deploy
    70
    Step 3

    View full-size slide

  70. @WillingCarol
    Notebooks to web
    71
    https://blog.jupyter.org/and-
    voil%C3%A0-f6a2c08a4a93

    View full-size slide

  71. @WillingCarol 72
    nteract
    Papermill
    Scrapbook
    Bookstore
    Commuter
    Production data at scale
    72
    https://medium.com/netflix-techblog/notebook-innovation-591ee3221233
    Bookstore

    View full-size slide

  72. @WillingCarol 73
    Papermill - parameterize / run
    Scrapbook - recording / reading
    Bookstore - store notebooks
    Commuter - share notebooks
    Production data at scale
    73

    View full-size slide

  73. @WillingCarol 74
    Enterprise data workflows
    74

    View full-size slide

  74. @WillingCarol 75
    https://medium.com/dagster-io/dagster-0-6-0-impossible-princess-898b459375e0
    Pipelines

    View full-size slide

  75. zero-to-jupyterhub.readthedocs.io

    View full-size slide

  76. https://blog.jupyter.org/simpler-authentication-for-small-
    scale-jupyterhubs-with-nativeauthenticator-999534c77a09
    Native Authenticator
    Thank you Leticia

    View full-size slide

  77. @WillingCarol 78 78
    Deploy
    binders

    View full-size slide

  78. 79
    Juliette Taka

    View full-size slide

  79. @WillingCarol 80
    From a
    phone in
    the park!
    80

    View full-size slide

  80. @WillingCarol
    Pangeo
    81
    https://pangeo.io

    View full-size slide

  81. @WillingCarol 82

    View full-size slide

  82. @WillingCarol 83
    https://simexp.github.io/vcog_hps_ad_book/intro.html
    Jupyter Book
    Binder
    Jupyter
    pandas
    scipy
    scikit learn
    matplotlib
    numpy
    seaborn
    Canadian Open Neuroscience Platform

    View full-size slide

  83. @WillingCarol 84
    Practical Data Science
    How?

    View full-size slide

  84. @WillingCarol
    Start
    85

    View full-size slide

  85. @WillingCarol
    Ask why
    86

    View full-size slide

  86. @WillingCarol
    Plan
    Execute
    Change
    87
    https://jupyterhub-team-compass.readthedocs.io
    https://github.com/jupyterhub/team-compass

    View full-size slide

  87. @WillingCarol
    Share binders.
    Foster research.
    88

    View full-size slide

  88. @WillingCarol
    Build Community
    89
    Python Brasil official photo.
    The sign represents Python in BSL (Brazilian Sign Language),
    made by Amanda and Sávio at Python Sul 2018
    https://pyfound.blogspot.com/2019/02/python-brasil-people-technology.html
    PyLadiesBRConf official photo
    PyLadiesBRConf official photo

    View full-size slide

  89. @WillingCarol 90
    Python Brasil
    Practical Data Science

    View full-size slide

  90. @WillingCarol 91
    Python Brasil
    Changing the
    world together

    View full-size slide

  91. @WillingCarol 92
    Thank you
    https://speakerdeck.com/willingc
    @WillingCarol

    View full-size slide

  92. @WillingCarol 93
    Join the

    View full-size slide

  93. @WillingCarol 94
    Questions
    https://speakerdeck.com/willingc

    View full-size slide

  94. @WillingCarol 95
    Karaoke
    Time
    Lorena, This is for you.
    https://www.youtube.com/watch?v=m41cVv3tChE

    View full-size slide

  95. @WillingCarol 96
    Thank you Python Brasil
    Project Jupyter Team
    Core Python Team
    PSF

    View full-size slide

  96. @WillingCarol
    Attributions
    97
    Attributions on slides.
    Photos
    [7-10] Carol Willing and Linnea Willing
    [38] The Carpentries, Tracy Teal, Bérénice Batut
    [17] Godzilla
    By Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha) © 1954 - movie
    poster made by Toho Company Ltd. (東宝株式会社, Tōhō Kabushiki-kaisha),
    Public Domain, https://commons.wikimedia.org/w/index.php?curid=3648684

    View full-size slide