Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical Python and Jupyter for Data Science and More

Practical Python and Jupyter for Data Science and More

Get started or level up your skills using Python, Jupyter, and many more libraries in the Python ecosystem. This keynote was presented at PyCon MY 2019 in Kuala Lumpur.

Carol Willing
PRO

August 25, 2019
Tweet

More Decks by Carol Willing

Other Decks in Technology

Transcript

  1. @WillingCarol
    Practical Python and Jupyter
    for Data Science and Beyond
    Carol Willing

    View Slide

  2. @WillingCarol
    Hi! I'm Carol.
    • Python Steering Council
    • Core Developer
    • PSF Fellow
    • PSF Director (Former)
    • Frank Willison Award 2019
    2

    View Slide

  3. @WillingCarol 3
    Core maintainer
    Papermill, Scrapbook,
    Bookstore, Commuter
    Steering Council
    Core Developer
    JupyterHub, BinderHub,
    mybinder.org
    I create data science tools.

    View Slide

  4. @WillingCarol 4
    Core maintainer
    Papermill, Scrapbook,
    Bookstore, Commuter
    Steering Council
    Core Developer
    JupyterHub, BinderHub,
    mybinder.org
    I create productivity data science tools.

    View Slide

  5. @WillingCarol
    Practical Data Science
    5
    Using data responsibly to solve real
    world issues and improve human lives

    View Slide

  6. @WillingCarol 6
    Practical Data Science
    Python + Jupyter +
    Knowledge + Community

    View Slide

  7. @WillingCarol
    Practical data science
    grew up around

    View Slide

  8. @WillingCarol 8

    View Slide

  9. @WillingCarol 9
    Python + Jupyter +
    Knowledge + Community

    View Slide

  10. @WillingCarol
    Python
    Growth
    Governance
    Python 3.8
    Farewell to Python 2
    10

    View Slide

  11. @WillingCarol
    Growth
    11
    Flexible, empowering language
    Strength of third-party libraries
    Community

    View Slide

  12. @WillingCarol 12
    What's behind the growth?
    Web
    DevOps
    Systems
    Testing
    Science
    Data Science
    Embedded systems
    Education
    Micropython,
    CircuitPython,
    Raspberry Pi

    View Slide

  13. @WillingCarol 13
    Language Design
    In reality, programming languages
    are how programmers express and
    communicate ideas — and the
    audience for those ideas is other
    programmers, not computers.
    http://neopythonic.blogspot.com/2016/04/kings-day-speech.html
    Guido van Rossum

    View Slide

  14. @WillingCarol
    Governance
    Post-BDFL
    14

    View Slide

  15. @WillingCarol
    PEP 8016
    The Steering Council Model
    15

    View Slide

  16. @WillingCarol
    Steering Council
    16
    Barry Warsaw
    Brett Cannon
    Carol Willing
    Guido van Rossum
    Nick Coghlan

    View Slide

  17. @WillingCarol
    PEP 13 Python Language Governance
    17
    ‣ Quality and Stability
    ‣ Contributing accessible, inclusive, sustainable
    ‣ Core team and PSF relationship
    ‣ Decision making processes for PEPs
    ‣ Seek consensus

    View Slide

  18. @WillingCarol
    Core Development
    18

    View Slide

  19. @WillingCarol
    Python 3.8
    19
    https://www.python.org/downloads/release/python-380b3/
    https://www.python.org/dev/peps/pep-0569/
    Schedule and Content
    Try it
    3.8-dev currently uses Python 3.8.0b3+
    Test your project on Travis CI

    View Slide

  20. @WillingCarol 20
    Positional only parameters
    Assignment operator (walrus operator)
    "equals" specifier added to f-string (debugging)
    PEP 587: Python Initialization Configuration
    PEP 574: pickle protocol 5
    Python 3.8

    View Slide

  21. @WillingCarol
    Farewell to Python 2
    21

    View Slide

  22. @WillingCarol 22
    py3readiness.org
    !22

    View Slide

  23. @WillingCarol 23
    2020-01-01
    pythonclock.org

    python3statement.org
    23 !23

    View Slide

  24. @WillingCarol 24
    Python + Jupyter +
    Knowledge + Community

    View Slide

  25. @WillingCarol 25
    jupyter.org

    View Slide

  26. @WillingCarol 26
    2014
    Now,
    5 years
    later...
    !26

    View Slide


  27. Millions of
    Notebooks
    https://github.com/trending/jupyter-notebook
    Over 5 million
    on GitHub

    View Slide

  28. @WillingCarol 28
    Jupyter
    Growth
    ACM Award
    Industry adoption
    Creative uses
    Open Source Book
    !28

    View Slide

  29. @WillingCarol 29
    https://www.youtube.com/watch?v=qbtDVdEr8SY

    View Slide

  30. @WillingCarol
    ecosystem
    30

    View Slide

  31. @WillingCarol 31
    Python + Jupyter +
    Knowledge + Community

    View Slide

  32. @WillingCarol 32

    View Slide

  33. @WillingCarol 33
    Prepare
    Explore
    Prototype
    Production

    View Slide

  34. @WillingCarol
    Prepare
    Getting Started and Leveling up
    34
    Step 1

    View Slide

  35. @WillingCarol
    Getting Started and Leveling Up
    35
    Try it in the browser
    Install Python
    Install Libraries
    Choose your tools

    View Slide

  36. @WillingCarol 36
    Binder 2.0 blog post
    elifesciences: Share
    your interactive
    research environment
    Nature article about
    Binder
    !36
    mybinder.org
    Try it in the
    browser

    View Slide

  37. @WillingCarol 37

    View Slide

  38. jupyter.org

    View Slide

  39. @WillingCarol
    pyodide
    39
    Try it in the Browser

    View Slide

  40. @WillingCarol
    Install Python
    40
    Use the OS Package Manager
    Use a Distribution - Anaconda, miniconda
    Download from Python.org

    View Slide

  41. @WillingCarol
    Install Libraries
    41
    Use a Distribution
    such as Anaconda
    Use pip
    Use miniconda, conda,
    and conda-forge

    View Slide

  42. @WillingCarol 42
    python3 -m venv myenv
    source myenv/bin/activate
    pip install jupyter
    jupyter notebook

    View Slide

  43. @WillingCarol 43
    conda create -n myenv python=3.7
    conda activate myenv
    conda install jupyter
    jupyter notebook

    View Slide

  44. @WillingCarol
    Choose your Tools
    44

    View Slide

  45. @WillingCarol
    JupyterLab
    45
    Congrats!
    JupyterLab
    team
    releases 1.0
    version.

    View Slide

  46. @WillingCarol 46
    nteract
    ReactJS front end

    View Slide

  47. @WillingCarol
    nteract
    47
    Jupyter's open notebook format, protocol, and language kernel
    standards enable different user interfaces.

    View Slide

  48. @WillingCarol
    VS Code
    48
    PyCharm

    View Slide

  49. @WillingCarol
    Exploration
    49
    Step 2

    View Slide

  50. @WillingCarol
    Exploring data and libraries
    50
    Start with your interests
    Try a tutorial
    Learning resources
    Community
    What's new

    View Slide

  51. @WillingCarol
    Start with your
    interests
    I usually create music
    projects when exploring new
    languages.
    51
    https://web.mit.edu/music21/

    View Slide

  52. @WillingCarol
    Try a library's tutorial
    Pandas
    Matplotlib
    52

    View Slide

  53. @WillingCarol 53

    View Slide

  54. @WillingCarol
    Learning resources
    54
    Online Books with Jupyter Notebooks
    Conferences, pyvideo.org and YouTube
    https://jakevdp.github.io/PythonDataScienceHandbook/

    View Slide

  55. @WillingCarol
    Learn AI online
    55

    View Slide

  56. @WillingCarol
    Community
    56
    User groups
    Meetups
    PyLadies
    Carpentries

    View Slide

  57. @WillingCarol
    What's new
    57
    Talk Python to Me
    Tracking Jupyter Newsletter
    https://tinyletter.com/TrackingJupyter/archive
    Open Source Directions webinar
    GitHub Trending
    Follow projects on Social Media

    View Slide

  58. @WillingCarol
    Prototyping
    58
    Step 3

    View Slide

  59. @WillingCarol 59
    Ten Simple Rules
    for Reproducible
    Research in
    Jupyter Notebooks
    Adam Rule et al.
    https://github.com/jupyter-guide/
    ten-rules-jupyter
    https://github.com/jupyter-guide/
    jupyter-guide

    View Slide

  60. @WillingCarol 60
    https://github.com/data-exp-lab/rust-yt-tools/
    npm package @data-exp-lab/yt-tools
    Irber Junior LC. Oxidizing Python: writing extensions
    in Rust [version 1; not peer
    reviewed]. F1000Research 2018, 7(ISCB Comm J):955
    (poster) (https://doi.org/10.7490/f1000research.
    1115726.1)
    https://github.com/munkm/widgyts
    yt and
    jupyter
    widgets
    !60

    View Slide

  61. @WillingCarol 61
    ipyvolume
    https://towardsdatascience.com/multivolume-
    rendering-in-jupyter-with-ipyvolume-cross-
    language-3d-visualization-64389047634a

    View Slide

  62. @WillingCarol 62

    View Slide

  63. @WillingCarol 63
    https://github.com/pydy/pydy-tutorial-human-standing
    https://mybinder.org/v2/gh/willingc/pydy-tutorial-human-standing/master

    View Slide

  64. @WillingCarol 64
    Animation
    !64
    jupyter.org

    View Slide

  65. @WillingCarol
    Production
    65
    Step 4

    View Slide

  66. @WillingCarol 66
    Focus on your impact

    View Slide

  67. @WillingCarol 67
    Papermill
    Scrapbook
    Bookstore
    Commuter
    Production data at scale
    !67
    https://medium.com/netflix-techblog/notebook-innovation-591ee3221233

    View Slide

  68. @WillingCarol 68
    Papermill - parameterize / run
    Scrapbook - recording / reading
    Bookstore - store notebooks
    Commuter - share notebooks
    Production data at scale
    !68

    View Slide

  69. @WillingCarol 69
    Enterprise data workflows
    !69

    View Slide

  70. zero-to-jupyterhub.readthedocs.io

    View Slide

  71. @WillingCarol 71 !71
    Deploy
    binders

    View Slide

  72. !72
    Juliette Taka

    View Slide

  73. @WillingCarol 73
    From a
    phone in
    the park!
    !73

    View Slide

  74. @WillingCarol 74
    Getting started & leveling up
    Exploration
    Prototyping
    Production

    View Slide

  75. @WillingCarol 75
    Python + Jupyter +
    Knowledge + Community

    View Slide

  76. @WillingCarol 76
    – Guido van Rossum
    ...a programming language created
    by a community fosters happiness
    in its users around the world.
    http://neopythonic.blogspot.com/2016/04/kings-day-speech.html

    View Slide

  77. @WillingCarol 77
    –Tim O'Reilly
    Contributions can encompass so
    much more than code. A successful
    software community requires time,
    dedication, communication, and
    education as well as elegant code.


    View Slide

  78. @WillingCarol 78
    Be Practical and Productive

    View Slide

  79. @WillingCarol 79
    Python + Jupyter +
    Knowledge + Community

    View Slide

  80. @WillingCarol 80
    Designed for success
    Try it today

    View Slide

  81. @WillingCarol 81
    Create your dreams

    View Slide

  82. @WillingCarol 82
    the future of
    Python
    depends on

    View Slide

  83. @WillingCarol 83
    YOU

    View Slide

  84. @WillingCarol 84
    Thank you
    @WillingCarol
    willingc on
    GitHub


    View Slide

  85. @WillingCarol 85
    Join the

    View Slide

  86. @WillingCarol 86
    Questions

    View Slide

  87. @WillingCarol 87
    Thank you
    Project Jupyter Team
    Core Python Team
    PSF

    View Slide

  88. @WillingCarol
    Attributions
    88
    Attributions on slides.
    Photos
    [2, 3, 4, 5, 6, 7, 45, 55, 56, 62, 67, 68, 70] Carol Willing and Linnea Willing
    [38] The Carpentries, Tracy Teal, Bérénice Batut

    View Slide