A Brief History of Jupyter Notebooks

The Jupyter Notebook: many Python users love it, many other Python users love to hate it. But where did it come from in the first place? How did we come to have this tool that combines code execution, visualization, Markdown, and more? In this talk we will dive into the development of the IPython (later Jupyter) Notebook, and the older ideas that it was built upon. We will examine the use cases that are driving adoption of the notebook in the present. And we will consider several open-source projects that point to the future of what the Jupyter Notebook could be.

Transcript

1. A Brief History of Jupyter Notebooks William Horton @hortonhearsafoo EuroPython 2020
2. Why this talk? @hortonhearsafoo EuroPython 2020
3. Two different worlds of Python @hortonhearsafoo EuroPython 2020
6. @hortonhearsafoo EuroPython 2020
7. @hortonhearsafoo EuroPython 2020
8. @hortonhearsafoo EuroPython 2020
9. My personal journey @hortonhearsafoo EuroPython 2020
10. Tensions between the two worlds @hortonhearsafoo EuroPython 2020
11. Joel Grus @hortonhearsafoo EuroPython 2020
12. The First Notebook War https://www.youtube.com/watch?v=QR7gR3njNWw @hortonhearsafoo EuroPython 2020
13. @hortonhearsafoo EuroPython 2020
14. Why not seek to understand the history? @hortonhearsafoo EuroPython 2020
15. The Jupyter Notebook didn’t come out of nowhere @hortonhearsafoo EuroPython 2020
16. No significant piece of software comes out of nowhere @hortonhearsafoo EuroPython 2020
17. Long-term trends: Scientific Computing Literate Programming Proprietary vs. Open Source …and Python @hortonhearsafoo EuroPython 2020
18. This talk: I. Past II. Present III. Future @hortonhearsafoo EuroPython 2020
19. Past @hortonhearsafoo EuroPython 2020
20. (Apologies to all the former science Ph.D’s out there) @hortonhearsafoo EuroPython 2020
21. @hortonhearsafoo EuroPython 2020
22. @hortonhearsafoo EuroPython 2020
23. Mathematica (1988) Created by Stephen Wolfram Notebook interface designed by Theodore Gray An all-in-one system: language (Wolfram), execution environment, scientific libraries @hortonhearsafoo EuroPython 2020
25. @hortonhearsafoo EuroPython 2020
26. @hortonhearsafoo EuroPython 2020
28. Key architectural details of Mathematica Notebooks Two parts to the system: kernel and front-end The front-end sends the input to the kernel, which returns the result, which is displayed Mathematica notebooks are objects that can be manipulated by Mathematica programs @hortonhearsafoo EuroPython 2020
29. The Mathematica Notebook Interface @hortonhearsafoo EuroPython 2020
30. @hortonhearsafoo
31. @hortonhearsafoo
32. Going further back... @hortonhearsafoo EuroPython 2020
33. @hortonhearsafoo EuroPython 2020
34. Literate Programming Donald Knuth Implemented the “WEB” system Concepts of “tangled” and “woven”: TANGLE: generates compilable code from the source WEAVE: generates formatted documentation from the source @hortonhearsafoo EuroPython 2020
35. Not just an abstract exercise @hortonhearsafoo EuroPython 2020
36. Maple Scientific computing environment + programming language First GUI for Maple released in 1989 1992 -- release of the “worksheet” interface: “Beginning with the Macintosh user interface for Maple V, the new user interfaces will all support the concept of a "worksheet" which integrates text, Maple input commands, Maple output, and graphics into one document.” @hortonhearsafoo EuroPython 2020
38. https://sites.math.rutgers.edu/courses/251/Maple/Help/errors2.html
39. Maple vs Mathematica @hortonhearsafoo EuroPython 2020
41. @hortonhearsafoo EuroPython 2020
42. @hortonhearsafoo EuroPython 2020
43. @hortonhearsafoo EuroPython 2020
44. The Rise of Open Source @hortonhearsafoo EuroPython 2020
45. SciPy + IPython + Matplotlib SciPy (2001): Created by Travis Oliphant, Eric Jones, and Pearu Peterson, it was a combination of scientific libraries for Python on top of the Numeric array type (later replaced by NumPy) IPython (2001): “The IPython project provides on enhanced interactive environment that includes, among other features, support for data visualization and facilities for distributed and parallel computation” (from “IPython: A System for Interactive Scientific Computing”, 2007) @hortonhearsafoo EuroPython 2020
46. @hortonhearsafoo EuroPython 2020
47. SciPy + IPython + Matplotlib (cont.) Matplotlib (2003): “It was conceived by John Hunter in 2002, originally as a patch to IPython to enable interactive MatLab-style plotting via gnuplot from the IPython command-line. Fernando Perez was, at the time, scrambling to finish his PhD, and let John know he wouldn’t have time to review the patch for several months. John took this as a cue to set out on his own, and the matplotlib package was born, with version 0.1 released in 2003.” (https://jakevdp.github.io/blog/2013/03/23/matplotlib-and-the-future-of-visualization-in-python/ ) @hortonhearsafoo EuroPython 2020
48. Fernando Pérez Created IPython in 2001 as a graduate student Currently: Associate Professor, Statistics, UC Berkeley and Berkeley Institute for Data Science Senior Fellow Recipient of the 2012 Award for the Advancement of Free Software from the Free Software Foundation, and the 2017 ACM Software System Award @hortonhearsafoo EuroPython 2020
49. initial IPython commit @hortonhearsafoo EuroPython 2020 (Thanks to Pavlo Andriychenko!) https://gist.github.com/fperez/1579699
50. Sage (2005) Created by William Stein Implemented in Python and Cython Open Source license (GPLv3) @hortonhearsafoo EuroPython 2020
51. Sage Notebook/Worksheets @hortonhearsafoo Screenshots from a tutorial circa 2012 ( http://doc.sagemath.org/html/en/prep/Intro-Tutorial.html#sagenbcommands ) EuroPython 2020
52. @hortonhearsafoo EuroPython 2020
53. @hortonhearsafoo EuroPython 2020
54. IPython Notebook (2011) @hortonhearsafoo EuroPython 2020
55. @hortonhearsafoo EuroPython 2020
56. Project Jupyter (2014) @hortonhearsafoo EuroPython 2020
57. Jupyter Ju(lia) + Pyt(hon) + R @hortonhearsafoo EuroPython 2020
58. Project Jupyter (2014) Spun off of IPython by Fernando Perez Includes the Notebook interface and other language-agnostic parts of IPython @hortonhearsafoo EuroPython 2020
59. Present @hortonhearsafoo EuroPython 2020
60. Trends: Data Science & Machine Learning Sharing & Interaction Growing ecosystem @hortonhearsafoo EuroPython 2020
61. Data Science & Machine Learning @hortonhearsafoo EuroPython 2020
62. Google Colab @hortonhearsafoo EuroPython 2020
63. Hosted solutions/notebook as an interface to compute @hortonhearsafoo EuroPython 2020
64. Example: Kubeflow @hortonhearsafoo EuroPython 2020
65. Sharing & Interaction @hortonhearsafoo EuroPython 2020
66. Binder @hortonhearsafoo EuroPython 2020
67. Voilà @hortonhearsafoo EuroPython 2020
68. ipywidgets @hortonhearsafoo EuroPython 2020
69. Growing ecosystem @hortonhearsafoo EuroPython 2020
70. Broad language support @hortonhearsafoo EuroPython 2020
72. Future(?) @hortonhearsafoo EuroPython 2020
73. Trends: Developer Experience Real-time Collaboration Productionizing @hortonhearsafoo EuroPython 2020
74. Present: JupyterLab @hortonhearsafoo EuroPython 2020
75. Future: More IDE-like Jupyter and more Jupyter in IDEs JupyterLab 2.0 -- released April 2020 Added JupyterLab Language Server Protocol + Debugger VSCode shipping improved native support for Jupyter Notebooks @hortonhearsafoo EuroPython 2020
76. Present: Github rendering @hortonhearsafoo EuroPython 2020
77. @hortonhearsafoo EuroPython 2020
78. Future: How to review & work with git? @hortonhearsafoo EuroPython 2020
79. @hortonhearsafoo EuroPython 2020
80. @hortonhearsafoo EuroPython 2020
81. Real-time collaboration @hortonhearsafoo EuroPython 2020
82. Productionizing @hortonhearsafoo EuroPython 2020
83. Netflix https://netflixtechblog.com/notebook-innovation-591ee3221233 @hortonhearsafoo EuroPython 2020
84. Netflix Notebook Templates/Parameterized Notebooks Scheduling Notebooks as workflows Open-source tools like nteract and Papermill @hortonhearsafoo EuroPython 2020
85. fastai v2 Writing the whole library (and documentation) in Jupyter Notebooks @hortonhearsafoo EuroPython 2020
86. Conclusion @hortonhearsafoo EuroPython 2020
87. The history of Jupyter Notebooks is not linear @hortonhearsafoo EuroPython 2020
88. @hortonhearsafoo EuroPython 2020
89. @hortonhearsafoo EuroPython 2020
90. The history of Jupyter Notebooks is not over. @hortonhearsafoo EuroPython 2020
91. @hortonhearsafoo EuroPython 2020
92. ?? @hortonhearsafoo EuroPython 2020
93. The history of Jupyter Notebooks is not over. It’s still being written...by you! @hortonhearsafoo EuroPython 2020
94. Thank you! #talk-history-jupyter-notebooks @hortonhearsafoo