The Future is Notebooks

Co-presented with Elijah Meeks, Chief Innovation Officer. ODSC East 2023 (https://odsc.com/boston/).

This year, OpenAI and ChatGPT, have boomed as popular tools for data and idea generation. With the generative power of these tools, a critical inspection of the resulting data grows in importance. Is the data valid for the use case? What aspects of the data may be harmful or a poor fit for the problem to be solved? Exploring and answering these questions will drive greater adoption of notebooks to make this new technology accessible to larger groups of people.

Notebooks with an elegant combination of prose, visualization, and code open the door for greater understanding and introspection of data. The ability to explain the data through words and illustrative charts provides a foundation for anyone to understand and critically reason about data and its application. While other tools like an IDE or BI tool provide artifacts, the notebooks compose information in a way that is greater than the sum of individual parts and enable more people to experiment and take action. The future is notebooks.

Transcript

1. The Future is Notebooks Carol Willing Elijah Meeks noteable.io
2. | 2 About the Speakers Elijah Meeks Chief Innovation Officer Carol Willing VP Engineering • Jupyter Software Steering Council • JupyterHub and Binder Core Developer • 2017 ACM Software System Award • Python Core Developer • Python Steering Council • PSF Fellow • Co-Founder at Noteable & Data Visualization Society • Author & Speaker on Data Visualization • Netflix, Apple • Great shirts • Unconventional path to tech • Late to Notebooks
3. | A Brief History of Notebooks • First Generation IPython Notebooks to Jupyter • Second Generation: 2018 - 2023 Collaboration & Modern Data Stack • Third Generation: Now - 2028 AI, Time, and Scale 3 Agenda: The Future is Notebooks
4. | 4 What Are Notebooks Notebooks rock You might not realize that, yet But we’re here to help
5. | 5 First Generation: IPython Notebooks to Jupyter
6. | 6 First Generation: IPython Notebooks to Jupyter
7. | 7 First Generation: 2017 ACM Software System Award
8. | 8 The Classic Notebook
9. | 9 Second Generation: 2018 - 2023
10. | 10 Second Generation: 2018 - 2023
11. | 11 Modern Data Notebooks
12. | 12 Paradigm Shift: AI, Value of Time, Scale We’re in the middle of a paradigm shift at scale AI is rapidly changing computing and humanity
13. | 13 The Third Generation of Notebooks is Here Aspirational and Inspirational • Productivity of experts • Unlocking context and capabilities for all • Available everywhere (mobile, office, global) • Visualize when a picture can convey a thousand words
14. | 14 Where will Notebooks Evolve? Notebook++ Isn’t the Answer • “Notebook for the Modern Data Stack” • Modern Data Stack Fatigue • Integrate new technologies more often than synthesizing new workflows • Creators and Consumers as distinct categories
15. | 15 The Third Generation of Notebooks: Onboarding Imagine a Radically Inclusive Data Tool • Natural language for greater access by all • Cognitive science and learning: success by a user motivates more usage and more possibility for complex analysis and tasks • Consolidation of help and docs in one natural language interface
16. | 16 The Third Generation of Notebooks: Onboarding What excites people about ChatGPT? • It does text well • It does code well • Code + Text is a notebook
17. | 17 The Third Generation of Notebooks: Onboarding Everyone can level up their impact
18. | 18 The Third Generation of Notebooks: Superpowers All Users get Superpowers • Create me a chart that shows… • Summarize the code cell in prose for my peer • Create a prose cell that draws conclusions from this dataframe
19. | 19 What if I don’t know how to code but I know how to bake?
20. | 20 Not a baker? Or a 5th grader interested in baseball Maybe you’re a Hollywood exec… Or a VA psychiatrist… Or you like dogs… or run a small business… or own a Rubik’s Cube…
21. | 21 The Third Generation of Notebooks: Superpowers All notebooks will be collaborative • Never work with data alone again ◦ Error response ◦ Code copilot ◦ ChatGPT in the comments • AI assistants for creation of prose and viz • AI guides to assist new users
22. | 22 The Third Generation of Notebooks: Superpowers Everyone can create / use notebooks • Guided visualization (DataPrism) • AI assistants (Noteable ChatGPT Plugin) • Fully web based for users • Guided pipelines for creators
23. | 23 The Third Generation of Notebooks: Superpowers It’s not just AI
24. | 24 The Third Generation of Notebooks: 2023 to 2028 Context is Key • Scenarios for business options • Improved decision making with confidence • Risk analysis
25. | 25 The Third Generation of Notebooks: Context Everyone can share context • Scenarios • Domain knowledge sharing • Risk assessments • From data to pipelines to business impact
26. | 26 The Third Generation of Notebooks: Context Where Will Notebooks Be in 5 Years Anyone can create a notebook anywhere, everyone in a notebook will be simultaneously creator, collaborator and consumer • What is a data engineer when notebooks have integrated ETLs? • What is a data analyst when they’re using the same tools with the same feature breadth as data scientists? • What is a data scientist when anyone can deploy any ML with just a prompt?
27. | 27 Takeaways Takeaways • Notebook++ Isn’t Enough • Include Everyone in Data Work • All Users Get Superpowers • Context is Key
28. | 28 The End Fin! Thank you
29. The Future is Notebooks Carol Willing Elijah Meeks noteable.io