Oracle Machine Learning Notebooks - Deep Dive Part I - AskTOM Office Hours

This is Part I of the deeper dive into Oracle Machine Learning Notebooks. In this Session we learned about administrative and collaborative functionality from the OML Notebooks, which is the Zeppelin–based interface available for the Oracle Autonomous Database.

For Part II we will check more specific and advanced OML4SQL API functionality, including partitioned models and text mining.

We reviewed demos and code, and learned what's possible today, as well as heard about what's coming on Oracle Machine Learning Notebooks for Autonomous Database.

Transcript

1. The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file. The image part with relationship ID rId3 was not found in the file. With Mark Hornick, Senior Director, Product Management, Data Science and Machine Learning @MarkHornick Marcos Arancibia, Product Manager, Data Science and Big Data @MarcosArancibia oracle.com/machine-learning Oracle Machine Learning Office Hours Oracle Machine Learning Notebooks Copyright © 2020, Oracle and/or its affiliates. All rights reserved
2. Topics Upcoming session Speaker Mark Hornick – Oracle Machine Learning Notebooks Q&A Copyright © 2020 Oracle and/or its affiliates.
3. Next Session to look for: April 8, 2020: Oracle Machine Office Hours, 9AM US Pacific Oracle Machine Learning Notebooks, Part 2 Continuation of today’s session. Mark Hornick, Senior Director, Product Management, Data Science and Machine Learning Copyright © 2020, Oracle and/or its affiliates. All rights reserved
4. For product downloads and more info… https://www.oracle.com/machine-learning Copyright © 2020 Oracle and/or its affiliates.
5. Copyright © 2020 Oracle and/or its affiliates. https://www.oracle.com/cloud/free/
6. Today’s Session: Oracle Machine Learning Notebooks with Mark Hornick Take a deeper dive into Oracle Machine Learning Notebooks through the Zeppelin–based interface and additional OML4SQL API functionality We will review demos and code, see what's possible today, and hear about what's coming on Oracle Machine Learning Notebooks for Autonomous Database Copyright © 2020, Oracle and/or its affiliates. All rights reserved
7. Oracle Machine Learning Notebooks Deep Dive Mark Hornick Oracle Machine Learning Product Management Copyright © 2020 Oracle and/or its affiliates. Oracle Machine Learning Office Hours
8. The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. Safe harbor statement Copyright © 2020, Oracle and/or its affiliates. All rights reserved
9. Oracle Machine Learning Notebooks Collaborative UI • Based on Apache Zeppelin • Supports data scientists, data analysts, application developers, DBAs • Easy sharing of notebooks and templates • Permissions, versioning, and execution scheduling Included with Autonomous Database • Automatically provisioned, managed, backed up • In-database SQL algorithms and analytics functions • Explore and prepare data, build and evaluate models, score data, deploy solutions • SQL today — soon to be augmented with Python and R Autonomous Database as a Data Science Platform Copyright © 2020 Oracle and/or its affiliates.
10. Copyright © 2020, Oracle and/or its affiliates 12 • Quick Orientation Demo • Objects and Relationships • Projects and Workspaces • Notebook Basics • Collaboration • Connection Groups and Interpreter Bindings • Scheduling Jobs for Notebook Execution • Output Formats in Notebooks • Useful “forms” in Notebooks • Lessor-known OML4SQL Features • OML Roadmap on ADB OML Notebooks Topics
11. Quick Orientation Demo Copyright © 2020, Oracle and/or its affiliates. All rights reserved
12. Copyright © 2020, Oracle and/or its affiliates 15 Login Hamburger menu: Home, Notebooks, Templates, Jobs, Connection Groups, Notebook Sessions Home page: How do I?, Quick Actions, Recent Activities, Recent Notebooks Project Selection and Workspace management User: preferences, password changes, Help, About, Log out Notebooks page Templates • Personal Page • Shared Page • Examples Jobs Connection groups Notebooks Sessions Script – quick overview of OML Notebooks environment
13. Objects and Relationships Copyright © 2020, Oracle and/or its affiliates. All rights reserved
14. Copyright © 2020, Oracle and/or its affiliates 17 Autonomous Database and OML Users Autonomous Database Pluggable Database (PDB) OML User - 1 OML User - 2 OML User - N …
15. Copyright © 2020, Oracle and/or its affiliates 18 OML Notebooks – workspaces, projects, notebooks OML User Workspace Workspace Project Project Project Notebook Notebook Notebook Notebook Scratchpad Notebook Notebook Notebook Scratchpad Notebook Scratchpad Notebook … … …
16. Copyright © 2020, Oracle and/or its affiliates 19 A Data Science Team Structure Data Scientist DBA Application Developer Executive Data Analyst Project Lead Provides reports to Oversees Data Management Support Provides analytics to
17. Copyright © 2020, Oracle and/or its affiliates 20 Autonomous Database and OML – Data Science Project Scenario DB1 Autonomous Database PDB MARK Project Lead PETE App Developer CINDY Data Analyst ADMIN ANNE Data Scientist SERGIO Executive TIM DBA GOLD_DATA
18. Copyright © 2020, Oracle and/or its affiliates 21 Users from ADMIN User Administration System Administrator • Create, edit, and delete OML user accounts • Read-only access to notebooks • Cannot create or run notebooks • Reassign user workspace when deleting users Developer • Create projects and workspaces • Create, edit, delete, version, and run notebooks • Use SQL and SQL Script Scratchpads • Schedule jobs for notebook execution
19. Copyright © 2020, Oracle and/or its affiliates 22 OML Notebooks – workspaces, projects, notebooks OML User: MARK – project lead MARK Workspace DevTest Workspace MARK Project Customer Segmentation Demand Forecasting Notebook Notebook Notebook Scratchpad Notebook Notebook Scratchpad Notebook Scratchpad Notebook … Workspace shared with other team members as Developer or Viewer
20. Copyright © 2020, Oracle and/or its affiliates 23 Data Science Team Workspace Permissions Data Scientist Developer DBA Developer Application Developer Viewer Executive Viewer Data Analyst Developer Project Lead DevTest Workspace owner • Create, edit, and delete OML user accounts • Read-only access to notebooks • Cannot create or run notebooks • Reassign user workspace when deleting users ADMIN • Provide notebook reports to Executive • Review team’s work and collaborate using concurrent notebook sessions • Explore, prepare, and analyze data • Build and test ML models • Deliver ML scripts to data analyst and application developer • View notebook reports on project from the project lead • View any DevTest workspace notebooks • Access machine learning scripts to include in applications and dashboards • Use SQL for scoring data with ML models • Read-only view of others’ scripts • Analyze data using Notebooks or BI tools like Oracle Analytics Cloud • Use machine learning scripts from data scientist • Load and prepare data for team • Collaborate with other users to assist with queries and data analysis
21. Copyright © 2020, Oracle and/or its affiliates 24 User Roles from MARK ‘DevTest’ Workspace Permissions
22. Projects and Workspaces Copyright © 2020, Oracle and/or its affiliates. All rights reserved
23. Copyright © 2020, Oracle and/or its affiliates 26 Initial workspace and default project created by OML service automatically when you first log in to OML CINDY Data Analyst create a new workspace within the Create Project dialog and shares her workspace with other users Manage Workspaces • On the top right corner in your home page, click Select a Project pulldown menu • Select Manage Workspaces • Manage Workspaces dialog box opens • Click Permissions to add and delete users • Click Edit to edit the selected workspace • Click Delete to delete the selected workspace • When finished, click Close. Projects and Workspaces
24. Copyright © 2020, Oracle and/or its affiliates 27 Collaborate with other users by granting permissions to access your workspace Grant permissions Manager, Developer, and Viewer to allow another user to view your workspace Granting permission type Manager or Developer allows user to drop tables, create tables, and run any scripts at any time on your account Workspace Permissions • On the top right corner in your home page, click Select a Project pulldown menu • Select Workspace Permissions • Workspace Permissions dialog box opens • Select user name and permission type, then click + Add • Select row and click Delete to remove user permissions Note: re-adding a user overwrites previous settings More on Workspace Permissions
25. Copyright © 2020, Oracle and/or its affiliates 28 Viewer • Project: view only • Workspace: view only • Notebooks: view only • Jobs: view jobs and job runs of shared notebooks only Note: Users are not authorized to run or make any changes to your notebooks – useful for “reports” Developer • Project: view only • Workspace: view only • Notebooks: create, update, run, and delete notebooks that a developer creates only • Jobs: view and run jobs of shared notebooks only. A developer cannot create jobs for notebooks that are shared Note: Users can drop tables and run any script at any time on the owner’s account Manager • Project: create, update, delete • Workspace: view only • Notebooks: create, update, run, delete • Jobs: schedule jobs Note: Users can drop tables and run any script at any time on the owner’s account Workspace Permissions
26. A Few Notebook Basics Copyright © 2020, Oracle and/or its affiliates. All rights reserved
27. Copyright © 2020, Oracle and/or its affiliates 30 On the Notebooks page, click Create Must be in a workspace/project you can write to ANNE Data Scientist creates a notebook in DevTest Workspace and Customer Segmentation Project This is now visible to other team members who have access to DevTest Workspace Create a Notebook
28. Copyright © 2020, Oracle and/or its affiliates 31 On the Notebooks page, click Import Must be in a workspace/project you can write to ANNE Data Scientist imports a Clustering notebook in DevTest Workspace and Customer Segmentation Project Discussed later: check interpreter bindings are set to enable proper notebooks execution Import a Notebook
29. Copyright © 2020, Oracle and/or its affiliates 32 Scratchpads are “glorified” notebooks SQL and Script scratchpads can have paragraphs of either %sql or %script TIM DBA opens a SQL Query Scratchpad and lists the tables in all_tables If you return to Notebooks page, your Scratchpad is saved with name “SQL Query Scratchpad” or “SQL Script Scratchpad” Users can rename a Scratchpad notebook Renaming or deleting a default-named Scratchpad notebook gets you a clean Scratchpad next access Save a Scratchpad as a Notebook
30. Copyright © 2020, Oracle and/or its affiliates 33 On the Notebooks page, MARK Project Lead selects notebook to version and clicks Version This allows seeing and manipulating existing versions, as well as create new ones On Versions page • Click + Version • Create Versions dialog opens • Enter comments for this version of your notebook, and click OK The versioned notebook is now listed in the Versions page. You can… • Click Revert Version to restore the older version of your notebook • Click Delete to delete the selected version of your notebook • Click New Notebook to create a new notebook from the selected notebook version • Results in a notebook with same name + “_i”, where i = 1..n each time invoked Version a Notebook
31. Copyright © 2020, Oracle and/or its affiliates 34 OML Templates UI provides another way to share work • Publish notebooks as read-only reports • Publish snapshots of notebooks • Allow users to create notebooks from templates ANNE Data Scientist, on the Notebooks page, selects a notebook and click Save as Template Save as Template dialog opens • Name field – enter a name for the notebook template • Comments field – enter comments, if any • Tags field – enter tags for the template, if any • In Save To, select: • Personal: If you want to save this notebook template to Personal • Shared: If you want to save and share this notebook template Save a Notebook as a Template
32. Collaboration Copyright © 2020, Oracle and/or its affiliates. All rights reserved
33. Copyright © 2020, Oracle and/or its affiliates 36 Share your work, publish your work as reports, or create notebooks from templates Granting Workspace Access To collaborate, grant access to your workspace to another user When other users log in, your workspace, projects in that workspace, and any workspace notebooks are visible Open the same notebook in different users’ accounts to dynamically see others’ changes All can open notebooks and work on or see changes in real time relative to granted permissions OML Templates UI Three types: Personal, Shared, and Examples View selected templates in read-only mode Create new editable/runnable notebooks from selected templates Collaborate with other users
34. Connection Groups and Interpreter Bindings Copyright © 2020, Oracle and/or its affiliates. All rights reserved
35. Copyright © 2020, Oracle and/or its affiliates 39 A connection group, also known as a Zeppelin interpreter set, is a collection of database connections Global Connection Group is created automatically when a new autonomous database is provisioned Developer • View database connections Administrator • View and Edit database connections: Name, Row Render Limit, Comment • Reset and Stop database connections • In the Database section, specify PL/SQL DBMS output option • Select Enabled to allow the PL/SQL interpreter to display messages sent to DBMS_OUTPUT in paragraph results • Requires users to log back in for change to take effect Connection Groups
36. Copyright © 2020, Oracle and/or its affiliates 40 Interpreters • A plug-in for using a specific data processing language at the backend, e.g., SQL, Python, R • Currently supported interpreters • sql and script (PL/SQL) interpreters within an Oracle Database interpreter group • md (MarkDown) interpreter for plain text formatting syntax for conversion to HTML Interpreter Bindings • Association of interpreters to notebooks, and individual paragraphs • Notebooks contain an internal ordered list of bindings, default order à Low, Medium, High • *Low: least level of resources to each SQL statement, maximizes concurrent SQL statements • Medium: lower level of resources to each SQL statement, allows more concurrent SQL statements • High: highest level of resources to each SQL statement resulting in the highest performance, but significantly limits number of concurrent SQL statements • Toggle bindings on/off and drag buttons to order – top bindings take precedence over lower bindings Interpreters and Interpreter Bindings * Only non-parallel Low access is supported with the Always Free account
37. Copyright © 2020, Oracle and/or its affiliates 41 May want to override execution priority / resources for an individual paragraph To change the interpreter bindings order for a particular paragraph in the notebook: • Click the paragraph where you want to change the interpreter • Invoke the interpreter with the specific binding • Run the paragraph For example, TIM DBA invokes Scratchpad with medium resource allocation, where db1 is the database name %db1_medium Validate the interpreter binding by executing SELECT SYS_CONTEXT ('USERENV', 'SERVICE_NAME') FROM DUAL; Override Interpreter Bindings within a Paragraph
38. Copyright © 2020, Oracle and/or its affiliates 42 If you do not bind any specific interpreter to your notebook, you receive the error: databasename_servicename not found TIM DBA explicitly sets the interpreter binding order in the following scenarios: • Notebook creation: notebook inherits the initial interpreter binding order, which is low (default), medium, high • Notebook import: notebook inherits the defined interpreter bindings – after importing a notebook, verify interpreter binding order and activation • Notebook export: notebook inherits the defined interpreter bindings • Interpreter bindings are not portable across Pluggable Databases (PDBs) • If export and import notebooks within the same PDB, interpreter bindings are preserved • If notebooks are created in different PDBs, interpreter bindings are not preserved after import, and must be manually set • Notebook creation from templates: notebook inherits the default order of interpreter bindings Interpreter Bindings
39. Copyright © 2020, Oracle and/or its affiliates 66 Q & A
40. For more information… oracle.com/machine-learning Copyright © 2020 Oracle and/or its affiliates.
41. Thank You Mark Hornick | mark.hornick@oracle.com Marcos Arancibia | marcos.arancibia@oracle.com Oracle Machine Learning Product Management