Hands-On Lab using Oracle Machine Learning AutoML UI on Autonomous Database

In this Hands on Lab, we experienced Oracle Machine Learning AutoML UI on Oracle Autonomous Database.

AutoML UI provides the features of OML Automated Machine Learning (AutoML) for algorithm selection, adaptive sampling, feature selection and hyperparameter tuning.

AutoML UI allows for an automatic creation of a OML4Py Notebook with content for the best tuned model and all hyperparameters chosen by AutoML for the model desired.

AutoML UI also deploys Models to OML Services with one click, which creates REST APIs for the native in-database OML models and makes them ready to score in real-time.

Sign up for this tour of OML AutoML UI, and we will distribute credentials for you to do the Live exercises using the environment during the Session.

Transcript

1. Oracle Machine Learning Office Hours Hands-on-Lab: OML AutoML UI OML User Interface for AutoML model building and deployment with Marcos Arancibia, Sherry LaMonica & Mark Hornick Product Management, Oracle Machine Learning oracle.com/machine-learning May 2021
2. • Test drive the new Oracle Machine Learning AutoML UI available with Oracle Autonomous Database • Explore different aspects of Automated Machine Learning model building with in-database machine learning models • Build models with OML AutoML UI and deploy them directly to OML Services • Auto-generate OML4Py notebooks that can build the top models, with all Hyperparameters and options already set • Learn about possible integrations with other Oracle Cloud Applications Goals Copyright © 2021, Oracle and/or its affiliates 2
3. • The main objective of this 90-minute Hands-on-Lab Session is to get you familiar with OML AutoML UI, to become self-sufficient with the Live Labs environment, and show many of the OML AutoML UI capabilities • These labs focus on the OML AutoML UI, but other OML components will be used in conjunction to show model scoring. • This series of labs is not intended as an introduction to machine learning or details of specific algorithms • To learn more about Machine Learning concepts, check out our recorded OML Office Hours ML 101/102 sessions at https://asktom.oracle.com/pls/apex/asktom.search?oh=6801 Setting expectations Copyright © 2021, Oracle and/or its affiliates 3
4. • Accessing your Live Labs instance • Introduction to OML AutoML UI • Lab overview • Work through the labs • Q&A throughout Agenda Copyright © 2021, Oracle and/or its affiliates 4
5. • Marcos Arancibia – Instructor • Mark Hornick– Expert • Sherry LaMonica – Expert Hands-on lab experts Need lab help? Input your questions into the Chat to the Panelists Have questions? Input your questions into the Q&A Please let us know if you have any problems All Links will be distributed via the Chat window
6. Copyright © 2021, Oracle and/or its affiliates 6 Accessing your Live Labs instance
7. If you ran any Hands-on-Lab session earlier today with a Live Labs reservation that is still running (with status "Available"), you need to delete it before joining a new one. Copyright © 2021, Oracle and/or its affiliates 7 Go to LiveLabs "My Reservations" and login with your Oracle Account at: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/my-reservations Click on the Trashcan icon to delete the Running reservation, and confirm you want to Delete it Note: If you try to run this LiveLabs while you have another one running (in Available state), you will get an error
8. Browse to the address below (in Firefox or Google Chrome), and click on the "Reserve Workshop on LiveLabs" (Green Button): Copyright © 2021, Oracle and/or its affiliates 8 https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=786
9. If you do not have a free Oracle.com account, you will be required to create one now. It is fast and free. Copyright © 2021, Oracle and/or its affiliates 9
10. Now that you have an Account, let's sign in! Copyright © 2021, Oracle and/or its affiliates 10 First.Last@e-mail.com PasswordJustGiven
11. In the Main Window, let's Submit a Reservation request Copyright © 2021, Oracle and/or its affiliates 11 Make sure "Start Workshop Now" is ON Check the box and "Submit Reservation"
12. If you click on "View Your Reservations", you can see the specific one for OML4Py Workshop. Keep refreshing the page to get an actual update. Copyright © 2021, Oracle and/or its affiliates 12 This is your timer. It will take approximately 20 minutes for the environment to be ready, however actual time may be shorter
13. Copyright © 2021, Oracle and/or its affiliates 13 Quick Video about OML AutoML UI Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
14. Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
15. Copyright © 2021, Oracle and/or its affiliates 15 Introduction Just joining ? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us for help in Chat
16. Oracle Machine Learning OML Services OML4SQL OML4Py OML4R OML Notebooks OML4Spark Oracle Data Miner OML AutoML UI Interfaces for 3 popular data science languages: SQL, R, and Python Collaborative notebook environment based on Apache Zeppelin with Autonomous Database SQL Developer extension to create, schedule, and deploy ML solutions through a drag-and-drop interface ML for the big data environment from R with scalable algorithms Code-free AutoML interface on Autonomous Database Model Deployment and Management, Cognitive Text Copyright © 2021, Oracle and/or its affiliates 16
17. Oracle Machine Learning Key Attributes Automated Get better results faster with less effort – even non-expert users Scalable Handle big data volumes using parallel, distributed algorithms – no data movement Production-ready Deploy and update data science solutions faster with integrated ML platform Increase productivity, Achieve enterprise goals, Innovate more Copyright © 2021, Oracle and/or its affiliates 17
18. Oracle Machine Learning Key Attributes Automated Get better results faster with less effort – even non-expert users Scalable Handle big data volumes using parallel, distributed algorithms – no data movement Production-ready Deploy and update data science solutions faster with integrated ML platform Increase productivity, Achieve enterprise goals, Innovate more Copyright © 2021, Oracle and/or its affiliates 18
19. Autonomous Database as a Data Science Platform Collaborative UI • Based on Apache Zeppelin • Supports data scientists, data analysts, application developers, DBAs with SQL and Python • Easy notebook sharing • Permissions, versioning, and scheduling of notebooks Included with Autonomous Database • Automatically provisioned and managed • In-database algorithms and analytics functions • Explore and prepare, build and evaluate models, score data, deploy solutions Oracle Machine Learning Notebooks Copyright © 2021 Oracle and/or its affiliates.
20. Simplify the machine learning modeling and deployment process OML AutoML UI OML Model Data Copyright © 2021, Oracle and/or its affiliates 20 Auto Algorithm Selection • Identify in-database algorithms likely to achieve higher model quality • Find best algorithm faster than exhaustive search Adaptive Sampling • Identify right sample size for training data • Adjust sample for unbalanced data Auto Feature Selection • De-noise data • Reduce features by identifying most predictive • Improve accuracy and performance Auto Model Tuning • Improves model accuracy • Automated tuning of hyperparameters • Avoid manual or exhaustive search techniques OML AutoML UI Experiment Pipeline Feature Prediction Impact • Rank features most influential for scoring • Algorithm-agnostic technique • For each final model per algorithm Plus…
21. No-code AutoML-based user interface supporting automated machine learning Powerful, easy to use UI Automates model building, tuning, and deployment • Supports model management • Enhance data scientist productivity • Empower data professionals who are not ML experts Featuring • Minimal user input: data, target • Model leaderboard • Model deployment via REST endpoints • Generate OML4Py notebooks from models OML AutoML UI Copyright © 2021 Oracle and/or its affiliates. 21
22. No-code AutoML based UI supporting automated end-to-end ML 22 Copyright © 2021, Oracle and/or its affiliates OML AutoML UI AutoML
23. Comparing OML4Py AutoML with OML AutoML UI Copyright © 2021, Oracle and/or its affiliates 23 Step in workflow OML4Py AutoML API OML AutoML UI Algorithm Selection ü Optional use ü Adaptive Sampling ü Optional use ü Feature Selection ü Optional use ü Model Tuning ü ü Model Selection ü Specific API function to return top model or user selection ü Leaderboard ranks models by score metric for use choice Feature Prediction Impact ü Optional use via MLX ü Generate notebook for model Not available ü Integrated model deployment to OML Services Explicit model export and REST API import ü Manual pipeline assembly Experiment assembles the full pipeline
24. On Autonomous Database Live Labs – this is what we are using today • 2 OCPUs – no Auto-scale • Intended to illustrate functionality on small data sets Always Free Tier • 1 OCPU • No parallelism • Intended to illustrate functionality on small data sets Paid Tier • Customer-configured ADB instance size N OCPU • Optional auto-scale for 3x OCPU elastically allocated OML AutoML UI experiments include all steps in the workflow Defaults • 5 top models and all algorithms selected • On LOW database service level • 8-hour limit Tips to reduce runtime • Choose “Faster Results” over “Better Accuracy” if that’s your goal • Reduce # top models to 2 or 3 • Eliminate algorithms not of interest (if known) • If on PAID tier with OCPU > 1, increase database service level to MEDIUM or HIGH Performance considerations for OML AutoML UI Copyright © 2021, Oracle and/or its affiliates 24
25. OML expected workflow: data loading and preparation Copyright © 2021, Oracle and/or its affiliates 25 .CSV CSV Imported as Database Table Prepared Database Table Load / Access Explore and Prepare SQL OML Notebooks Database Actions: Transforms SQL Worksheet
26. OML expected workflow: ML model build, evaluation and deployment Copyright © 2021, Oracle and/or its affiliates 26 Prepared Database Table Model and Evaluate Deploy Generate notebook {REST:API} OML Services Enterprise Applications Deploy in-database model OML AutoML UI Build in-db model Export and deploy in-db model In-database SQL scoring In-database SQL scoring In-database SQL scoring Oracle APEX
27. Contact us via "Chat to the Panelists" if your environment is not ready yet, of if you have problems Quick check on the environment readiness Copyright © 2021, Oracle and/or its affiliates 27 Have you received the email indicating that your LiveLab environment is ready to use?
28. Check your Inbox for the e-mail from livelabs-noreply@oracle.com Copyright © 2021, Oracle and/or its affiliates 28 Make sure it says that the environment is ready! You can click here to go to the My Reservations page, or just refresh the page you have open at: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/my-reservations
29. In the "My Reservations" window (refresh the page periodically), after around 20 minutes, the Status will change to Available. Click on "Launch Workshop" to open this window. Copyright © 2021, Oracle and/or its affiliates 29
30. Copyright © 2021, Oracle and/or its affiliates 30 Labs overview
31. 31 Copyright © 2021, Oracle and/or its affiliates Lab high-level outline Lab 1: Access OML Notebooks and Create your First model using OML AutoML UI Lab 2: Create an auto-generated OML Notebook from your First model Lab 3: Deploy an AutoML UI model to REST API on OML Services Lab 4: Create a second Experiment with more models and a Recall model metric Lab 5: Run AutoML using OML4Py as a comparison Lab 6: Bonus Section: Use Postman to access OML Services REST APIs to score the OML AutoML UI model deployments
32. Copyright © 2021, Oracle and/or its affiliates 32 Lab 1: Access OML Notebooks and Create your First model using OML AutoML UI Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
33. Our first step is to get to the OML Notebooks and then to the OML AutoML UI: In the "My Reservations" window at: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/my-reservations Click on "Launch Workshop", then "Launch Console". You will need to use the Initial Password. Copyright © 2021, Oracle and/or its affiliates 33 2. We will need this password before the next step, so you should copy it 3. Launch the Console (the login will contain your Username LLXXXX- USER) 1. Launch the Workshop
34. In the Single Sign-On screen, enter the Password given in the Workshop screen for your user LL####-USER. You will need to change the Password (you can use the same), then click on the Hamburger Menu on the Top left of OCI, then Autonomous Database. Copyright © 2021, Oracle and/or its affiliates 34 2. Click the Hamburger Menu in the main OCI screen 3. Click on Autonomous Database Important note: remember to sign-out of all other OCI Account Tabs you might have open 1. Paste the Password copied here, and repeat it 3 more times when the "Change Password" menu appears
35. Select the specific ADW Compartment indicated in the Workshop Main screen, and then click on the Autonomous Database Copyright © 2021, Oracle and/or its affiliates 35 2. Click on the Autonomous Database Name assigned to you 1. Click and select the Compartment associated with your Account
36. Once inside the Autonomous Database, click on "Service Console". In there, select "Development" and then click on "Oracle Machine Learning Notebooks" Copyright © 2021, Oracle and/or its affiliates 36 1. Click on the Service Console 2. Click on Development 3. Click on OML Notebooks
37. In the Sign-In screen of OML Notebooks, use OMLUSER and the Password AAbbcc123456. Once you are in, click on the AutoML link Copyright © 2021, Oracle and/or its affiliates 37 Click on AutoML Sign in with OMLUSER and the Password AAbbcc123456
38. In the AutoML Experiments screen, click "Create" to create a new one Copyright © 2021, Oracle and/or its affiliates 38 1. Click on +Create 2. Give the Experiment a name 3. Click on the Loupe to search for a Data Source
39. In the menu that appears, select: Schema = SH & Table = SUPPLEMENTARY_DEMOGRAPHICS Copyright © 2021, Oracle and/or its affiliates 39 1. Click on the "SH" Schema 2. Select the "SUPPLEMENTARY_DEMOGRAPHICS" table and click the "OK" button
40. In the Create Experiments screen, enter the Predict, Case ID, and adjust performance settings Copyright © 2021, Oracle and/or its affiliates 40 1. For the "Predict" attribute, select AFFINITY_CARD from the pull-down menu 2. For the "Case ID" attribute, select CUST_ID from the pull- down menu 3. Expand the "Additional Settings" and reduce the Maximum Top Models to "2" to save time in this exercise. 4. Change the "Database Service Level" to Medium to get increased parallelism
41. We are ready to start the Experiment Click on the Start -> Faster Results Copyright © 2021, Oracle and/or its affiliates 41 1. Click on Start 2. Click on Faster Results to make sure we have a quick result for this Session 3. Experiment will start
42. Click on the three dots (…) to see the Progress Copyright © 2021, Oracle and/or its affiliates 42 Click on the three dots to open the Progress Report
43. Completed vs Running Steps Copyright © 2021, Oracle and/or its affiliates 43 Checkboxes are shown for completed steps, while running steps show an animation
44. Top algorithms selected by AutoML Copyright © 2021, Oracle and/or its affiliates 44 Only the Top algorithms identified in the "Algorithm Selection" phase are shown in the Leader Board
45. A completed Experiment looks like this Copyright © 2021, Oracle and/or its affiliates 45 The Experiment finishes when you see the "Completed" message at the top Checkboxes are shown for all completed steps
46. 2. After an experiment run is completed, the Features grid displays an additional column Importance. Feature Importance indicates the overall level of sensitivity of prediction to a particular feature. The value is always depicted in the range 0 to 1, with values closer to 1 being more important. Let's check the global Feature Importance Copyright © 2021, Oracle and/or its affiliates 46 1. Scroll down to reveal the Features list
47. Let's open the Model Detail to inspect the Model Copyright © 2021, Oracle and/or its affiliates 47 Click on the Naïve Bayes model name in blue
48. Review the Model Detail – Prediction Impacts Copyright © 2021, Oracle and/or its affiliates 48 A new window with the Model Details show the Prediction Impacts of the Attributes. It uses OML's Machine Learning Explainability module to provide model-agnostic functionality to identify the important features that impact a trained model’s predictions.
49. Review the Model Detail – Confusion Matrix Copyright © 2021, Oracle and/or its affiliates 49 The Confusion Matrix shows an evaluation of the Model on the Validation Data selected by AutoML at the end of the Process
50. Copyright © 2021, Oracle and/or its affiliates 50 Lab 2: Create an auto-generated OML Notebook from your First model Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
51. Auto-generation of an OML Notebook with OML4Py Let's see how to access a Notebook with the entire model building process Copyright © 2021, Oracle and/or its affiliates 51 1. Click anywhere in the area around the Model to highlight the it (the whole row will become a shade of blue), but not in exactly in the model name (in blue) 2. The Create Notebook button will become available. Click on it to get to the Create Notebook form.
52. Auto-generation of an OML Notebook with OML4Py Give the new Notebook a name Copyright © 2021, Oracle and/or its affiliates 52 Click on OK to Create the Notebook In the new "Create Notebook" window, give it a Short name
53. Auto-generation of an OML Notebook with OML4Py Give the new Notebook a name Copyright © 2021, Oracle and/or its affiliates 53 The message at the Top of the screen confirms that the Notebook was created successfully
54. Let's check the Notebook code We need to get to the Notebooks list Copyright © 2021, Oracle and/or its affiliates 54 1. Click on the "Hamburger Menu" on the top left corner of the screen. 2. It reveals the section in black. Select Notebooks from there
55. The auto-generated Notebook is available Let's access the Notebook listed Copyright © 2021, Oracle and/or its affiliates 55 Click on the Notebook name to open it
56. The Notebook is opened, but it has not been run yet Let's run it so we can see all statistics and the resulting output Copyright © 2021, Oracle and/or its affiliates 56 Click on the "Play" button to Run all paragraphs
57. Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 57 This first section is dedicated to mention the Metadata about this Notebook
58. Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 58 This section builds the Dataset the way the Model needs, only with the required columns
59. Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 59 This section splits the Target column from the Input Attributes, to prepare the Training data for the Model Using the Algorithm settings identified by AutoML to be the best ones for this model, we proceed to building the model (nb_mod.fit)
60. Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 60 This section shows the model details. You can scroll down inside the paragraph to see more information about the model Metadata This is just a notice to mention that the Prediction section will use the same input data for Scoring. You can provide a different Table for it, by using: my_scoring_data = oml.sync(table="MY_SCORING").
61. Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 61 This section runs a prediction on the original Build Data. It also selects only the PREDICTION column, and brings it into Python's memory for using with Sci-Kit Learn in the next paragraph This paragraph imports the Sci-Kit Learn package and uses the metrics.balanced_accuracy_score function to check the predictions quality if one were to use the same input data as Test Data. We expect a value similar to the one we see in the OML AutoML UI Leader Board, but it can be slightly different since that one is not computed over the entire dataset, but on a validation subset (or cross-validation) generated by AutoML
62. Bonus Round 1 – check the predictions and probabilities Add your own code to the Notebook to check the predictions, probabilities and Unique IDs Copyright © 2021, Oracle and/or its affiliates 62 Type this code on a new paragraph after running the entire Notebook, to check the Predictions and Probabilities without downloading the results into Python's local memory. We will use the special option for adding some columns from the Table being scored into the final dataset, and also check the Probability of that specific Prediction z.show( nb_mod.predict(build_data, proba=True, supplemental_cols=build_data[['CUST_ID','EDUCATION','HOUSEHOLD_SIZE']]) )
63. Bonus Round 2 – check the predictions and probabilities Add your own code to the Notebook to check the predictions, probabilities and Unique IDs Copyright © 2021, Oracle and/or its affiliates 63 Type this code on a new paragraph after running the entire Notebook, to check the Predictions and Probabilities without downloading the results into Python's local memory. We will use a different function, prediction_proba, that can return the probability for each of the categories in the Target (in our case "0" or "1"). z.show( nb_mod.predict_proba(build_data, supplemental_cols=build_data[['CUST_ID','EDUCATION','HOUSEHOLD_SIZE']]) )
64. Bonus Round 3 – check the predictions and probabilities Add your own code to the Notebook to check the predictions, probabilities and Unique IDs Copyright © 2021, Oracle and/or its affiliates 64 Type this code on a new paragraph after running the entire Notebook, to check the Predictions and Probabilities without downloading the results into Python's local memory. This option now uses the special topN_attrs option to show the Top Attributes that helped determine that specific customer's Predictions, also known as the Prediction Details. z.show( nb_mod.predict(build_data, proba=True, supplemental_cols=build_data['CUST_ID'], topN_attrs=2).round(4) )
65. Copyright © 2021, Oracle and/or its affiliates 65 Lab 3: Deploy an AutoML UI model to REST API on OML Services Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
66. Model deployment as a REST API Let's see how to deploy the models to OML Services from here. Copyright © 2021, Oracle and/or its affiliates 66 1. Click anywhere in the area around the ROW of the Model to highlight it (the whole row will become a shade of blue). Do not click on the model name itself (in blue), since it's a link to the Model Details 2. The Deploy button will be available. Click on it to get to the Deploy Model form.
67. Model deployment as a REST API Let's see how to deploy the models to OML Services from here. Copyright © 2021, Oracle and/or its affiliates 67 The Shared check box makes sure other users from the same ADB (PDB) also have access to the Deployment REST API. Click on OK to Register In the new "Deploy Model" window, add a name, URI, Version, and enter the Namespace OML_MODELS.
68. Model deployment as a REST API Confirmation that the model is deployed Copyright © 2021, Oracle and/or its affiliates 68 The message at the Top of the screen confirms that the Model was deployed successfully
69. Let's check the OML Models menu We need to get to the Models list Copyright © 2021, Oracle and/or its affiliates 69 1. Click on the "Hamburger Menu" on the top left corner of the screen. 2. It reveals the section in black. Select Models from there
70. Let's check the OML Models menu We can check all models currently registered with OML4SQL Copyright © 2021, Oracle and/or its affiliates 70 User Models shows all of the models created by the user, including models built manually (not via AutoML UI). These models have the standard unique model name given by AutoML UI, or a model name given by the user via OML4SQL or OML4Py.
71. Let's check the OML Deployments We can check deployments registered with OML Services as REST APIs (including ONNX) Copyright © 2021, Oracle and/or its affiliates 71 Deployments show all of the REST endpoints for models deployed by the user, or models deployed by teammates and registered as "Shared". This list would also include any model deployed to OML Services by hand and would include models in ONNX format as well.
72. Let's check the OML Deployments We can check deployments registered with OML Services as REST APIs (including ONNX) Copyright © 2021, Oracle and/or its affiliates 72 Clicking on the Deployment "Name", it will show you the basic features of the model metadata, including the Mining Function, algorithm used, input attributes and target information
73. Let's check the OML Deployments We can check deployments registered with OML Services as REST APIs (including ONNX) Copyright © 2021, Oracle and/or its affiliates 73 Clicking on the Deployment "URI", it will show you the full Open API specification of the model, including the address on where to make calls to the model, as a complete Swagger file
74. Copyright © 2021, Oracle and/or its affiliates 74 Lab 4: Create a second Experiment with more models and a Recall model metric Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
75. In the AutoML Experiments screen, click "Create" to create a new one Copyright © 2021, Oracle and/or its affiliates 75 1. Click on +Create 2. Give the Experiment a name 3. Click on the Loupe to search for a Data Source
76. In the menu that appears, select: Schema = SH & Table = SUPPLEMENTARY_DEMOGRAPHICS Copyright © 2021, Oracle and/or its affiliates 76 1. Click on the "SH" Schema 2. Select the "SUPPLEMENTARY_DEMOGRAPHICS" table and click the "OK" button
77. In the Create Experiments screen, enter the Predict, Case ID, and adjust performance settings Copyright © 2021, Oracle and/or its affiliates 77 1. For the "Predict" attribute, select AFFINITY_CARD from the pull-down menu 2. For the "Case ID" attribute, select CUST_ID from the pull- down menu 3. Expand the "Additional Settings" and leave the Maximum Top Models at "5" 4. Change the "Database Service Level" to High to get increased parallelism 5. Change the Model Metric to Recall -> Weighted
78. We are ready to start the Experiment Click on the Start -> Faster Results Copyright © 2021, Oracle and/or its affiliates 78 1. Click on Start 2. Click on Faster Results to make sure we have a quick result for this Session 3. Experiment will start
79. Click on the three dots (…) to see the Progress Copyright © 2021, Oracle and/or its affiliates 79 Click on the three dots to open the Progress Report
80. The completed Second Experiment should finish in around 4 minutes Copyright © 2021, Oracle and/or its affiliates 80
81. Let's open the Model Detail to inspect the RF Model Copyright © 2021, Oracle and/or its affiliates 81 Click on the Random Forest model name in blue
82. Review the Model Detail – Prediction Impacts Copyright © 2021, Oracle and/or its affiliates 82 A new window with the Model Details show the Prediction Impacts of the Attributes. It uses OML's Machine Learning Explainability module to provide model-agnostic functionality to identify the important features that impact a trained model’s predictions.
83. Review the Model Detail – Confusion Matrix Copyright © 2021, Oracle and/or its affiliates 83 The Confusion Matrix shows an evaluation of the Model on the Validation Data selected by AutoML at the end of the Process
84. Let's deploy the Random Forest model to OML Services Copyright © 2021, Oracle and/or its affiliates 84 Deploy the Model to OML Services
85. Let's create the Notebook for the Random Forest Model Copyright © 2021, Oracle and/or its affiliates 85 Create the auto- generated Notebook for the Random Forest
86. The Notebook looks like the following Copyright © 2021, Oracle and/or its affiliates 86 Optimized hyperparameters determined by AutoML
87. Copyright © 2021, Oracle and/or its affiliates 87 Lab 5: Run AutoML using OML4Py as a comparison Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
88. Back in the LiveLabs Main page (where all links and passwords are), scroll down to the bottom of the page and click "Open the workshop instructions in a new tab" Copyright © 2021, Oracle and/or its affiliates 88 New TAB with the Instructions 1. Collapse this section
89. Option 1: Download All Labs Notebooks in a single ZIP file from: http://www.oracle.com/a/tech/docs/otn-batch1/oml4py-hol-notebooks.zip Option 2: At each Lab, under the Optional Section, you can download a copy of each of the Notebooks to import them into your own. Another way is to copy and paste each paragraph on a new Notebook Let's import the Notebooks for the Labs Copyright © 2021, Oracle and/or its affiliates 89
90. Import the Notebooks into your OML Notebooks Session It should say 6 out of 6 notebooks imported successfully Copyright © 2021, Oracle and/or its affiliates 90 You can import multiple by selecting the 6 Notebook JSON files on your machine
91. Goal: Become familiar with the AutoML workflow and related functions Step 1: Import libraries supporting OML4Py Step 2: Automated Algorithm Selection Step 3: Automated Feature Selection Step 4: Automated Model Tuning Step 5: Automated Model Selection (the most similar to AutoML UI) Note: Some AutoML function invocations can take a few minutes to complete. A lot of going on behind the scenes. Please be patient. J Let's open the Notebook called "Lab 6 : Use AutoML" Copyright © 2021, Oracle and/or its affiliates 91
92. Open the Notebook named "Lab 6 : Use AutoML" Copyright © 2021, Oracle and/or its affiliates 92 1. Click on the Notebook "Lab 6: use AutoML" name
93. First task: change the interpreter binding to leave only the HIGH one Copyright © 2021, Oracle and/or its affiliates 93 1. Click on the Interpreter Binding gear icon at the Top on the right of the Notebook 2. Unselect the Low and Medium ones by just clicking on them and making sure they stay white in color 3. Click "Save"
94. Notebook Contents – Algorithm Selection Copyright © 2021, Oracle and/or its affiliates 94
95. Notebook Contents – Feature Selection Copyright © 2021, Oracle and/or its affiliates 95
96. Notebook Contents – Model Tuning Copyright © 2021, Oracle and/or its affiliates 96
97. Notebook Contents – Model Selection Copyright © 2021, Oracle and/or its affiliates 97
98. Copyright © 2021, Oracle and/or its affiliates 98 Lab 6: Bonus Section: Use Postman to access OML Services REST APIs to score the OML AutoML UI model deployments Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat
99. 99 Connectivity and use from Client Oracle Machine Learning Services architecture REST Client user/pass GET Token Token + Actions & Text/Objects GET POST DELETE Oracle Autonomous Database /omlusers PDB /omlmod OML Services Copyright © 2021, Oracle and/or its affiliates
100. Components with built-in Oracle Machine Learning Admin • Token using ADB user and password Generic • Metadata for all Versions: Version 1 Metadata • Open API Specification Deployment • Create Model Endpoint • Score Model using Endpoint • Endpoints • Endpoint Details • Open API Specification for Endpoint • Endpoint Repository • Store Model • Update Model Namespace • Models list • Model Info • Model Metadata • Model Content • Model Cognitive Text • Get Most Relevant Topics • Get Most Relevant Keywords • Get Summaries • Get Sentiments • Get Semantic Similarities • Numeric Features • Get Endpoints Oracle Machine Learning Services - Methods GET POST DELETE GET POST DELETE GET POST GET POST Copyright © 2021, Oracle and/or its affiliates 100
101. Copyright © 2021, Oracle and/or its affiliates 101 Lab 6 A: Installation and configuration of Postman client for REST API Installing the Desktop client Application to test the REST API collections
102. On your browser, go to https://www.postman.com/downloads/ , and click on the Orange Button to "Download the App" Copyright © 2021, Oracle and/or its affiliates 102 For Microsoft Windows For MacOS For Linux
103. As an example, in the MacOS it will download a ZIP file. Extract the zip file with a double-click, and the Postman.app will be uncompressed Copyright © 2021, Oracle and/or its affiliates 103 Download it On Mac OS, double click to Extract the App. On Windows you can run the Installer ".exe" On Mac OS, you can Drag the "Postman.app" to the Applications folder
104. Launch Postman on your Desktop Copyright © 2021, Oracle and/or its affiliates 104 For this Hands-on-Lab, we will NOT need to create an Account with Postman, so you can click on "Skip and go to the app" on either the MacOS or Windows version
105. First step is to Import the Collections and Environment that will be distributed via a link: https://www.oracle.com/a/tech/docs/otn-batch1/oml-services-hol-postman-collections.zip Launch Postman on your Desktop Copyright © 2021, Oracle and/or its affiliates 105 After downloading the files: 1 - Unzip the 7 JSON files 2 - Click on the "Import Button"
106. You can just drag and drop the Postman collection JSON files you unzipped into the Window Import the OML Services HOL Postman collections Copyright © 2021, Oracle and/or its affiliates 106 Drag the 7 files from your File Manager into the area indicated in Postman
107. You can just drag and drop the Postman collection JSON files you unzipped into the Window Import the OML Services HOL Postman collections Copyright © 2021, Oracle and/or its affiliates 107 Confirm you have 6 Collections and 1 Environment file, and click "Import"
108. Verify that you have the 6 collections on the Collections section Let's check that we have the collections imported Copyright © 2021, Oracle and/or its affiliates 108 Confirm you have 6 the Collections in the "Collections" Tab You can close this Yellow Notice too by clicking the "x"
109. If your Live Labs Session was created in Montreal under the Tenancy c4u04, all you have to do is to enter your Database Name and put it in here: Now let's setup the Postman Environment to your specific Live Labs Copyright © 2021, Oracle and/or its affiliates 109 1. Click on the "Environments" Tab, then select the "OML Services LiveLabs" 2. All we need to do is to copy the "Database Name" assigned to you from the Workshop UI into the Current value of "database", then Save it.
110. Click the pull-down menu to select your environment, otherwise you will get http errors. Back to the main Collection Tab, don't forget to select your Environment Copyright © 2021, Oracle and/or its affiliates 110 1. Click on the "Collections" Tab to go back to the main section for the Demo 2. Click on the pull-down menu next to the "No Environment" text, and select the "OML Services LiveLabs" environment settings
111. In case you have a different Data Center or want to use your own Tenancy, here are the settings for all OML Services endpoints The main REST URL for Oracle Autonomous Database, where the OML Services endpoints exist, looks like the following: omlserver/omlmod/v1/ (for version 1) And to obtain a token to access OML Services you need to request a Token with a valid ADB user and password, from the OML request endpoint at: omlserver/omlusers/tenants/tenant/databases/database where: • omlserver = OML Services server for your Autonomous Database, for example, for Ashburn: https://adb.us-ashburn-1.oraclecloud.com • tenant = Oracle Cloud Tenancy OCID (not to be confused with the ADB OCID), in the form of: OCID1.TENANCY.OC1..AAAAAAAAFCUE47PQMRF4VIG…… • database = Oracle Autonomous Database PDB name (or database name), for example: ADW3900 If your Tenancy is not c4u04, or your Data Center is not Montreal Copyright © 2021, Oracle and/or its affiliates 111
112. Copyright © 2021, Oracle and/or its affiliates 112 Lab 6B: Getting the Token for Authorization of OML Services REST requests
113. Initial call to get a Token and be able to access all other OML Services endpoints Request a Token To request a Token for accessing all other OML Services endpoints, you need a valid user and password for your Oracle Autonomous Database that has the proper grants as an OML Developer from the OML Administrator. For the following REST call, we will consider: omlserver/omlusers/tenants/tenant/databases/database = ADB_URL $ curl –I \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ –d '{"grant_type":"password", "username": "YourOMLuser", "password": "YourOMLpass"}'\ "ADB_URL/api/oauth2/v1/token" 1.1 OML Services – Request a Token POST Copyright © 2021, Oracle and/or its affiliates 113
114. Initial call to get a Token and be able to access all other OML Services endpoints Request a Token To request a Token for accessing all other OML Services endpoints, you need a valid user and password for your Oracle Autonomous Database that has the proper grants as an OML Developer from the OML Administrator, usually created by the OMLADMIN 1.1 OML Services – Request a Token POST Copyright © 2021, Oracle and/or its affiliates 114
115. Copyright © 2021, Oracle and/or its affiliates 115 Lab 6C: Scoring the AutoML UI models via REST APIs in Oracle Machine Learning Service
116. Call to list all the Scoring Endpoints currently active in OML Services repository Call the REST endpoint to check all current registered scoring endpoints By passing a valid token to this REST API, OML Services will return a list of all scoring endpoints currently available to the user (and any shared models). These models endpoints are the ones currently available for scoring. The result is a JSON with a list of the Scoring Endpoints with a lot of metadata about them and the uri of each model. For the following REST call, we will consider: omlserver/omlmod/v1 = OML_URL , and remember to provide the full Token after "Bearer" $ curl --location --request GET 'OML_URL/deployment' \ --header 'Authorization: Bearer eyJhbGciOiJSUzI.... ==' 3.11 OML Services – Repository – List Scoring Endpoints in OML Services Copyright © 2021, Oracle and/or its affiliates 116 GET
117. Call to list all the Scoring Endpoints currently active in OML Services repository Call the REST endpoint to check all current registered scoring endpoints By passing a valid token to this REST API, OML Services will return a list of all scoring endpoints currently available to the user (and any shared models). These models endpoints are the ones currently available for scoring. The result is a JSON with a list of the Scoring Endpoints with a lot of metadata about them and the uri of each model 3.11 OML Services – Repository – List Scoring Endpoints in OML Services Copyright © 2021, Oracle and/or its affiliates 117 GET
118. 3.11 Using the Token stored, listing the Model deployments Copyright © 2021, Oracle and/or its affiliates 118 Our OML AutoML UI Model deployments will be shown here
119. 3.11 Using the Token stored, listing the Model deployments Copyright © 2021, Oracle and/or its affiliates 119 Click on Visualize to see a better formatted table with the data
120. Call to score a record of data using an existing Scoring Endpoint uri in OML Services Call the REST endpoint to use a scoring endpoint uri to get the prediction on data By passing a valid token to this REST API, plus passing the data columns in the "data-raw" section, OML Services will return the result of the model scoring on the data sent. Multiple records can be sent in a Mini- batch as well. Additionally, for OML Models, you can also request the Prediction Details to be returned using the "topNdetails" optional setting. The result is a JSON with a the result of the numeric scoring and/or labels and probabilities (in case of classification models), plus any additional information requested, like the Prediction Details (optional). For the following REST call, we will consider: omlserver/omlmod/v1 = OML_URL , and remember to provide the full Token after "Bearer" $ curl --location --request POST 'OML_URL/deployment/MyOMLModelURI/score' \ --header 'Authorization: Bearer eyJhbGciOiJSUzI.... ==' \ --data-raw '{"topNdetails":5, \ "inputRecords": [ [ {"AGE":41, "BOOKKEEPING_APPLICATION": 1, "CUST_GENDER":"M", "CUST_MARITAL_STATUS":"NeverM", \ "EDUCATION":"HS-grad", "HOME_THEATER_PACKAGE":1, "HOUSEHOLD_SIZE":"4", "OCCUPATION":"Crafts", \ "YRS_RESIDENCE":6, "Y_BOX_GAMES":1 } ] }' 3.13 OML Services – Deployment – Scoring an OML Model in OML Services Copyright © 2021, Oracle and/or its affiliates 120 POST
121. Call to score a record of data using an existing Scoring Endpoint uri in OML Services Call the REST endpoint to use a scoring endpoint uri to get the prediction on data By passing a valid token to this REST API, plus passing the data columns in the "data-raw" section, OML Services will return the result of the model scoring on the data sent. Multiple records can be sent in a Mini-batch as well. Additionally, for OML Models, you can also request the Prediction Details to be returned using the "topNdetails" optional setting. The result is a JSON with a the result of the numeric scoring and/or labels and probabilities (in case of classification models), plus any additional information requested, like the Prediction Details (optional). 3.13 OML Services – Deployment – Scoring an OML Model in OML Services Copyright © 2021, Oracle and/or its affiliates 121 POST
122. 3.13 Change the URI at the top of the POST command to score our model Copyright © 2021, Oracle and/or its affiliates 122 Change this portion of the POST command with the proper URI you gave your model in the OML AutoML UI deployment form Do not change the Body portion
123. 3.13 Run the Scoring by clicking "Send" Copyright © 2021, Oracle and/or its affiliates 123 Click Send to run the Scoring These are the Prediction Probabilities of both labels from the single record we requested
124. 3.13 Visualize it better Copyright © 2021, Oracle and/or its affiliates 124 Click on Visualize for a better view
125. 3.13 Score the Second Experiment model Copyright © 2021, Oracle and/or its affiliates 125 If you replace the name with the second model deployed in OML AutoML UI, you can now score that model too
126. 3.13 Add Prediction Details to the Scoring Copyright © 2021, Oracle and/or its affiliates 126 To the same request, just add the "topNdetails":5, inside the bracket (just before "inputRecords") to see the Prediction Details
127. Call to score a record of data using an existing Scoring Endpoint uri in OML Services Example 3.14 Scoring an OML Model with data in a mini-batch (two records in this example) Example 3.19 Scoring an OML Model with data in a mini-batch (two records in this example) & Prediction Details 3.14 OML Services – Deployment – Scoring an OML Model in OML Services Copyright © 2021, Oracle and/or its affiliates 127 POST
128. Copyright © 2021, Oracle and/or its affiliates 128 Where to go from here?
129. For more information on OML AutoML UI Copyright © 2021, Oracle and/or its affiliates 129 Current Documentation available at: https://docs.oracle.com/en/database/oracle/machine-learning/oml-automl-ui/index.html
130. Helpful Links 130 ORACLE MACHINE LEARNING ON O.COM https://www.oracle.com/machine-learning OML TUTORIALS OML LiveLab: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?p180_id=560 OML4Py LiveLab: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=786 Interactive tour: https://docs.oracle.com/en/cloud/paas/autonomous-database/oml-tour OML OFFICE HOURS https://asktom.oracle.com/pls/apex/asktom.search?office=6801#sessionss ORACLE ANALYTICS CLOUD https://www.oracle.com/solutions/business-analytics/data-visualization/examples.html OML4PY ORACLE AUTOML UI OML SERVICES Oracle Machine Learning AutoML UI (2m video) Oracle Machine Learning Demonstration (6m video) OML AutoML UI Technical Brief Blog: Introducing Oracle Machine Learning AutoML UI Oracle Machine Learning Services (2m video) OML Services Technical Brief Oracle Machine Learning Services Documentation Blog: Introducing Oracle Machine Learning Services GitHub Repository with OML Services examples OML4Py (2m video) OML4Py Introduction (17m video) OML4Py Technical Brief OML4Py User’s Guide Blog: Introducing OML4Py GitHub Repository with Python notebooks
131. Sessions every Tuesday at 8AM Pacific Time Previous Sessions Oracle Machine Learning AskTom Office Hours sessions https://asktom.oracle.com/pls/apex/asktom.search?office=6801#sessionss 131 Machine Learning 101 sessions
132. On our GitHub, you can find: Copyright © 2021, Oracle and/or its affiliates 132 github.com/oracle/oracle-db-examples/tree/master/machine-learning • Example Notebooks in OML4SQL and OML4Python • SQL code examples for DB 18c, 19c and 21c • Labs folder with OML4Py HOL Labs • OML Services demos including Cognitive Text Demos, in PostMan collections
133. Q & A Copyright © 2021, Oracle and/or its affiliates 133
134. Thank you ! Copyright © 2021, Oracle and/or its affiliates 134