Slide 1

Slide 1 text

Oracle Machine Learning Office Hours Hands-on-Lab: OML AutoML UI OML User Interface for AutoML model building and deployment with Marcos Arancibia, Sherry LaMonica & Mark Hornick Product Management, Oracle Machine Learning oracle.com/machine-learning May 2021

Slide 2

Slide 2 text

• Test drive the new Oracle Machine Learning AutoML UI available with Oracle Autonomous Database • Explore different aspects of Automated Machine Learning model building with in-database machine learning models • Build models with OML AutoML UI and deploy them directly to OML Services • Auto-generate OML4Py notebooks that can build the top models, with all Hyperparameters and options already set • Learn about possible integrations with other Oracle Cloud Applications Goals Copyright © 2021, Oracle and/or its affiliates 2

Slide 3

Slide 3 text

• The main objective of this 90-minute Hands-on-Lab Session is to get you familiar with OML AutoML UI, to become self-sufficient with the Live Labs environment, and show many of the OML AutoML UI capabilities • These labs focus on the OML AutoML UI, but other OML components will be used in conjunction to show model scoring. • This series of labs is not intended as an introduction to machine learning or details of specific algorithms • To learn more about Machine Learning concepts, check out our recorded OML Office Hours ML 101/102 sessions at https://asktom.oracle.com/pls/apex/asktom.search?oh=6801 Setting expectations Copyright © 2021, Oracle and/or its affiliates 3

Slide 4

Slide 4 text

• Accessing your Live Labs instance • Introduction to OML AutoML UI • Lab overview • Work through the labs • Q&A throughout Agenda Copyright © 2021, Oracle and/or its affiliates 4

Slide 5

Slide 5 text

• Marcos Arancibia – Instructor • Mark Hornick– Expert • Sherry LaMonica – Expert Hands-on lab experts Need lab help? Input your questions into the Chat to the Panelists Have questions? Input your questions into the Q&A Please let us know if you have any problems All Links will be distributed via the Chat window

Slide 6

Slide 6 text

Copyright © 2021, Oracle and/or its affiliates 6 Accessing your Live Labs instance

Slide 7

Slide 7 text

If you ran any Hands-on-Lab session earlier today with a Live Labs reservation that is still running (with status "Available"), you need to delete it before joining a new one. Copyright © 2021, Oracle and/or its affiliates 7 Go to LiveLabs "My Reservations" and login with your Oracle Account at: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/my-reservations Click on the Trashcan icon to delete the Running reservation, and confirm you want to Delete it Note: If you try to run this LiveLabs while you have another one running (in Available state), you will get an error

Slide 8

Slide 8 text

Browse to the address below (in Firefox or Google Chrome), and click on the "Reserve Workshop on LiveLabs" (Green Button): Copyright © 2021, Oracle and/or its affiliates 8 https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=786

Slide 9

Slide 9 text

If you do not have a free Oracle.com account, you will be required to create one now. It is fast and free. Copyright © 2021, Oracle and/or its affiliates 9

Slide 10

Slide 10 text

Now that you have an Account, let's sign in! Copyright © 2021, Oracle and/or its affiliates 10 First.Last@e-mail.com PasswordJustGiven

Slide 11

Slide 11 text

In the Main Window, let's Submit a Reservation request Copyright © 2021, Oracle and/or its affiliates 11 Make sure "Start Workshop Now" is ON Check the box and "Submit Reservation"

Slide 12

Slide 12 text

If you click on "View Your Reservations", you can see the specific one for OML4Py Workshop. Keep refreshing the page to get an actual update. Copyright © 2021, Oracle and/or its affiliates 12 This is your timer. It will take approximately 20 minutes for the environment to be ready, however actual time may be shorter

Slide 13

Slide 13 text

Copyright © 2021, Oracle and/or its affiliates 13 Quick Video about OML AutoML UI Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 14

Slide 14 text

Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 15

Slide 15 text

Copyright © 2021, Oracle and/or its affiliates 15 Introduction Just joining ? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us for help in Chat

Slide 16

Slide 16 text

Oracle Machine Learning OML Services OML4SQL OML4Py OML4R OML Notebooks OML4Spark Oracle Data Miner OML AutoML UI Interfaces for 3 popular data science languages: SQL, R, and Python Collaborative notebook environment based on Apache Zeppelin with Autonomous Database SQL Developer extension to create, schedule, and deploy ML solutions through a drag-and-drop interface ML for the big data environment from R with scalable algorithms Code-free AutoML interface on Autonomous Database Model Deployment and Management, Cognitive Text Copyright © 2021, Oracle and/or its affiliates 16

Slide 17

Slide 17 text

Oracle Machine Learning Key Attributes Automated Get better results faster with less effort – even non-expert users Scalable Handle big data volumes using parallel, distributed algorithms – no data movement Production-ready Deploy and update data science solutions faster with integrated ML platform Increase productivity, Achieve enterprise goals, Innovate more Copyright © 2021, Oracle and/or its affiliates 17

Slide 18

Slide 18 text

Oracle Machine Learning Key Attributes Automated Get better results faster with less effort – even non-expert users Scalable Handle big data volumes using parallel, distributed algorithms – no data movement Production-ready Deploy and update data science solutions faster with integrated ML platform Increase productivity, Achieve enterprise goals, Innovate more Copyright © 2021, Oracle and/or its affiliates 18

Slide 19

Slide 19 text

Autonomous Database as a Data Science Platform Collaborative UI • Based on Apache Zeppelin • Supports data scientists, data analysts, application developers, DBAs with SQL and Python • Easy notebook sharing • Permissions, versioning, and scheduling of notebooks Included with Autonomous Database • Automatically provisioned and managed • In-database algorithms and analytics functions • Explore and prepare, build and evaluate models, score data, deploy solutions Oracle Machine Learning Notebooks Copyright © 2021 Oracle and/or its affiliates.

Slide 20

Slide 20 text

Simplify the machine learning modeling and deployment process OML AutoML UI OML Model Data Copyright © 2021, Oracle and/or its affiliates 20 Auto Algorithm Selection • Identify in-database algorithms likely to achieve higher model quality • Find best algorithm faster than exhaustive search Adaptive Sampling • Identify right sample size for training data • Adjust sample for unbalanced data Auto Feature Selection • De-noise data • Reduce features by identifying most predictive • Improve accuracy and performance Auto Model Tuning • Improves model accuracy • Automated tuning of hyperparameters • Avoid manual or exhaustive search techniques OML AutoML UI Experiment Pipeline Feature Prediction Impact • Rank features most influential for scoring • Algorithm-agnostic technique • For each final model per algorithm Plus…

Slide 21

Slide 21 text

No-code AutoML-based user interface supporting automated machine learning Powerful, easy to use UI Automates model building, tuning, and deployment • Supports model management • Enhance data scientist productivity • Empower data professionals who are not ML experts Featuring • Minimal user input: data, target • Model leaderboard • Model deployment via REST endpoints • Generate OML4Py notebooks from models OML AutoML UI Copyright © 2021 Oracle and/or its affiliates. 21

Slide 22

Slide 22 text

No-code AutoML based UI supporting automated end-to-end ML 22 Copyright © 2021, Oracle and/or its affiliates OML AutoML UI AutoML

Slide 23

Slide 23 text

Comparing OML4Py AutoML with OML AutoML UI Copyright © 2021, Oracle and/or its affiliates 23 Step in workflow OML4Py AutoML API OML AutoML UI Algorithm Selection ü Optional use ü Adaptive Sampling ü Optional use ü Feature Selection ü Optional use ü Model Tuning ü ü Model Selection ü Specific API function to return top model or user selection ü Leaderboard ranks models by score metric for use choice Feature Prediction Impact ü Optional use via MLX ü Generate notebook for model Not available ü Integrated model deployment to OML Services Explicit model export and REST API import ü Manual pipeline assembly Experiment assembles the full pipeline

Slide 24

Slide 24 text

On Autonomous Database Live Labs – this is what we are using today • 2 OCPUs – no Auto-scale • Intended to illustrate functionality on small data sets Always Free Tier • 1 OCPU • No parallelism • Intended to illustrate functionality on small data sets Paid Tier • Customer-configured ADB instance size N OCPU • Optional auto-scale for 3x OCPU elastically allocated OML AutoML UI experiments include all steps in the workflow Defaults • 5 top models and all algorithms selected • On LOW database service level • 8-hour limit Tips to reduce runtime • Choose “Faster Results” over “Better Accuracy” if that’s your goal • Reduce # top models to 2 or 3 • Eliminate algorithms not of interest (if known) • If on PAID tier with OCPU > 1, increase database service level to MEDIUM or HIGH Performance considerations for OML AutoML UI Copyright © 2021, Oracle and/or its affiliates 24

Slide 25

Slide 25 text

OML expected workflow: data loading and preparation Copyright © 2021, Oracle and/or its affiliates 25 .CSV CSV Imported as Database Table Prepared Database Table Load / Access Explore and Prepare SQL OML Notebooks Database Actions: Transforms SQL Worksheet

Slide 26

Slide 26 text

OML expected workflow: ML model build, evaluation and deployment Copyright © 2021, Oracle and/or its affiliates 26 Prepared Database Table Model and Evaluate Deploy Generate notebook {REST:API} OML Services Enterprise Applications Deploy in-database model OML AutoML UI Build in-db model Export and deploy in-db model In-database SQL scoring In-database SQL scoring In-database SQL scoring Oracle APEX

Slide 27

Slide 27 text

Contact us via "Chat to the Panelists" if your environment is not ready yet, of if you have problems Quick check on the environment readiness Copyright © 2021, Oracle and/or its affiliates 27 Have you received the email indicating that your LiveLab environment is ready to use?

Slide 28

Slide 28 text

Check your Inbox for the e-mail from livelabs-noreply@oracle.com Copyright © 2021, Oracle and/or its affiliates 28 Make sure it says that the environment is ready! You can click here to go to the My Reservations page, or just refresh the page you have open at: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/my-reservations

Slide 29

Slide 29 text

In the "My Reservations" window (refresh the page periodically), after around 20 minutes, the Status will change to Available. Click on "Launch Workshop" to open this window. Copyright © 2021, Oracle and/or its affiliates 29

Slide 30

Slide 30 text

Copyright © 2021, Oracle and/or its affiliates 30 Labs overview

Slide 31

Slide 31 text

31 Copyright © 2021, Oracle and/or its affiliates Lab high-level outline Lab 1: Access OML Notebooks and Create your First model using OML AutoML UI Lab 2: Create an auto-generated OML Notebook from your First model Lab 3: Deploy an AutoML UI model to REST API on OML Services Lab 4: Create a second Experiment with more models and a Recall model metric Lab 5: Run AutoML using OML4Py as a comparison Lab 6: Bonus Section: Use Postman to access OML Services REST APIs to score the OML AutoML UI model deployments

Slide 32

Slide 32 text

Copyright © 2021, Oracle and/or its affiliates 32 Lab 1: Access OML Notebooks and Create your First model using OML AutoML UI Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 33

Slide 33 text

Our first step is to get to the OML Notebooks and then to the OML AutoML UI: In the "My Reservations" window at: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/my-reservations Click on "Launch Workshop", then "Launch Console". You will need to use the Initial Password. Copyright © 2021, Oracle and/or its affiliates 33 2. We will need this password before the next step, so you should copy it 3. Launch the Console (the login will contain your Username LLXXXX- USER) 1. Launch the Workshop

Slide 34

Slide 34 text

In the Single Sign-On screen, enter the Password given in the Workshop screen for your user LL####-USER. You will need to change the Password (you can use the same), then click on the Hamburger Menu on the Top left of OCI, then Autonomous Database. Copyright © 2021, Oracle and/or its affiliates 34 2. Click the Hamburger Menu in the main OCI screen 3. Click on Autonomous Database Important note: remember to sign-out of all other OCI Account Tabs you might have open 1. Paste the Password copied here, and repeat it 3 more times when the "Change Password" menu appears

Slide 35

Slide 35 text

Select the specific ADW Compartment indicated in the Workshop Main screen, and then click on the Autonomous Database Copyright © 2021, Oracle and/or its affiliates 35 2. Click on the Autonomous Database Name assigned to you 1. Click and select the Compartment associated with your Account

Slide 36

Slide 36 text

Once inside the Autonomous Database, click on "Service Console". In there, select "Development" and then click on "Oracle Machine Learning Notebooks" Copyright © 2021, Oracle and/or its affiliates 36 1. Click on the Service Console 2. Click on Development 3. Click on OML Notebooks

Slide 37

Slide 37 text

In the Sign-In screen of OML Notebooks, use OMLUSER and the Password AAbbcc123456. Once you are in, click on the AutoML link Copyright © 2021, Oracle and/or its affiliates 37 Click on AutoML Sign in with OMLUSER and the Password AAbbcc123456

Slide 38

Slide 38 text

In the AutoML Experiments screen, click "Create" to create a new one Copyright © 2021, Oracle and/or its affiliates 38 1. Click on +Create 2. Give the Experiment a name 3. Click on the Loupe to search for a Data Source

Slide 39

Slide 39 text

In the menu that appears, select: Schema = SH & Table = SUPPLEMENTARY_DEMOGRAPHICS Copyright © 2021, Oracle and/or its affiliates 39 1. Click on the "SH" Schema 2. Select the "SUPPLEMENTARY_DEMOGRAPHICS" table and click the "OK" button

Slide 40

Slide 40 text

In the Create Experiments screen, enter the Predict, Case ID, and adjust performance settings Copyright © 2021, Oracle and/or its affiliates 40 1. For the "Predict" attribute, select AFFINITY_CARD from the pull-down menu 2. For the "Case ID" attribute, select CUST_ID from the pull- down menu 3. Expand the "Additional Settings" and reduce the Maximum Top Models to "2" to save time in this exercise. 4. Change the "Database Service Level" to Medium to get increased parallelism

Slide 41

Slide 41 text

We are ready to start the Experiment Click on the Start -> Faster Results Copyright © 2021, Oracle and/or its affiliates 41 1. Click on Start 2. Click on Faster Results to make sure we have a quick result for this Session 3. Experiment will start

Slide 42

Slide 42 text

Click on the three dots (…) to see the Progress Copyright © 2021, Oracle and/or its affiliates 42 Click on the three dots to open the Progress Report

Slide 43

Slide 43 text

Completed vs Running Steps Copyright © 2021, Oracle and/or its affiliates 43 Checkboxes are shown for completed steps, while running steps show an animation

Slide 44

Slide 44 text

Top algorithms selected by AutoML Copyright © 2021, Oracle and/or its affiliates 44 Only the Top algorithms identified in the "Algorithm Selection" phase are shown in the Leader Board

Slide 45

Slide 45 text

A completed Experiment looks like this Copyright © 2021, Oracle and/or its affiliates 45 The Experiment finishes when you see the "Completed" message at the top Checkboxes are shown for all completed steps

Slide 46

Slide 46 text

2. After an experiment run is completed, the Features grid displays an additional column Importance. Feature Importance indicates the overall level of sensitivity of prediction to a particular feature. The value is always depicted in the range 0 to 1, with values closer to 1 being more important. Let's check the global Feature Importance Copyright © 2021, Oracle and/or its affiliates 46 1. Scroll down to reveal the Features list

Slide 47

Slide 47 text

Let's open the Model Detail to inspect the Model Copyright © 2021, Oracle and/or its affiliates 47 Click on the Naïve Bayes model name in blue

Slide 48

Slide 48 text

Review the Model Detail – Prediction Impacts Copyright © 2021, Oracle and/or its affiliates 48 A new window with the Model Details show the Prediction Impacts of the Attributes. It uses OML's Machine Learning Explainability module to provide model-agnostic functionality to identify the important features that impact a trained model’s predictions.

Slide 49

Slide 49 text

Review the Model Detail – Confusion Matrix Copyright © 2021, Oracle and/or its affiliates 49 The Confusion Matrix shows an evaluation of the Model on the Validation Data selected by AutoML at the end of the Process

Slide 50

Slide 50 text

Copyright © 2021, Oracle and/or its affiliates 50 Lab 2: Create an auto-generated OML Notebook from your First model Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 51

Slide 51 text

Auto-generation of an OML Notebook with OML4Py Let's see how to access a Notebook with the entire model building process Copyright © 2021, Oracle and/or its affiliates 51 1. Click anywhere in the area around the Model to highlight the it (the whole row will become a shade of blue), but not in exactly in the model name (in blue) 2. The Create Notebook button will become available. Click on it to get to the Create Notebook form.

Slide 52

Slide 52 text

Auto-generation of an OML Notebook with OML4Py Give the new Notebook a name Copyright © 2021, Oracle and/or its affiliates 52 Click on OK to Create the Notebook In the new "Create Notebook" window, give it a Short name

Slide 53

Slide 53 text

Auto-generation of an OML Notebook with OML4Py Give the new Notebook a name Copyright © 2021, Oracle and/or its affiliates 53 The message at the Top of the screen confirms that the Notebook was created successfully

Slide 54

Slide 54 text

Let's check the Notebook code We need to get to the Notebooks list Copyright © 2021, Oracle and/or its affiliates 54 1. Click on the "Hamburger Menu" on the top left corner of the screen. 2. It reveals the section in black. Select Notebooks from there

Slide 55

Slide 55 text

The auto-generated Notebook is available Let's access the Notebook listed Copyright © 2021, Oracle and/or its affiliates 55 Click on the Notebook name to open it

Slide 56

Slide 56 text

The Notebook is opened, but it has not been run yet Let's run it so we can see all statistics and the resulting output Copyright © 2021, Oracle and/or its affiliates 56 Click on the "Play" button to Run all paragraphs

Slide 57

Slide 57 text

Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 57 This first section is dedicated to mention the Metadata about this Notebook

Slide 58

Slide 58 text

Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 58 This section builds the Dataset the way the Model needs, only with the required columns

Slide 59

Slide 59 text

Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 59 This section splits the Target column from the Input Attributes, to prepare the Training data for the Model Using the Algorithm settings identified by AutoML to be the best ones for this model, we proceed to building the model (nb_mod.fit)

Slide 60

Slide 60 text

Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 60 This section shows the model details. You can scroll down inside the paragraph to see more information about the model Metadata This is just a notice to mention that the Prediction section will use the same input data for Scoring. You can provide a different Table for it, by using: my_scoring_data = oml.sync(table="MY_SCORING").

Slide 61

Slide 61 text

Check the Notebook output We can now verify each paragraph Copyright © 2021, Oracle and/or its affiliates 61 This section runs a prediction on the original Build Data. It also selects only the PREDICTION column, and brings it into Python's memory for using with Sci-Kit Learn in the next paragraph This paragraph imports the Sci-Kit Learn package and uses the metrics.balanced_accuracy_score function to check the predictions quality if one were to use the same input data as Test Data. We expect a value similar to the one we see in the OML AutoML UI Leader Board, but it can be slightly different since that one is not computed over the entire dataset, but on a validation subset (or cross-validation) generated by AutoML

Slide 62

Slide 62 text

Bonus Round 1 – check the predictions and probabilities Add your own code to the Notebook to check the predictions, probabilities and Unique IDs Copyright © 2021, Oracle and/or its affiliates 62 Type this code on a new paragraph after running the entire Notebook, to check the Predictions and Probabilities without downloading the results into Python's local memory. We will use the special option for adding some columns from the Table being scored into the final dataset, and also check the Probability of that specific Prediction z.show( nb_mod.predict(build_data, proba=True, supplemental_cols=build_data[['CUST_ID','EDUCATION','HOUSEHOLD_SIZE']]) )

Slide 63

Slide 63 text

Bonus Round 2 – check the predictions and probabilities Add your own code to the Notebook to check the predictions, probabilities and Unique IDs Copyright © 2021, Oracle and/or its affiliates 63 Type this code on a new paragraph after running the entire Notebook, to check the Predictions and Probabilities without downloading the results into Python's local memory. We will use a different function, prediction_proba, that can return the probability for each of the categories in the Target (in our case "0" or "1"). z.show( nb_mod.predict_proba(build_data, supplemental_cols=build_data[['CUST_ID','EDUCATION','HOUSEHOLD_SIZE']]) )

Slide 64

Slide 64 text

Bonus Round 3 – check the predictions and probabilities Add your own code to the Notebook to check the predictions, probabilities and Unique IDs Copyright © 2021, Oracle and/or its affiliates 64 Type this code on a new paragraph after running the entire Notebook, to check the Predictions and Probabilities without downloading the results into Python's local memory. This option now uses the special topN_attrs option to show the Top Attributes that helped determine that specific customer's Predictions, also known as the Prediction Details. z.show( nb_mod.predict(build_data, proba=True, supplemental_cols=build_data['CUST_ID'], topN_attrs=2).round(4) )

Slide 65

Slide 65 text

Copyright © 2021, Oracle and/or its affiliates 65 Lab 3: Deploy an AutoML UI model to REST API on OML Services Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 66

Slide 66 text

Model deployment as a REST API Let's see how to deploy the models to OML Services from here. Copyright © 2021, Oracle and/or its affiliates 66 1. Click anywhere in the area around the ROW of the Model to highlight it (the whole row will become a shade of blue). Do not click on the model name itself (in blue), since it's a link to the Model Details 2. The Deploy button will be available. Click on it to get to the Deploy Model form.

Slide 67

Slide 67 text

Model deployment as a REST API Let's see how to deploy the models to OML Services from here. Copyright © 2021, Oracle and/or its affiliates 67 The Shared check box makes sure other users from the same ADB (PDB) also have access to the Deployment REST API. Click on OK to Register In the new "Deploy Model" window, add a name, URI, Version, and enter the Namespace OML_MODELS.

Slide 68

Slide 68 text

Model deployment as a REST API Confirmation that the model is deployed Copyright © 2021, Oracle and/or its affiliates 68 The message at the Top of the screen confirms that the Model was deployed successfully

Slide 69

Slide 69 text

Let's check the OML Models menu We need to get to the Models list Copyright © 2021, Oracle and/or its affiliates 69 1. Click on the "Hamburger Menu" on the top left corner of the screen. 2. It reveals the section in black. Select Models from there

Slide 70

Slide 70 text

Let's check the OML Models menu We can check all models currently registered with OML4SQL Copyright © 2021, Oracle and/or its affiliates 70 User Models shows all of the models created by the user, including models built manually (not via AutoML UI). These models have the standard unique model name given by AutoML UI, or a model name given by the user via OML4SQL or OML4Py.

Slide 71

Slide 71 text

Let's check the OML Deployments We can check deployments registered with OML Services as REST APIs (including ONNX) Copyright © 2021, Oracle and/or its affiliates 71 Deployments show all of the REST endpoints for models deployed by the user, or models deployed by teammates and registered as "Shared". This list would also include any model deployed to OML Services by hand and would include models in ONNX format as well.

Slide 72

Slide 72 text

Let's check the OML Deployments We can check deployments registered with OML Services as REST APIs (including ONNX) Copyright © 2021, Oracle and/or its affiliates 72 Clicking on the Deployment "Name", it will show you the basic features of the model metadata, including the Mining Function, algorithm used, input attributes and target information

Slide 73

Slide 73 text

Let's check the OML Deployments We can check deployments registered with OML Services as REST APIs (including ONNX) Copyright © 2021, Oracle and/or its affiliates 73 Clicking on the Deployment "URI", it will show you the full Open API specification of the model, including the address on where to make calls to the model, as a complete Swagger file

Slide 74

Slide 74 text

Copyright © 2021, Oracle and/or its affiliates 74 Lab 4: Create a second Experiment with more models and a Recall model metric Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 75

Slide 75 text

In the AutoML Experiments screen, click "Create" to create a new one Copyright © 2021, Oracle and/or its affiliates 75 1. Click on +Create 2. Give the Experiment a name 3. Click on the Loupe to search for a Data Source

Slide 76

Slide 76 text

In the menu that appears, select: Schema = SH & Table = SUPPLEMENTARY_DEMOGRAPHICS Copyright © 2021, Oracle and/or its affiliates 76 1. Click on the "SH" Schema 2. Select the "SUPPLEMENTARY_DEMOGRAPHICS" table and click the "OK" button

Slide 77

Slide 77 text

In the Create Experiments screen, enter the Predict, Case ID, and adjust performance settings Copyright © 2021, Oracle and/or its affiliates 77 1. For the "Predict" attribute, select AFFINITY_CARD from the pull-down menu 2. For the "Case ID" attribute, select CUST_ID from the pull- down menu 3. Expand the "Additional Settings" and leave the Maximum Top Models at "5" 4. Change the "Database Service Level" to High to get increased parallelism 5. Change the Model Metric to Recall -> Weighted

Slide 78

Slide 78 text

We are ready to start the Experiment Click on the Start -> Faster Results Copyright © 2021, Oracle and/or its affiliates 78 1. Click on Start 2. Click on Faster Results to make sure we have a quick result for this Session 3. Experiment will start

Slide 79

Slide 79 text

Click on the three dots (…) to see the Progress Copyright © 2021, Oracle and/or its affiliates 79 Click on the three dots to open the Progress Report

Slide 80

Slide 80 text

The completed Second Experiment should finish in around 4 minutes Copyright © 2021, Oracle and/or its affiliates 80

Slide 81

Slide 81 text

Let's open the Model Detail to inspect the RF Model Copyright © 2021, Oracle and/or its affiliates 81 Click on the Random Forest model name in blue

Slide 82

Slide 82 text

Review the Model Detail – Prediction Impacts Copyright © 2021, Oracle and/or its affiliates 82 A new window with the Model Details show the Prediction Impacts of the Attributes. It uses OML's Machine Learning Explainability module to provide model-agnostic functionality to identify the important features that impact a trained model’s predictions.

Slide 83

Slide 83 text

Review the Model Detail – Confusion Matrix Copyright © 2021, Oracle and/or its affiliates 83 The Confusion Matrix shows an evaluation of the Model on the Validation Data selected by AutoML at the end of the Process

Slide 84

Slide 84 text

Let's deploy the Random Forest model to OML Services Copyright © 2021, Oracle and/or its affiliates 84 Deploy the Model to OML Services

Slide 85

Slide 85 text

Let's create the Notebook for the Random Forest Model Copyright © 2021, Oracle and/or its affiliates 85 Create the auto- generated Notebook for the Random Forest

Slide 86

Slide 86 text

The Notebook looks like the following Copyright © 2021, Oracle and/or its affiliates 86 Optimized hyperparameters determined by AutoML

Slide 87

Slide 87 text

Copyright © 2021, Oracle and/or its affiliates 87 Lab 5: Run AutoML using OML4Py as a comparison Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 88

Slide 88 text

Back in the LiveLabs Main page (where all links and passwords are), scroll down to the bottom of the page and click "Open the workshop instructions in a new tab" Copyright © 2021, Oracle and/or its affiliates 88 New TAB with the Instructions 1. Collapse this section

Slide 89

Slide 89 text

Option 1: Download All Labs Notebooks in a single ZIP file from: http://www.oracle.com/a/tech/docs/otn-batch1/oml4py-hol-notebooks.zip Option 2: At each Lab, under the Optional Section, you can download a copy of each of the Notebooks to import them into your own. Another way is to copy and paste each paragraph on a new Notebook Let's import the Notebooks for the Labs Copyright © 2021, Oracle and/or its affiliates 89

Slide 90

Slide 90 text

Import the Notebooks into your OML Notebooks Session It should say 6 out of 6 notebooks imported successfully Copyright © 2021, Oracle and/or its affiliates 90 You can import multiple by selecting the 6 Notebook JSON files on your machine

Slide 91

Slide 91 text

Goal: Become familiar with the AutoML workflow and related functions Step 1: Import libraries supporting OML4Py Step 2: Automated Algorithm Selection Step 3: Automated Feature Selection Step 4: Automated Model Tuning Step 5: Automated Model Selection (the most similar to AutoML UI) Note: Some AutoML function invocations can take a few minutes to complete. A lot of going on behind the scenes. Please be patient. J Let's open the Notebook called "Lab 6 : Use AutoML" Copyright © 2021, Oracle and/or its affiliates 91

Slide 92

Slide 92 text

Open the Notebook named "Lab 6 : Use AutoML" Copyright © 2021, Oracle and/or its affiliates 92 1. Click on the Notebook "Lab 6: use AutoML" name

Slide 93

Slide 93 text

First task: change the interpreter binding to leave only the HIGH one Copyright © 2021, Oracle and/or its affiliates 93 1. Click on the Interpreter Binding gear icon at the Top on the right of the Notebook 2. Unselect the Low and Medium ones by just clicking on them and making sure they stay white in color 3. Click "Save"

Slide 94

Slide 94 text

Notebook Contents – Algorithm Selection Copyright © 2021, Oracle and/or its affiliates 94

Slide 95

Slide 95 text

Notebook Contents – Feature Selection Copyright © 2021, Oracle and/or its affiliates 95

Slide 96

Slide 96 text

Notebook Contents – Model Tuning Copyright © 2021, Oracle and/or its affiliates 96

Slide 97

Slide 97 text

Notebook Contents – Model Selection Copyright © 2021, Oracle and/or its affiliates 97

Slide 98

Slide 98 text

Copyright © 2021, Oracle and/or its affiliates 98 Lab 6: Bonus Section: Use Postman to access OML Services REST APIs to score the OML AutoML UI model deployments Technical questions? Make sure to follow the Instructions in the PDF (link in Zoom's Chat window), and Ask us in Chat

Slide 99

Slide 99 text

99 Connectivity and use from Client Oracle Machine Learning Services architecture REST Client user/pass GET Token Token + Actions & Text/Objects GET POST DELETE Oracle Autonomous Database /omlusers PDB /omlmod OML Services Copyright © 2021, Oracle and/or its affiliates

Slide 100

Slide 100 text

Components with built-in Oracle Machine Learning Admin • Token using ADB user and password Generic • Metadata for all Versions: Version 1 Metadata • Open API Specification Deployment • Create Model Endpoint • Score Model using Endpoint • Endpoints • Endpoint Details • Open API Specification for Endpoint • Endpoint Repository • Store Model • Update Model Namespace • Models list • Model Info • Model Metadata • Model Content • Model Cognitive Text • Get Most Relevant Topics • Get Most Relevant Keywords • Get Summaries • Get Sentiments • Get Semantic Similarities • Numeric Features • Get Endpoints Oracle Machine Learning Services - Methods GET POST DELETE GET POST DELETE GET POST GET POST Copyright © 2021, Oracle and/or its affiliates 100

Slide 101

Slide 101 text

Copyright © 2021, Oracle and/or its affiliates 101 Lab 6 A: Installation and configuration of Postman client for REST API Installing the Desktop client Application to test the REST API collections

Slide 102

Slide 102 text

On your browser, go to https://www.postman.com/downloads/ , and click on the Orange Button to "Download the App" Copyright © 2021, Oracle and/or its affiliates 102 For Microsoft Windows For MacOS For Linux

Slide 103

Slide 103 text

As an example, in the MacOS it will download a ZIP file. Extract the zip file with a double-click, and the Postman.app will be uncompressed Copyright © 2021, Oracle and/or its affiliates 103 Download it On Mac OS, double click to Extract the App. On Windows you can run the Installer ".exe" On Mac OS, you can Drag the "Postman.app" to the Applications folder

Slide 104

Slide 104 text

Launch Postman on your Desktop Copyright © 2021, Oracle and/or its affiliates 104 For this Hands-on-Lab, we will NOT need to create an Account with Postman, so you can click on "Skip and go to the app" on either the MacOS or Windows version

Slide 105

Slide 105 text

First step is to Import the Collections and Environment that will be distributed via a link: https://www.oracle.com/a/tech/docs/otn-batch1/oml-services-hol-postman-collections.zip Launch Postman on your Desktop Copyright © 2021, Oracle and/or its affiliates 105 After downloading the files: 1 - Unzip the 7 JSON files 2 - Click on the "Import Button"

Slide 106

Slide 106 text

You can just drag and drop the Postman collection JSON files you unzipped into the Window Import the OML Services HOL Postman collections Copyright © 2021, Oracle and/or its affiliates 106 Drag the 7 files from your File Manager into the area indicated in Postman

Slide 107

Slide 107 text

You can just drag and drop the Postman collection JSON files you unzipped into the Window Import the OML Services HOL Postman collections Copyright © 2021, Oracle and/or its affiliates 107 Confirm you have 6 Collections and 1 Environment file, and click "Import"

Slide 108

Slide 108 text

Verify that you have the 6 collections on the Collections section Let's check that we have the collections imported Copyright © 2021, Oracle and/or its affiliates 108 Confirm you have 6 the Collections in the "Collections" Tab You can close this Yellow Notice too by clicking the "x"

Slide 109

Slide 109 text

If your Live Labs Session was created in Montreal under the Tenancy c4u04, all you have to do is to enter your Database Name and put it in here: Now let's setup the Postman Environment to your specific Live Labs Copyright © 2021, Oracle and/or its affiliates 109 1. Click on the "Environments" Tab, then select the "OML Services LiveLabs" 2. All we need to do is to copy the "Database Name" assigned to you from the Workshop UI into the Current value of "database", then Save it.

Slide 110

Slide 110 text

Click the pull-down menu to select your environment, otherwise you will get http errors. Back to the main Collection Tab, don't forget to select your Environment Copyright © 2021, Oracle and/or its affiliates 110 1. Click on the "Collections" Tab to go back to the main section for the Demo 2. Click on the pull-down menu next to the "No Environment" text, and select the "OML Services LiveLabs" environment settings

Slide 111

Slide 111 text

In case you have a different Data Center or want to use your own Tenancy, here are the settings for all OML Services endpoints The main REST URL for Oracle Autonomous Database, where the OML Services endpoints exist, looks like the following: omlserver/omlmod/v1/ (for version 1) And to obtain a token to access OML Services you need to request a Token with a valid ADB user and password, from the OML request endpoint at: omlserver/omlusers/tenants/tenant/databases/database where: • omlserver = OML Services server for your Autonomous Database, for example, for Ashburn: https://adb.us-ashburn-1.oraclecloud.com • tenant = Oracle Cloud Tenancy OCID (not to be confused with the ADB OCID), in the form of: OCID1.TENANCY.OC1..AAAAAAAAFCUE47PQMRF4VIG…… • database = Oracle Autonomous Database PDB name (or database name), for example: ADW3900 If your Tenancy is not c4u04, or your Data Center is not Montreal Copyright © 2021, Oracle and/or its affiliates 111

Slide 112

Slide 112 text

Copyright © 2021, Oracle and/or its affiliates 112 Lab 6B: Getting the Token for Authorization of OML Services REST requests

Slide 113

Slide 113 text

Initial call to get a Token and be able to access all other OML Services endpoints Request a Token To request a Token for accessing all other OML Services endpoints, you need a valid user and password for your Oracle Autonomous Database that has the proper grants as an OML Developer from the OML Administrator. For the following REST call, we will consider: omlserver/omlusers/tenants/tenant/databases/database = ADB_URL $ curl –I \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ –d '{"grant_type":"password", "username": "YourOMLuser", "password": "YourOMLpass"}'\ "ADB_URL/api/oauth2/v1/token" 1.1 OML Services – Request a Token POST Copyright © 2021, Oracle and/or its affiliates 113

Slide 114

Slide 114 text

Initial call to get a Token and be able to access all other OML Services endpoints Request a Token To request a Token for accessing all other OML Services endpoints, you need a valid user and password for your Oracle Autonomous Database that has the proper grants as an OML Developer from the OML Administrator, usually created by the OMLADMIN 1.1 OML Services – Request a Token POST Copyright © 2021, Oracle and/or its affiliates 114

Slide 115

Slide 115 text

Copyright © 2021, Oracle and/or its affiliates 115 Lab 6C: Scoring the AutoML UI models via REST APIs in Oracle Machine Learning Service

Slide 116

Slide 116 text

Call to list all the Scoring Endpoints currently active in OML Services repository Call the REST endpoint to check all current registered scoring endpoints By passing a valid token to this REST API, OML Services will return a list of all scoring endpoints currently available to the user (and any shared models). These models endpoints are the ones currently available for scoring. The result is a JSON with a list of the Scoring Endpoints with a lot of metadata about them and the uri of each model. For the following REST call, we will consider: omlserver/omlmod/v1 = OML_URL , and remember to provide the full Token after "Bearer" $ curl --location --request GET 'OML_URL/deployment' \ --header 'Authorization: Bearer eyJhbGciOiJSUzI.... ==' 3.11 OML Services – Repository – List Scoring Endpoints in OML Services Copyright © 2021, Oracle and/or its affiliates 116 GET

Slide 117

Slide 117 text

Call to list all the Scoring Endpoints currently active in OML Services repository Call the REST endpoint to check all current registered scoring endpoints By passing a valid token to this REST API, OML Services will return a list of all scoring endpoints currently available to the user (and any shared models). These models endpoints are the ones currently available for scoring. The result is a JSON with a list of the Scoring Endpoints with a lot of metadata about them and the uri of each model 3.11 OML Services – Repository – List Scoring Endpoints in OML Services Copyright © 2021, Oracle and/or its affiliates 117 GET

Slide 118

Slide 118 text

3.11 Using the Token stored, listing the Model deployments Copyright © 2021, Oracle and/or its affiliates 118 Our OML AutoML UI Model deployments will be shown here

Slide 119

Slide 119 text

3.11 Using the Token stored, listing the Model deployments Copyright © 2021, Oracle and/or its affiliates 119 Click on Visualize to see a better formatted table with the data

Slide 120

Slide 120 text

Call to score a record of data using an existing Scoring Endpoint uri in OML Services Call the REST endpoint to use a scoring endpoint uri to get the prediction on data By passing a valid token to this REST API, plus passing the data columns in the "data-raw" section, OML Services will return the result of the model scoring on the data sent. Multiple records can be sent in a Mini- batch as well. Additionally, for OML Models, you can also request the Prediction Details to be returned using the "topNdetails" optional setting. The result is a JSON with a the result of the numeric scoring and/or labels and probabilities (in case of classification models), plus any additional information requested, like the Prediction Details (optional). For the following REST call, we will consider: omlserver/omlmod/v1 = OML_URL , and remember to provide the full Token after "Bearer" $ curl --location --request POST 'OML_URL/deployment/MyOMLModelURI/score' \ --header 'Authorization: Bearer eyJhbGciOiJSUzI.... ==' \ --data-raw '{"topNdetails":5, \ "inputRecords": [ [ {"AGE":41, "BOOKKEEPING_APPLICATION": 1, "CUST_GENDER":"M", "CUST_MARITAL_STATUS":"NeverM", \ "EDUCATION":"HS-grad", "HOME_THEATER_PACKAGE":1, "HOUSEHOLD_SIZE":"4", "OCCUPATION":"Crafts", \ "YRS_RESIDENCE":6, "Y_BOX_GAMES":1 } ] }' 3.13 OML Services – Deployment – Scoring an OML Model in OML Services Copyright © 2021, Oracle and/or its affiliates 120 POST

Slide 121

Slide 121 text

Call to score a record of data using an existing Scoring Endpoint uri in OML Services Call the REST endpoint to use a scoring endpoint uri to get the prediction on data By passing a valid token to this REST API, plus passing the data columns in the "data-raw" section, OML Services will return the result of the model scoring on the data sent. Multiple records can be sent in a Mini-batch as well. Additionally, for OML Models, you can also request the Prediction Details to be returned using the "topNdetails" optional setting. The result is a JSON with a the result of the numeric scoring and/or labels and probabilities (in case of classification models), plus any additional information requested, like the Prediction Details (optional). 3.13 OML Services – Deployment – Scoring an OML Model in OML Services Copyright © 2021, Oracle and/or its affiliates 121 POST

Slide 122

Slide 122 text

3.13 Change the URI at the top of the POST command to score our model Copyright © 2021, Oracle and/or its affiliates 122 Change this portion of the POST command with the proper URI you gave your model in the OML AutoML UI deployment form Do not change the Body portion

Slide 123

Slide 123 text

3.13 Run the Scoring by clicking "Send" Copyright © 2021, Oracle and/or its affiliates 123 Click Send to run the Scoring These are the Prediction Probabilities of both labels from the single record we requested

Slide 124

Slide 124 text

3.13 Visualize it better Copyright © 2021, Oracle and/or its affiliates 124 Click on Visualize for a better view

Slide 125

Slide 125 text

3.13 Score the Second Experiment model Copyright © 2021, Oracle and/or its affiliates 125 If you replace the name with the second model deployed in OML AutoML UI, you can now score that model too

Slide 126

Slide 126 text

3.13 Add Prediction Details to the Scoring Copyright © 2021, Oracle and/or its affiliates 126 To the same request, just add the "topNdetails":5, inside the bracket (just before "inputRecords") to see the Prediction Details

Slide 127

Slide 127 text

Call to score a record of data using an existing Scoring Endpoint uri in OML Services Example 3.14 Scoring an OML Model with data in a mini-batch (two records in this example) Example 3.19 Scoring an OML Model with data in a mini-batch (two records in this example) & Prediction Details 3.14 OML Services – Deployment – Scoring an OML Model in OML Services Copyright © 2021, Oracle and/or its affiliates 127 POST

Slide 128

Slide 128 text

Copyright © 2021, Oracle and/or its affiliates 128 Where to go from here?

Slide 129

Slide 129 text

For more information on OML AutoML UI Copyright © 2021, Oracle and/or its affiliates 129 Current Documentation available at: https://docs.oracle.com/en/database/oracle/machine-learning/oml-automl-ui/index.html

Slide 130

Slide 130 text

Helpful Links 130 ORACLE MACHINE LEARNING ON O.COM https://www.oracle.com/machine-learning OML TUTORIALS OML LiveLab: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?p180_id=560 OML4Py LiveLab: https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=786 Interactive tour: https://docs.oracle.com/en/cloud/paas/autonomous-database/oml-tour OML OFFICE HOURS https://asktom.oracle.com/pls/apex/asktom.search?office=6801#sessionss ORACLE ANALYTICS CLOUD https://www.oracle.com/solutions/business-analytics/data-visualization/examples.html OML4PY ORACLE AUTOML UI OML SERVICES Oracle Machine Learning AutoML UI (2m video) Oracle Machine Learning Demonstration (6m video) OML AutoML UI Technical Brief Blog: Introducing Oracle Machine Learning AutoML UI Oracle Machine Learning Services (2m video) OML Services Technical Brief Oracle Machine Learning Services Documentation Blog: Introducing Oracle Machine Learning Services GitHub Repository with OML Services examples OML4Py (2m video) OML4Py Introduction (17m video) OML4Py Technical Brief OML4Py User’s Guide Blog: Introducing OML4Py GitHub Repository with Python notebooks

Slide 131

Slide 131 text

Sessions every Tuesday at 8AM Pacific Time Previous Sessions Oracle Machine Learning AskTom Office Hours sessions https://asktom.oracle.com/pls/apex/asktom.search?office=6801#sessionss 131 Machine Learning 101 sessions

Slide 132

Slide 132 text

On our GitHub, you can find: Copyright © 2021, Oracle and/or its affiliates 132 github.com/oracle/oracle-db-examples/tree/master/machine-learning • Example Notebooks in OML4SQL and OML4Python • SQL code examples for DB 18c, 19c and 21c • Labs folder with OML4Py HOL Labs • OML Services demos including Cognitive Text Demos, in PostMan collections

Slide 133

Slide 133 text

Q & A Copyright © 2021, Oracle and/or its affiliates 133

Slide 134

Slide 134 text

Thank you ! Copyright © 2021, Oracle and/or its affiliates 134

Slide 135

Slide 135 text

No content