Tweet
Tweet

Slide 1

Slide 1 text

Validating your Machine learning models & Data with minimal effort using Deepchecks

Slide 2

Slide 2 text

Who am I? - Data Scientist. - Founder of The African Data Community Newsletter. - Technical Writer, Public Speaker, Social Media Content creator, Machine Learning Thought Leader (Global AI Hub), Open- source & Community Advocate. Gift Ojeabulu CBBAnalytics @GiftOjeabulu_

Slide 3

Slide 3 text

- What is Deepchecks? - When should I use Deepchecks? - Deepchecks validation structure. - When should I validate? - Use case. - Conclusion. Learning Objective

Slide 4

Slide 4 text

Let’s discuss Data validation & Model validation?

Slide 5

Slide 5 text

Validating the accuracy, clarity, and details of data is necessary to mitigate any project defects. Without validating data, you risk basing decisions on data with imperfections that are not accurately representative of the situation at hand.

Slide 6

Slide 6 text

What is DeepChecks?

Slide 7

Slide 7 text

Deep Checks is the leading tool for validating your machine learning models and data, and it enables doing so with minimal effort. Deep Checks accompany you through various validation needs such as: - Verifying your Data Integrity. - Inspect its distribution. - Validating data splits. - Evaluating your model & comparing different models.

Slide 8

Slide 8 text

• Building the ML Model • Data: Collection, Preparation, Processing • Model: Evaluation, Analysis > 80% < 20%

Slide 9

Slide 9 text

When should I validate?

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Typical Validation Scenarios 1. When you start working with a new dataset: Validate New Data. 2. When you split the data (before training / various cross-validation split / hold-out test set/ …): Validate the Split. 3. When you evaluate a model: Validate Model Performance.

Slide 12

Slide 12 text

12 Types of Issues – What Can Go Wrong? Data Integrity Methodological Flaws Model Performance Fairness & Bias Data Distribution 9

Slide 13

Slide 13 text

Deepchecks Validation Structure

Slide 14

Slide 14 text

Validation Structure Test Suites Check s Built-in or Custom Display and Result 13 Condition s Pass / Fail / Warning

Slide 15

Slide 15 text

Validation with Deepchecks Checks Condition s Pass / Fail / Warning Test Suites https://github.com/deepchecks/deepchecs

Slide 16

Slide 16 text

Examples of Checks in Deepchecks Integrity Evaluation Methodology Distribution https://docs.deepchecks.com /

Slide 17

Slide 17 text

Check Each check enables inspecting a specific aspect of your data and models, such as data drift, duplicate values, etc. Each check can have 2 types of results: - A visual result meant for display (e.g. a figure or a table) - A return value that can be used for validating the expected check results

Slide 18

Slide 18 text

Condition A condition is a function that can be added to a Check, which returns a pass ✓, fail ✖ or warning ! result, intended for validating the Check’s return value.

Slide 19

Slide 19 text

Suite A suite is an ordered collection of checks that can have conditions added to them. The Suite enables displaying a concluding report for all of the Checks that ran.

Slide 20

Slide 20 text

Use-case

Slide 21

Slide 21 text

How to install Deepchecks Deep Checks requires Python 3 and can be installed using pip or conda, depending on the package manager you’re working with for most of your packages. - Using Pip - PIp Install Deepchecks - Using Conda - conda install -c conda-forge deepchecks - Using Google Colab or Kaggle Kernel - !pip install deepchecks --user

Slide 22

Slide 22 text

All you need to run your first Deepchecks suite

Slide 23

Slide 23 text

In order to run your first Deepchecks Suite all you need to have is the data and model that you wish to validate. More specifically, you need: ● Your train and test data (in Pandas DataFrames or Numpy Arrays) ● (optional) A supported model (including XGBoost, scikit-learn models, and many more). Required for running checks that need the model’s predictions for running.

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

Load Data, Split Train-Val, and Train a Simple Model

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Define a Dataset Object

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Run a DeepChecks suite

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Run a check

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

Conclusion

Slide 36

Slide 36 text

Deepchecks is fantastic and will only serve to make machine learning model valuation an easier experience. Running all of those tests by hand would take hundreds of lines of code.

Slide 37

Slide 37 text

Questions?

Slide 38

Slide 38 text

Thank you! Gift Ojeabulu CBB Analytics @GiftOjeabulu_