Amazon’s system TAUGHT ITSELF that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. “
Amazon’s system TAUGHT ITSELF that male candidates were preferable. It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. They did not specify the names of the schools. “ LEARNED FROM HUMANS
IT IS HUMANS WHO COLLECT/LABEL DATA DEFINE METRICS WRITE ALGORITHMS - WHAT IS THE IMPACT OF DIFFERENT ERROR TYPES ON DIFFERENT GROUPS? - WHAT DO YOU OPTIMIZE FOR?
Practitioners consistently: - overestimate their model’s accuracy. - propagate feedback loops. - fail to notice data leaks. “ ” “Why Should I Trust You?” Explaining the Predictions of Any Classifier https:/ /arxiv.org/pdf/1602.04938.pdf
3- Weight points according to their proximity to the selected point. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
5- Explain the black-box model prediction using the local model. LIME (Tabular Data) Based on an example in “Interpretable Machine Learning” Book by Christoph Molnar
set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species,p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') TRAIN
set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) EXPLAIN
set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) ## explain new observations in test data explanation <- explain(test_data[, 1], explainer, n_labels = 1, n_features = 4) EXPLAIN
set.seed(5658) ## load libraries library(caret) library(lime) ## partition the data intrain <- createDataPartition(y = iris$Species, p = 0.8, list = F) ## create train and test data train_data <- iris[intrain, ] test_data <- iris[-intrain, ] ## train Random Forest model on train_data model <- train(x = train_data[, 1:4], y = train_data[, 5], method = 'rf') ## create an explainer object using train_data explainer <- lime(train_data, model) ## explain new observations in test data explanation <- explain(test_data[, 1:4], explainer, n_labels = 1, n_features = 4) https:/ /github.com/OmaymaS/satRday2019_talk_scripts/blob/master/R/lime_tabular.R
Pros LIME Cons - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. - The definition of proximity is not totally resolved in tabular data. - Instability of explanations.
Pros - Provides human-friendly explanations. - Gives a fidelity measure. - Can use other features than the original model. Cons - Instability of explanations. LIME - The definition of proximity is not totally resolved in tabular data.
SHAPLEY VALUES The contribution of temp value (4.416) to the difference between the actual prediction and the mean prediction is the estimated Shapley value (~ -1000).
Pros - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. SHAPLEY VALUES
Pros Cons - Solid theory - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
Pros Cons - Solid theory. - The difference between the prediction and the average prediction is fairly distributed among the feature values of the instance. - Computationally expensive. - Can be misinterpreted. SHAPLEY VALUES - Uses all features (not ideal for explanations that contain few features).
omaymas
line_developers
oracle4engineer
yusuke0519
mii3king
non97
netmarkjp
benevolent0505
yu4u
whywaita
tacck
okuzawats
colly
samanthasiow
bermonpainter
holman
morganepeng
eileencodes
bcantrill
bkeepers
shlominoach
reverentgeek
sachag