Workflow in a team of data scientists (tech talk for colleagues)

I was speaking of CRISP-DM way of organizing the process of ML models development and the modifications we implemented. I also described all the stages of model development.

Transcript

1. Workflow in a team of data scientists
2. Understanding of data science process data models
3. Understanding of data science process data models
4. Types of ML
5. Supervised Machine Learning
6. Data Science workflow by CRISP-DM
7. Business understanding • Define problem in terms of business - define business question to the future model Example: detect and prevent frauds intrusion • Define data science problem Example: who are considered to be frauds, how to detect frauds • Define what we need to solve the problem - what data to gather and analyze
8. Data understanding • Gather dataset
9. Data Understanding • EDA (Exploratory Data Analysis) Objectives: 1. Discover patterns 2. Spot anomalies 3. Frame hypothesis 4. Check assumptions
10. Data Preparation (50-70% of project time) • Data Preprocessing (handle missing values, wrong data types, etc.) • Dataset Labeling one class problem
11. Data Preparation • Divide dataset into train/test (validate, folds) • Feature Engineering
12. Modeling • Creating baseline model • Choosing algorithms • Feature selection
13. Modeling • Tune model hyperparameters - to achieve higher accuracy - to improve model performance sklearn.ensemble.GradientBoostingClassifier(loss=’deviance’, learning_rate=0.1, n_estimators=100, subsample=1.0, criterion=’friedman_mse’, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_depth=3, min_impurity_decrease=0.0, min_impurity_split=None, init=None, random_state=None, max_features=None, verbose=0, max_leaf_nodes=None, warm_start=False, presort=’auto’)
14. Modeling • Tune model hyperparameters - to avoid overfitting
15. Evaluation - Model Performance Metrics (AUC, Gini, F1, Confusion matrix, etc.) - Business metrics (profits, approval rate, default rate, etc.) - Evaluate achievement of business Purposes Some models may not get to deployment stage after evaluation.
16. De data about an application OneKarma Scoring service API trained model { model score fetch data
17. trained model Response JSON Request JSON
18. Terms for each stage Business understanding - 1 week Data understanding - 3 weeks Data preparation - 5 weeks Modeling - 2 weeks Evaluation - 1 week Deployment - 1 week Full model development process - ~ 13 weeks
19. Data scientists Data engineers Data scientists Business side Data scientists Data engineers Data scientists Data scientists Data engineers Development team QA team Data scientists Business side
20. Thank you for your attention.