Applying deployment oriented mindset for building Machine Learning models at Data Science fwdays '19

Slide 1

Slide 1 text

APPLYING DEPLOYMENT ORIENTED MINDSET FOR BUILDING MACHINE LEARNING MODELS Marianna Diachuk WOMEN WHO CODE KYIV

Slide 2

Slide 2 text

AGENDA ➔ Why am I telling you this? ➔ What I mean by deployment? ➔ Deployment problems ➔ Model development process ➔ What deployment oriented mindset gives you 2

Slide 3

Slide 3 text

WHY AM TELLING YOU THIS? - I developed and deployed to production 7 scoring and antifraud ensemble models - I leaded a small but proud team of 2 data scientists and 1 data engineer - I’m a mother of 3 dragons ducks 3

Slide 4

Slide 4 text

WHY AM TELLING YOU THIS? WHAT WHY 4

Slide 5

Slide 5 text

WHAT I MEAN BY DEPLOYMENT? 5

Slide 6

Slide 6 text

I HAD MY SHARE OF FAILS TOO… 6

Slide 7

Slide 7 text

DEPLOYMENT PROBLEMS (SOME OF THEM) ➢ Model response inconsistency 7 research environment development environment

Slide 8

Slide 8 text

DEPLOYMENT PROBLEMS (SOME OF THEM) ➢ Model response inconsistency 8 research environment development environment dataset

Slide 9

Slide 9 text

DEPLOYMENT PROBLEMS (SOME OF THEM) ➢ Features inconsistency 9 research environment development environment dataset

Slide 10

Slide 10 text

DEPLOYMENT PROBLEMS (SOME OF THEM) ➢ Model response inconsistency ➢ Impossibility to implement features calculations ➢ Features inconsistency ➢ Model is not scalable and so on and so on… 10

Slide 11

Slide 11 text

MODEL DEVELOPMENT PROCESS LOOKS LIKE... 11

Slide 12

Slide 12 text

OR LIKE THIS... Research process Agile process 12

Slide 13

Slide 13 text

MODEL DEVELOPMENT PROCESS Remember about deployment. 13

Slide 14

Slide 14 text

How fast should our model respond? Are there any lim itations for deploym ent? Can we fetch the data from DB quickly? BUSINESS UNDERSTANDING STAGE Can developers help us with deploym ent? Should we worry about features calculation time? 14

Slide 15

Slide 15 text

your model the system model object features calculation fetching raw data BUSINESS UNDERSTANDING STAGE 15

Slide 16

Slide 16 text

BUSINESS UNDERSTANDING STAGE Pay attention to: ! Model response time ! Feature calculation time ! Database response ! Human resources availability 16

Slide 17

Slide 17 text

DATA UNDERSTANDING STAGE You can: ! Limit data sources 17

Slide 18

Slide 18 text

DATA UNDERSTANDING STAGE 1 request per 1 min. 1 application per <1 min. 18

Slide 19

Slide 19 text

DATA UNDERSTANDING STAGE You can: ! Limit data sources ! Work closely with colleagues 19

Slide 20

Slide 20 text

Ouch… It was one of the top features :( We removed one field from the application form. WORK CLOSELY WITH YOUR COLLEAGUES. 20

Slide 21

Slide 21 text

MODELLING You can: ! Refactor your code 21

Slide 22

Slide 22 text

REFACTOR YOUR CODE 1. Write your feature calculation 4. Improve your code efficiency 2. Test the feature 3. Improve your code readability 22

Slide 23

Slide 23 text

REFACTOR YOUR CODE 23

Slide 24

Slide 24 text

MODELLING You can: ! Refactor your code ! Test model response 24

Slide 25

Slide 25 text

TEST MODEL RESPONSE TIME ExecuteTime from jupyter_contrib_nbextensions time module %%time 25

Slide 26

Slide 26 text

Keep track of versions. Always! but not like this JUST TO KEEP IN MIND… 26

Slide 27

Slide 27 text

KEEP TRACK OF VERSIONS. 27

Slide 28

Slide 28 text

Document everything! So your future self and your teammates would be happy. JUST TO KEEP IN MIND… 28

Slide 29

Slide 29 text

DOCUMENT EVERYTHING ➔ keep clean Jupyter notebooks ➔ keep all changes in code 29

Slide 30

Slide 30 text

WHAT DEPLOYMENT ORIENTED MINDSET GIVES YOU. - Easier to estimate tasks - Less risks in the late phases -> less postponed deadlines - Better to understand responsibilities distribution 30