Automating Fraud Detection - Continuous Model Deployment

Slide 1

Slide 1 text

Stephen Whitworth | 08.02.18 ravelin.com Continuous model deployment

Slide 2

Slide 2 text

ravelin.com ravelin.com Credit card fraud detection platform for merchants

Slide 3

Slide 3 text

ravelin.com ravelin.com Score customers in real time for likelihood of fraud

Slide 4

Slide 4 text

ravelin.com ravelin.com Machine learning sits at the core of our detection strategy

Slide 5

Slide 5 text

ravelin.com ravelin.com Normal ML deployment cycle: release few times a quarter

Slide 6

Slide 6 text

ravelin.com ravelin.com Ravelin deployment cycle: deploy new models many times a week

Slide 7

Slide 7 text

ravelin.com ravelin.com Frequency reduces difficulty: if something is hard, do it more often. (Martin Fowler)

Slide 8

Slide 8 text

Training infrastructure ● Python / Go hybrid pipeline ● Packaged/distributed through Docker ● On demand compute on big machines ● One line to build a new model, run experiments

Slide 9

Slide 9 text

Pipeline output ● New model, trained from scratch ● All output archived to Google Cloud Storage ● Performance metrics posted to internal registry ● Model deployed to asynchronous live cluster ● HTML report of performance for team

Slide 10

Slide 10 text

Summary report

Slide 11

Slide 11 text

Comparing two models

Slide 12

Slide 12 text

● Summarisation over raw details ● Minimise manual toil at all costs ● Automation reigns king ● Unit test output of models ● Make model deployment ‘boring’ Principles for high-performing ML teams

Slide 13

Slide 13 text

● Data Scientists - join my team! ● Head of Product ● Product Managers ● Javascript Engineer ● Investigations Analyst ● Full Stack Engineers ● Backend Engineers ● Devops Engineer We’re hiring - www.angel.co/ravelin