Automating Fraud Detection - Continuous Model Deployment

Stephen Whitworth, Co-Founder & Machine Learning Engineer, Ravelin

Transcript

1. Stephen Whitworth | 08.02.18 ravelin.com Continuous model deployment
2. ravelin.com ravelin.com Credit card fraud detection platform for merchants
3. ravelin.com ravelin.com Score customers in real time for likelihood of fraud
4. ravelin.com ravelin.com Machine learning sits at the core of our detection strategy
5. ravelin.com ravelin.com Normal ML deployment cycle: release few times a quarter
6. ravelin.com ravelin.com Ravelin deployment cycle: deploy new models many times a week
7. ravelin.com ravelin.com Frequency reduces difficulty: if something is hard, do it more often. (Martin Fowler)
8. Training infrastructure • Python / Go hybrid pipeline • Packaged/distributed through Docker • On demand compute on big machines • One line to build a new model, run experiments
9. Pipeline output • New model, trained from scratch • All output archived to Google Cloud Storage • Performance metrics posted to internal registry • Model deployed to asynchronous live cluster • HTML report of performance for team
10. Summary report
11. Comparing two models
12. • Summarisation over raw details • Minimise manual toil at all costs • Automation reigns king • Unit test output of models • Make model deployment ‘boring’ Principles for high-performing ML teams
13. • Data Scientists - join my team! • Head of Product • Product Managers • Javascript Engineer • Investigations Analyst • Full Stack Engineers • Backend Engineers • Devops Engineer We’re hiring - www.angel.co/ravelin