Software engineering practices applied to ML

PyData London Meetup - April 2018 (https://www.meetup.com/PyData-London-Meetup/events/248973985/)

Stefano and Pavlos are engineers at HomeAway, and they'll talk about the benefits of applying good engineering practices when building ML-powered systems. The agenda will include:
- Feature engineering: does it have the same behaviour in training and prediction phases?
- CI/CD: are you checking for performance regressions in an automated fashion?
- Monitoring of your ML pipeline: are you sure your ML model works in production?

Transcript

12. CI/CD DESIGN AND DEVELOPMENT MONITORING
13. CI/CD DESIGN AND DEVELOPMENT MONITORING
14. Country LTV
16. id country listingtype US apartment UK id country listingtype USA flat United Kingdom
17. Unit Tests Feature Engineering Service
24. CI/CD DESIGN AND DEVELOPMENT MONITORING
25. CI/CD DESIGN AND DEVELOPMENT MONITORING
35. def test_ml_models(hold_out_data_set): # For Baseline Model baseline_mae, baseline_rmse = _calculate_error_for_baseline_model( hold_out_data_set ) # For New Model new_model_mae, new_model_rmse = _calculate_error_for_new_model( hold_out_data_set ) assert new_model_mae < baseline_mae assert new_model_rmse < baseline_rmse
38. CI/CD DESIGN AND DEVELOPMENT MONITORING
39. CI/CD DESIGN AND DEVELOPMENT MONITORING
49. 12 13
51. CI/CD DESIGN AND DEVELOPMENT MONITORING
52. CI/CD DESIGN AND DEVELOPMENT MONITORING
53. CI/CD DESIGN AND DEVELOPMENT MONITORING
54. CI/CD DESIGN AND DEVELOPMENT MONITORING