Software engineering practices applied to ML

Software engineering practices applied to ML

PyData London Meetup - April 2018 (https://www.meetup.com/PyData-London-Meetup/events/248973985/)

Stefano and Pavlos are engineers at HomeAway, and they'll talk about the benefits of applying good engineering practices when building ML-powered systems. The agenda will include:
- Feature engineering: does it have the same behaviour in training and prediction phases?
- CI/CD: are you checking for performance regressions in an automated fashion?
- Monitoring of your ML pipeline: are you sure your ML model works in production?

9a8cf536a1568332247e4ba0074558ab?s=128

Stefano Bonetti

April 03, 2018
Tweet

Transcript

  1. None
  2. None
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. CI/CD DESIGN AND DEVELOPMENT MONITORING

  13. CI/CD DESIGN AND DEVELOPMENT MONITORING

  14. Country LTV

  15. None
  16. id country listingtype US apartment UK id country listingtype USA

    flat United Kingdom
  17. Unit Tests Feature Engineering Service

  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. CI/CD DESIGN AND DEVELOPMENT MONITORING

  25. CI/CD DESIGN AND DEVELOPMENT MONITORING

  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. def test_ml_models(hold_out_data_set): # For Baseline Model baseline_mae, baseline_rmse = _calculate_error_for_baseline_model(

    hold_out_data_set ) # For New Model new_model_mae, new_model_rmse = _calculate_error_for_new_model( hold_out_data_set ) assert new_model_mae < baseline_mae assert new_model_rmse < baseline_rmse
  36. None
  37. None
  38. CI/CD DESIGN AND DEVELOPMENT MONITORING

  39. CI/CD DESIGN AND DEVELOPMENT MONITORING

  40. None
  41. None
  42. None
  43. None
  44. None
  45. None
  46. None
  47. None
  48. None
  49. 12 13

  50. None
  51. CI/CD DESIGN AND DEVELOPMENT MONITORING

  52. CI/CD DESIGN AND DEVELOPMENT MONITORING

  53. CI/CD DESIGN AND DEVELOPMENT MONITORING

  54. CI/CD DESIGN AND DEVELOPMENT MONITORING

  55. None
  56. None
  57. None