Managing Machine Learning Models in Production - Strata Singapore 2017

Slides of talk "Managing Machine Learning Models in Production" presented at Strata Data Conference in Singapore.

Transcript

1. Managing Machine Learning Models in Production Anand Chitipothu rorodata 1
2. Who is Speaking? Anand Chitipothu @anandology Co-founder and platform architect of @rorodata Worked at Internet Archive & Strand Life Sciences Advanced programming courses at @pipalacademy 2
3. Machine Learning [Machine learning is the] field of study that gives computers the ability to learn without being explicitly programmed. — Arthur Samuel 3
4. Machine Learning 4
5. Machine Learning in Production When you start ... 5
6. Machine Learning in Production As you grow ... 6
7. Machine Learning in Production Over time ... 7
8. The Key Elements 8
9. Challenges: Build — Compute: switch between different compute needs seamlessly — Setup: installing require software dependencies — Automation: re-train models periodically 9
10. Challenges: Deploy — Designing: Designing and documenting APIs — Wiring: Running the service and configuring URL endpoints — Scaling: Scale to meet the usage 10
11. Challenges: Manage — Model versions: Keep track of multiple versions of models — Model metadata: What went into building each model? — Collaboration: Sharing models in and across the teams 11
12. Challenges: Monitor — Performance: Request rate & Latency — Logging: Record requested data and predicted values — Effectiveness: Model accuracy in production 12
13. The pace of innovation of a data-driven business is limited by the bottlenecks in their data science workflows 13
14. How to Solve it? — Duct tape approach — Build a data platform — Using a managed platform-as-a-service 14
15. The rorodata Platform Cloud platform to address the challenges in managing machine learning in production. 15
16. rorodata: Build — Elastic Compute $ roro run -size M2 python train.py Created new job b42c12a0 $ roro run:notebook --gpu Jupyter notebook is available at: https://a098cd79.rorocloud.io/ 16
17. rorodata: Build — Elastic Compute — Seamless Setup # roro.yml runtime: python3-keras # requirements.txt hyperopt>=0.1 17
18. rorodata: Build — Elastic Compute — Seamless Setup — Flexible Scheduling # roro.yml runtime: python3-keras tasks: - name: retrain command: python train.py when: "every day at 10:00 AM" 18
19. rorodata: Deploy — Python functions as API # roro.yml services: - name: api function: predict.predict - name: rating function: credit_grade.get_credit_grade 19
20. rorodata: Deploy — Python functions as API — Automatic endpoint management $ roro deploy Deploying credit-risk... Building docker image... done. Updating scheduled jobs... done. Restarting services... api: https://credit-risk--api.rorocloud.io/ rating: https://credit-risk--rating.rorocloud.io/ Deployed v4 of credit-risk project. 20
21. rorodata: Deploy — Python functions as API — Automatic endpoint management — Simple Scaling # roro.yml services: - name: api function: predict.predict size: M2 21
22. rorodata: Manage — Model Metadata Model-ID: c54e00eb Model-Name: iris Model-Version: 3 Date: 2017-08-02T10:20:30Z ... Input-Source: s3://iris-sample-data Dataset-Features: Sepal-Length,Sepal-Width,Petal-... Training-Algorithm: SVM Training-Parameters: C=10; alpha=0.4; kernel=rbf Training-Accuracy: 0.85 22
23. rorodata: Manage — Model Metadata — Model Versioning +---------------------------------------+ | Model Repository A | | | | ModelImage - v1 ModelImage - v2 | | +---------------+ +---------------+ | | | Model v1 | | Model v2 | | | +---------------+ +---------------+ | | | Metadata v1 | | Metadata v2 | | | +---------------+ +---------------+ | +---------------------------------------+ 23
24. rorodata: Manage — Model Metadata — Model Versioning - Python API project = roro.get_current_project() repo = project.get_model_repository("iris") model_image = repo.new_image(model) model_image["Dataset-Features"] = "Sepal-Length,Sepal-Width,Petal-..." model_image["Training-Accuracy"] = 0.85 model_image.save(comment="Built a new model using SVM") 24
25. rorodata: Manage — Model Metadata — Model Versioning — Collaboration Add other people as collaborators to your project. 25
26. rorodata: Monitor — Performance Metrics Dashboard to inspect the request rate and latency of every API. 26
27. rorodata: Monitor — Performance Metrics — Data Logging # Data Logger to record key-value pairs. roro.record("credit-risk-api", model_name="credit-risk", request_id=request_id, user_id=user_id, input=input_data, predicted=predicted_value) 27
28. rorodata: Monitor — Performance Metrics — Data Logging — Feedback Loop API endpoint to record the user action to close the feedback loop. This allows computing the effectiveness of the live model. 28
29. The rorodata Platform 29
30. Solving Machine Learning in Production — Start early — Take a platform approach — Learn from others 30
31. Our Open Source Components Firefly Function as service. https://github.com/rorodata/firefly rorolite Lite version of the platform to deploy ML apps on your server. https://github.com/rorodata/rorolite 31
32. Thank You! Anand Chitipothu @anandology http://rorodata.com/ 32