Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Managing Machine Learning Models in Production - Strata Singapore 2017

Managing Machine Learning Models in Production - Strata Singapore 2017

Slides of talk "Managing Machine Learning Models in Production" presented at Strata Data Conference in Singapore.

7e03bd91a64c190bd4e54bd87d1ab0b2?s=128

Anand Chitipothu

December 06, 2017
Tweet

Transcript

  1. Managing Machine Learning Models in Production Anand Chitipothu rorodata 1

  2. Who is Speaking? Anand Chitipothu @anandology Co-founder and platform architect

    of @rorodata Worked at Internet Archive & Strand Life Sciences Advanced programming courses at @pipalacademy 2
  3. Machine Learning [Machine learning is the] field of study that

    gives computers the ability to learn without being explicitly programmed. — Arthur Samuel 3
  4. Machine Learning 4

  5. Machine Learning in Production When you start ... 5

  6. Machine Learning in Production As you grow ... 6

  7. Machine Learning in Production Over time ... 7

  8. The Key Elements 8

  9. Challenges: Build — Compute: switch between different compute needs seamlessly

    — Setup: installing require software dependencies — Automation: re-train models periodically 9
  10. Challenges: Deploy — Designing: Designing and documenting APIs — Wiring:

    Running the service and configuring URL endpoints — Scaling: Scale to meet the usage 10
  11. Challenges: Manage — Model versions: Keep track of multiple versions

    of models — Model metadata: What went into building each model? — Collaboration: Sharing models in and across the teams 11
  12. Challenges: Monitor — Performance: Request rate & Latency — Logging:

    Record requested data and predicted values — Effectiveness: Model accuracy in production 12
  13. The pace of innovation of a data-driven business is limited

    by the bottlenecks in their data science workflows 13
  14. How to Solve it? — Duct tape approach — Build

    a data platform — Using a managed platform-as-a-service 14
  15. The rorodata Platform Cloud platform to address the challenges in

    managing machine learning in production. 15
  16. rorodata: Build — Elastic Compute $ roro run -size M2

    python train.py Created new job b42c12a0 $ roro run:notebook --gpu Jupyter notebook is available at: https://a098cd79.rorocloud.io/ 16
  17. rorodata: Build — Elastic Compute — Seamless Setup # roro.yml

    runtime: python3-keras # requirements.txt hyperopt>=0.1 17
  18. rorodata: Build — Elastic Compute — Seamless Setup — Flexible

    Scheduling # roro.yml runtime: python3-keras tasks: - name: retrain command: python train.py when: "every day at 10:00 AM" 18
  19. rorodata: Deploy — Python functions as API # roro.yml services:

    - name: api function: predict.predict - name: rating function: credit_grade.get_credit_grade 19
  20. rorodata: Deploy — Python functions as API — Automatic endpoint

    management $ roro deploy Deploying credit-risk... Building docker image... done. Updating scheduled jobs... done. Restarting services... api: https://credit-risk--api.rorocloud.io/ rating: https://credit-risk--rating.rorocloud.io/ Deployed v4 of credit-risk project. 20
  21. rorodata: Deploy — Python functions as API — Automatic endpoint

    management — Simple Scaling # roro.yml services: - name: api function: predict.predict size: M2 21
  22. rorodata: Manage — Model Metadata Model-ID: c54e00eb Model-Name: iris Model-Version:

    3 Date: 2017-08-02T10:20:30Z ... Input-Source: s3://iris-sample-data Dataset-Features: Sepal-Length,Sepal-Width,Petal-... Training-Algorithm: SVM Training-Parameters: C=10; alpha=0.4; kernel=rbf Training-Accuracy: 0.85 22
  23. rorodata: Manage — Model Metadata — Model Versioning +---------------------------------------+ |

    Model Repository A | | | | ModelImage - v1 ModelImage - v2 | | +---------------+ +---------------+ | | | Model v1 | | Model v2 | | | +---------------+ +---------------+ | | | Metadata v1 | | Metadata v2 | | | +---------------+ +---------------+ | +---------------------------------------+ 23
  24. rorodata: Manage — Model Metadata — Model Versioning - Python

    API project = roro.get_current_project() repo = project.get_model_repository("iris") model_image = repo.new_image(model) model_image["Dataset-Features"] = "Sepal-Length,Sepal-Width,Petal-..." model_image["Training-Accuracy"] = 0.85 model_image.save(comment="Built a new model using SVM") 24
  25. rorodata: Manage — Model Metadata — Model Versioning — Collaboration

    Add other people as collaborators to your project. 25
  26. rorodata: Monitor — Performance Metrics Dashboard to inspect the request

    rate and latency of every API. 26
  27. rorodata: Monitor — Performance Metrics — Data Logging # Data

    Logger to record key-value pairs. roro.record("credit-risk-api", model_name="credit-risk", request_id=request_id, user_id=user_id, input=input_data, predicted=predicted_value) 27
  28. rorodata: Monitor — Performance Metrics — Data Logging — Feedback

    Loop API endpoint to record the user action to close the feedback loop. This allows computing the effectiveness of the live model. 28
  29. The rorodata Platform 29

  30. Solving Machine Learning in Production — Start early — Take

    a platform approach — Learn from others 30
  31. Our Open Source Components Firefly Function as service. https://github.com/rorodata/firefly rorolite

    Lite version of the platform to deploy ML apps on your server. https://github.com/rorodata/rorolite 31
  32. Thank You! Anand Chitipothu @anandology http://rorodata.com/ 32