Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning as a Service

Machine Learning as a Service

Often, the most convenient way to deploy a machine model is an API. It allows accessing it from various programming environments and also decouples the development and deployment of the models from its use.

However, building an good API is hard. It involves many nitty-gritties and many of them need to repeated every time an API is built. It requires understanding of some web framework, worrying about data validation, authentication and deploying etc. Also, it is very important to have a client library so that the API can be easily accessed. If you ever plan to use it from Javascript directly, then you need to worry about cross-origin-resource-sharing etc.

In this talk demonstrates how deploying machine learning models an APIs can be made fun by using right programming abstractions.

Presented at:

- PyCon UK 2018 - Sep 16, 2018
- PyCon DE 2018 - Oct 25, 2018

7e03bd91a64c190bd4e54bd87d1ab0b2?s=128

Anand Chitipothu

September 16, 2018
Tweet

Transcript

  1. Machine Learning as a Service How to deploy ML Models

    as APIs without going nuts Anand Chitipothu @anandology rorodata 1
  2. Hello! Anand Chitipothu @anandology Co-founder of rorodata Platform-as-a-Service designed for

    data scientists 2
  3. Overview — Overview of Machine Learning — Why Machine Learning

    as a Service — Traditional Approach — Importance of Right Abstractions — Our Approach — Future work — Summary 3
  4. Machine Learning 4

  5. Machine Learning as a Service 5

  6. Model -> API 6

  7. ML as a Service: The Traditional Approach 7

  8. Origami Flapping Bird 8

  9. Flapping Bird - Crease Patterns 9

  10. Model -> API 10

  11. Model -> API 11

  12. Model -> Function -> API 12

  13. Function -> API 13

  14. Firefly! Open-source tool built to turn Python functions into API.

    https://github.com/rorodata/firefly pip install firefly-python 14
  15. Step 1: Write your function # sq.py def square(n): """compute

    square of a number. """ return n*n You just need to make sure the arguments and return value are JSON-friendly. 15
  16. Step 2: Run it as an API $ firefly sq.square

    http://127.0.0.1:8000/ ... 16
  17. Step 3: Use it >>> import firefly >>> api =

    firefly.Client("http://127.0.0.1:8000/") >>> api.square(n=4) 16 Firefly comes with in-built client for accessing the API. 17
  18. Behind the Scenes It is a RESTful API! $ curl

    -d '{"n": 4}' http://127.0.0.1:8000/square 16 18
  19. What about ML models? Write your predict function and run

    it as API. # image_recognition.py import joblib model = joblib.load("model.pkl") def predict(image): ... 19
  20. api = firefly.Client("http://127.0.0.1:8080") response = api.predict(image={"url": "http://example.com/foo.jpg"}) 20

  21. Deployment? How to deploy the API on a server? 21

  22. Deployment Platform (1/2) Write a config file in the project

    specifying what services to run. project: image-recognition runtime: python3-keras services: - name: api function: image_recognition.predict 22
  23. Deployment Platform (2/2) $ roro deploy ... Started 1 service.

    api: https://image-recognition--api.rorocloud.io/ ... 23
  24. Behind the Scenes — It builts the docker images —

    Starts the specified services — Provides URL endpoints with HTTPS 24
  25. Custom Dependencies? Write a requirements.txt file! The platform borrows the

    repo2docker interface from the Binder project. 25
  26. Building an App 26

  27. Boom! 27

  28. Cross-Origin-Resource-Sharing Just add an additional field in your service description.

    services: - name: api function: image_recognition.predict cors_allow_origins: * 28
  29. Instance Size & Scale Specify instance size and scale to

    deploy. services: - name: api function: image_recognition.predict cors_allow_origins: * size: S2 scale: 4 29
  30. Open-Source Version Built a light version of the platform to

    deploy to your own server. http://github.com/rorodata/rorolite Same API! 30
  31. Firefly - Upcoming Features — Extensible API — Integration with

    Python 3 type hints — Automatic API documentation — Support for gRPC 31
  32. Firefly - New API (WIP) app = Firefly() @app.function def

    detect_outliers(df: pd.DataFrame) -> pd.DataFrame: ... @app.function def detect_faces(image: Image) -> List[Image]: ... if __name__ == "__main__": app.run() 32
  33. Summary — Deployment Machine Learning Models need not be very

    complex! — The key is to focus on reducing the complexity of the interface. — Our work is open-source, please build on top of it! 33
  34. Thank you Slides - http://bit.ly/ml-service Anand Chitipothu @anandology rorodata 34