Slide 1

Slide 1 text

Micro Service Architecture with Machine Learning Application !1 Plone Conference Tokyo, Nov, 7, 2018 Masato Fujitake(@Swall0wTech)

Slide 2

Slide 2 text

Who am I !2 • Research • Master student in SIT • Object Detection for Robotics | Computer Vision • Startup • Co-Founder of FAST ACCOUNTING CO.,LTD • Research Development for OCR, Architecture • Social Media • Twitter https://twitter.com/Swall0wTech • Github https://github.com/Swall0w https://github.com/Swall0w/microservice-ml-demo

Slide 3

Slide 3 text

Overview !3 Introduction to Machine Learning (ML) How to deploy ML application on Flask Benefits of Micro Service Architecture (Micro services) How to realize it Demo

Slide 4

Slide 4 text

Introduction to Machine Learning !4 “Machine Learning is a field of artificial intelligence that uses statistical techniques to give computer systems the ability to learn from data, without being explicitly programmed.” From Wikipedia Definition →Algorithm which extract meaningful patterns from data. Applications • Email Spam and Malware Filtering • Machine Translation • Forecasting such as sales • Product Recommendations →ML applications have become one of important part of Web.

Slide 5

Slide 5 text

Let’s deploy ML applications !5 Details • Web Framework: Flask • ML Framework: Chainer, ChainerCV • ML Application: Object Detection Internet Web application ML Model REST API Simple Architecture

Slide 6

Slide 6 text

Object Detection !6 YOLOv3:https://bit.ly/2fXFbXS

Slide 7

Slide 7 text

How to use ML model !7 1 import chainer 2 from chainercv.datasets import voc_bbox_label_names 3 from chainercv.links import YOLOv3 4 from chainercv import utils 5 6 7 def main(image): 8 model = YOLOv3( 9 n_fg_class=len(voc_bbox_label_names), 10 pretrained_model='voc0712') 11 12 img = utils.read_image(image, color=True) 13 bboxes, labels, scores = model.predict([img]) 14 bbox, label, score = bboxes[0], labels[0], scores[0] Minimal code Load model predict method

Slide 8

Slide 8 text

How to use ML model !8 Example Input image Output image Results of Variable bbox: [[101.38964, 197.98589, 362.8902, 403.87808]] label: [11] score: [0.9999248]

Slide 9

Slide 9 text

How to deploy ML model !9 1 app = flask.Flask(__name__) 2 model = YOLOv3( 3 n_fg_class=len(voc_bbox_label_names), 4 pretrained_model='voc0712') 5 6 @app.route("/predict", methods=["POST"]) 7 def predict(): 8 data = {"success": False} 9 10 if flask.request.method == "POST": 11 if flask.request.files.get("image"): 12 image = flask.request.files["image"].read() 13 image = prepare_image(image) 14 bboxes, labels, scores = model.predict([image]) 15 16 data["predictions"] = pack_result_each_item( 17 bboxes, labels, scores) 18 data["success"] = True 19 20 return flask.jsonify(data) 21 22 if __name__ == "__main__": 23 app.run() predict method Load model

Slide 10

Slide 10 text

Problems of this code !10 difficult to update model This simple ML application works well but… • Parameters • Algorithm Separate ML application from Flask

Slide 11

Slide 11 text

Benefits of Micro services Separate ML application from Flask Internet Web application ML Model REST API The micro services allows a system to be divided into a smaller, individual and independent services. !11 • Simpler to deploy • Simpler to understand • Flexible in using technologies The micro services makes a system

Slide 12

Slide 12 text

How do we connect services Web application ML Model !12 Transport Protocols -> synchronous communications Serialization format -> Protocol Buffers (gRPC) gRPC Benefits • Multi languages(Python, C++…) • Quick to use because of its Interface Definition Language • Faster than HTTP + JSON

Slide 13

Slide 13 text

How to use gRPC !13 syntax = “proto3"; service MLServer { rpc predict(stream Chunk) returns (Reply) {} } message Chunk { bytes buffer = 1; } message Boundingbox { repeated float box = 1; } message Reply { repeated Boundingbox boxes = 1; repeated string classes = 2; repeated float confs = 3; } Write Interface Definition Language ɾmessage: key-value pairs ɾservice: data access classes Generate modules with command ɾdetection_pb2.py ɾdetection_pb2_grpc.py detection.proto For image For result

Slide 14

Slide 14 text

Implement server and client !14 1 class Servicer(detection_pb2_grpc.MLServerServicer): 2 def __init__(self, predictor): 3 self._predictor = predictor 4 5 def predict(self, request_iterator, context): 6 image = save_chunks_to_file_object(request_iterator) 7 # predict 8 boxes, classes, confs = self._predictor.predict(image) 9 bboxes = [detection_pb2.Boundingbox(box=box) for box in boxes] 10 return detection_pb2.Reply(boxes=bboxes, classes=classes, confs=confs) 1 class MLClient(object): 2 def __init__(self, address, parse=None): 3 channel = grpc.insecure_channel(address) 4 self.stub = detection_pb2_grpc.MLServerStub(channel) 5 self._parse = parse 6 7 def predict(self, in_file): 8 chunks_generator = get_virtual_file_chunks(in_file) 9 response = self.stub.predict(chunks_generator) 10 if self._parse != None: 11 response = self._parse(response) 12 return response Client Code Service Code

Slide 15

Slide 15 text

How does web application change !15 1 app = flask.Flask(__name__) 2 object_detection = detection_grpc.MLClient('localhost:8888', 3 parse=chainercv_parse) 4 5 @app.route("/predict", methods=["POST"]) 6 def predict(): 7 data = {"success": False} 8 9 if flask.request.method == "POST": 10 if flask.request.files.get("image"): 11 image = flask.request.files["image"].read() 12 13 predictions = object_detection.predict(image) 14 15 data["predictions"] = predictions 16 data["success"] = True 17 18 return flask.jsonify(data) The web apps doesn't care ML algorithm.

Slide 16

Slide 16 text

Demo !16

Slide 17

Slide 17 text

Conclusion !17 Internet Web application ML Model REST API gRPC Introduction to Machine Learning (ML) How to deploy ML application on Flask Deployment as Micro Services • Simpler to deploy • Simpler to understand • Flexible in using technologies

Slide 18

Slide 18 text

Thank you for listening Source code and sample are available at Github https://github.com/Swall0w/microservice-ml-demo