ࡾ୐༔հ / Pepabo R&D Institute, GMO Pepabo, Inc. 2017.01.28 GCPUG Fukuoka 5th ʙMachine Learning ࡇʙ Google Cloud MLΛ༻͍ͨ ػցֶशج൫ͷߏஙͱӡ༻

ϓϦϯγύϧΤϯδχΞ ࡾ୐༔հ!NPOPDISPNFHBOF ϖύϘݚڀॴݚڀһ IUUQCMPHNPOPDISPNFHBOFDPN

ࠓ೔͓࿩͢͠Δ͜ͱ •ϖύϘݚڀॴͱɺͳΊΒ͔ͳγεςϜ •ͳΊΒ͔ͳγεςϜΛ࣮ݱ͢Δػցֶशج൫ •Google Cloud ML ͱ StarChart Λ༻͍ͨػցֶशج ൫ͷӡ༻ https://icons8.com/

ϖύϘݚڀॴ

ϖύϘݚڀॴ ུশʮϖύݚʯ ͸ɺࣄۀΛࠩ ผԽͰ͖Δٕज़Λ࡞Γग़ͨ͢ΊʹʮͳΊΒ͔ ͳγεςϜʯͱ͍͏ίϯηϓτͷԼͰݚڀ։ ൃʹऔΓ૊Ή૊৫Ͱ͢ɻ ϖύϘݚڀॴʹ͍ͭͯ http://rand.pepabo.com/

ͳΊΒ͔ͳγεςϜ

γεςϜͷ֤ཁૉ͕໌ࣔతͳૢ࡞Λܦͣʹಛ ௃Λೝࣝ͠ɺͦͷಛ௃΍ؔ܎ੑʹج͖ͮɺͦ ͷ࣌ʑͷঢ়گʹԠͨ͡࠷దͳαʔϏεΛఏڙ ͢Δ ͳΊΒ͔ͳγεςϜ http://rand.pepabo.com/

ػցֶशͱɺͳΊΒ͔ͳγεςϜ

ྫ͑͹

WebαʔϏεͷΞΫηε਺Λ༧ଌ͢Δ •ैྔ՝ۚͷԾ૝Ϧιʔεӡ༻ʹ͓͍ͯ࠷దͳϦιʔεधཁͷ༧ ଌ͸ίετΧοτʹͭͳ͕Δ •WebαʔϏεͷϦιʔεधཁ͸ϦΫΤετॲཧ݅਺ɺͭ·ΓΞ Ϋηε਺ͱ૬͕ؔ͋Δ͸ͣ •Ϧιʔεͷ૿ݮʹ΋͋Δఔ౓ͷ͕͔͔࣌ؒΔͨΊɺϦΞϧλΠ ϜͰ͸ͳ͘ҰఆִؒͰͷΞΫηε਺༧ଌͰे෼ͱߟ͑Δ

WebαʔϏεͷΞΫηε਺Λ༧ଌ͢Δ ΞΫηε਺Λ༧ଌͰ͖ΔΑ͏ʹͳΕ ͹ɺϐʔΫλΠϜʹ͋Θͤͨ୆਺ݟੵ ΋Γ͔Β࣌ؒ͝ͱͷ࠷దʢͱࢥΘΕ Δʣ୆਺ݟੵ΋Γ͕ՄೳʹͳΔ

LSTM Long Short Term Memory

WebαʔϏεͷΞΫηε਺Λ༧ଌ͢Δ -45.Λ༻͍ͨΞΫηε਺༧ଌ w ೖྗ͸ظ ೔෼ ͷΞΫηε਺ͱΧϨϯμʔ৘ใ w ग़ྗ͸༧ଌͨ͠ظઌͷΞΫηε਺ w ظઌͷ༧ଌʹ͸લճͷ༧ଌΛೖྗʹؚΊͳ͕Β ظઌ·Ͱͷ༧ଌ w ࠨਤ͸िؒ෼Λ܁Γฦ͠༧ଌ IUUQSBOEQFQBCPDPNBSUJDMFNPOPDISPNFHBOF

ϖύݚ

ΞΧσϛοΫͳਫ४ʹ͓͚Δ৽نੑɾ༗ޮੑɾ ৴པੑΛ௥ٻ͢ΔݚڀΛߦ͏ͱͱ΋ʹɺݚڀ ։ൃٕͨ͠ज़Λ࣮ࡍͷγεςϜͱ࣮ͯ͠૷ɾ ఏڙ͢Δ͜ͱΛ௨ͯ͠ɺࣄۀͷ੒௕ʹߩݙ͠ ·͢ɻ ϖύϘݚڀॴʹ͍ͭͯ http://rand.pepabo.com/

Service meets ML. ࢖ͬͯφϯϘ

ػցֶशج൫ʹ ٻΊΒΕΔ΋ͷ

ྫ͑͹ɺΞΫηε਺༧ଌΛαʔϏεͰ࢖͏ Users Service log rack-bigfoot Bigfoot Activity ML Platform training dataset train input data prediction tune Access count prediction and schedule scaling

ྫ͑͹ɺΞΫηε਺༧ଌΛαʔϏεͰ࢖͏ Users Service log rack-bigfoot Bigfoot Activity ML Platform training dataset train input data prediction tune Access count prediction and schedule scaling ͍͔ͳΔಛ௃Λ͔࣋ͭΛ ਫ਼៛ʹೝࣝ͢Δ ਓؒʹ໌ࣔతͳૢ࡞Λ ՝͞ͳ͍ ͦͷ࣌ʑͷঢ়گʹԠͯ͡ ࠷దͳαʔϏεΛఏڙ͢Δ

OK, ͳΊΒ͔

ػցֶशج൫ͷཁ݅Λߟ͑Δ ϩά΍%#ͳͲͷαʔϏεࢿ࢈ͱ࿈ܞͰ͖Δ ൺֱత༰қʹϞσϧͷߏஙͱࢼߦ͕ߦ͑Δ ֶश݁ՌΛར༻͢ΔͨΊͷखஈͱͯ͠"1*Λఏڙ͢Δ ֶश݁ՌͷϩʔΧϧར༻͕Ͱ͖Δͱͳ͓Α͍ ্هͷ࢓૊Έ͕εέʔϥϒϧͰ͋Δ͜ͱ

Google Cloud ML Ͱߟ͑Δ ೖग़ྗ͕$MPVE4UPSBHFܦ༝ ܇࿅ϓϩάϥϜͱͯ͠5FOTPS'MPXΛ࠾༻ ΦϯϥΠϯ༧ଌαʔϏεʹΑΓϞσϧͷ"1*Խ ֶश݁Ռ͸$MPVE4UPSBHFʹอଘɺϩʔΧϧͰͷར༻΋ ෼ࢄܕͷτϨʔχϯάΠϯϑϥͱෛՙ෼ࢄαʔϏεͱͷ࿈ܞ ※ ݕ౼ʹؔ͢Δৄࡉ: http://rand.pepabo.com/article/2017/01/18/pepabo-ml-platform-and-workflow/

ػցֶशج൫Λ ӡ༻͢Δ

ֶश݁Ռ͸Ӭଓతͳ΋ͷͰ͸ͳ͍ expose Create 1. ϞσϧΛߏஙͯ͠ެ։͢Δ 2. αʔϏεࢿ࢈ͷมԽʹ߹ΘͤͯϞσϧΛվળ͢Δ tune

ͦͷϞσϧɺαʔϏεʹఏڙͯ͠େৎ෉ʁ expose create ֶशͨ͠಺༰͕֬ೝͳ͘ద༻͞Εͯ͠·͏͜ͱͰҙਤ͠ͳ͍݁ Ռ͕ར༻͞ΕΔ͜ͱΛ๷͍͗ͨ tune Ϟσϧֶश͠ͳ͓ͨ͠ͷͰɺ APIͷ݁Ռ͕มΘͬͯ·͢ ਫ਼౓Լ͕ͬͯΔؾ͕͢Δʁ ग़ྗͷ࣍ݩ਺มΘͬͯͳ͍ʁ

ͦͷϞσϧɺαʔϏεʹఏڙͯ͠େৎ෉ʁ expose create ֶशͨ͠಺༰͕֬ೝͳ͘ద༻͞Εͯ͠·͏͜ͱͰҙਤ͠ͳ͍݁ Ռ͕ར༻͞ΕΔ͜ͱΛ๷͍͗ͨ -> Ϟσϧͷόʔδϣϯ؅ཧ tune Ϟσϧֶशͯ͠৽όʔδϣϯ ͭͬͯ͘·͢ɻ੾ସ͍͍Ͱ͔͢ʁ ͍͖ͳΓมΘΒͳͯ͘ศར ͚ͩͲɺมߋ಺༰͕Θ͔ΒΜͳʁ default

ͦͷϞσϧɺαʔϏεʹఏڙͯ͠େৎ෉ʁ expose create ֶशͨ͠಺༰͕֬ೝͳ͘ద༻͞Εͯ͠·͏͜ͱͰҙਤ͠ͳ͍݁ Ռ͕ར༻͞ΕΔ͜ͱΛ๷͍͗ͨ -> Ϟσϧͷίʔυ؅ཧ tune Ϟσϧͷ৽όʔδϣϯɺมߋ಺༰ͷ ϨϏϡʔ͓ئ͍͠·͢ʂ LGTM!!! όʔδϣϯ੾Γସ͑·͢ʂʂ default management

StarChart https://github.com/monochromegane/starchart

StarChart is a tool to manage Google Cloud Machine Learning training programs and model versions. StarChart

StarChart Train job, model default version Train programs and model versions on GitHub train, expose, apply StarChart • όʔδϣϯ؅ཧͷ੾ସʹ͓͚Δ൑அج४ͱͳΔ܇࿅ϓϩάϥϜɺύϥϝλɺδϣϒ ৘ใ·ͰؚΊͯίʔυͰ؅ཧ • ֶश࣌ͷδϣϒID΍Cloud Storageͷύεɺόʔδϣϯʹඥͮ͘ύϥϝλ৘ใͷऔ ಘʹ·ͭΘΔCloud MLͷࡉ͔ͳ࢖͍উख΋վળ

Let’s try

DCGAN on Cloud ML using StarChart

DCGAN TensorFlowʹΑΔDCGANͰΞΠ υϧͷإը૾ੜ੒ http://memo.sugyan.com/entry/20160516/1463359395

܇࿅ϓϩάϥϜΛGit؅ཧ͢Δ . !"" dcgan #"" setup.py !"" trainer #"" __init__.py #"" dcgan.py !"" task.py Ϟσϧ໊ͷ഑Լʹύοέʔδߏ੒ʹͳ ΔΑ͏ʹ܇࿅ϓϩάϥϜΛ഑ஔ ґଘύοέʔδ͕͋Δ৔߹͸ɺ setup.pyΛ४උ ࠓճ͸face-generatorͷdcgan.pyͱ main.pyΛར༻ https://github.com/sugyan/face-generator

܇࿅ϓϩάϥϜΛδϣϒͱͯ͠ొ࿥ $ starchart train \ -m dcgan \ # MODEL_NAME -M trainer.task \ # MODULE_NAME -- \ --train_dir=TRAIN_PATH/model \ # YOUR_TRAIN_PARAMS --images_dir=TRAIN_PATH/images \ --data_dir=gs://$BUCKET_NAME/data/dcgan • ύοέʔδϯάɺCloud Storage΁ͷΞοϓϩʔυɺδϣϒొ࿥Λ࣮ߦ • `TRAIN_PATH`͸Cloud Storage্ʹδϣϒ͝ͱʹ࡞੒͞ΕΔσΟϨΫτϦ໊ʹղऍ • ϓϩδΣΫτIDɺϦʔδϣϯɺΫϨσϯγϟϧ͸direnvܦ༝ͷ؀ڥม਺ࢦఆ͕ศར

δϣϒͷ࣮ߦΛ଴ͭ $ starchart state -m dcgan jobId: dcgan_20170125191521 (FAILED) • δϣϒIDͱεςʔλεΛ֬ೝ • ϩάදࣔػೳ͸ະ࣮૷

FAILED??

܇࿅ϓϩάϥϜΛ$MPVE.-ʹରԠͤ͞Δ import os os.listdir() os.mkdir(path) os.path.exists() with open(filename, 'wb') as f: • δϣϒ࣮ߦ࣌ͷFileIO͸Cloud StorageΛର৅ͱ͢Δ • tensorflow.python.lib.io.file_ioύοέʔδΛ࢖͏͜ͱͰϩʔΧϧύεࢦఆɺCloud Storageࢦఆ(gs://)Λಁաతʹѻ͑Δ • ύεࢦఆ͸ίϚϯυϥΠϯҾ਺Ͱ౉ͤΔΑ͏࣮૷͓ͯ͘͠ͱศར from tensorflow.python.lib.io import file_io file_io.list_directory() file_io.create_dir(path) file_io.file_exists() with file_io.FileIO(filename, 'w') as f:

܇࿅ϓϩάϥϜΛδϣϒͱͯ͠ొ࿥ $ starchart train \ -m dcgan \ # MODEL_NAME -M trainer.task \ # MODULE_NAME -- \ --train_dir=TRAIN_PATH/model \ # YOUR_TRAIN_PARAMS --images_dir=TRAIN_PATH/images \ --data_dir=gs://$BUCKET_NAME/data/dcgan

δϣϒͷ࣮ߦΛ଴ͭ $ starchart state -m dcgan jobId: dcgan_20170125194440 (SUCCESSED) jobId: dcgan_20170125191521 (FAILED) • δϣϒIDͱεςʔλεΛ֬ೝ

ϞσϧΛެ։͢Δ $ starchart expose -m dcgan • ੒ޭͨ͠δϣϒΛݩʹϞσϧΛొ࿥ • Ϟσϧొ࿥࣌ʹόʔδϣϯ΋ొ࿥ͯ͠༧ଌαʔϏεAPIͱͯ͠ެ։ • όʔδϣϯ໊͸ v + δϣϒ໊ model v20170125194440 (default) Cloud ML & Storage

Not working…

༧ଌαʔϏε"1*ͷ࢓૊Έʢٖࣅίʔυʣ request_params = {'instances': [{'sample_inputs': np.zeros((1, 40)).tolist()}]} def feed_from_request(request, tensor_keys): feed = {} request_keys = request['instances'][0].keys() for key in request_keys: feed[tensor_keys[key]] = [instance[key] for instance in request['instances']] return feed with tf.Session() as sess: new_saver = tf.train.import_meta_graph(‘TRAIN_PATH/model/export.meta’) new_saver.restore(sess, ‘TRAIN_PATH/model/export’) tensor_keys = json.loads(tf.get_collection('inputs')[0]) feed = feed_from_request(request_params, tensor_keys) op = json.loads(tf.get_collection('outputs')[0]) result = sess.run(op, feed_dict=feed) print(result) ֶश݁Ռͷ.FUB(SBQIΛ෮ݩ ίϨΫγϣϯ JOQVUT ͷςϯι ϧͱ"1*ϦΫΤετύϥϝλΛ ඥ෇͚ ίϨΫγϣϯ PVUQVUT ͷςϯ ιϧΛΦϖϨʔγϣϯͱͯ͠ඥ ෇͚ͨGFFEΛҾ਺ʹ࣮ߦ

܇࿅ϓϩάϥϜΛ"1*ʹରԠͤ͞Δ saver.save(sess, os.path.join(FLAGS.train_dir, 'export')) • APIͰར༻͢ΔͨΊɺ࠷ऴͷֶश݁ՌΛΤΫεϙʔτ͢Δ • ΤΫεϙʔτ໊͸exportͰͳ͚Ε͹ͳΒͳ͍ • StarChartͷ৔߹ɺ`TRAIN_PATH/model/export` ͱͯ͠ग़ྗ͠ͳ͚Ε͹ͳΒͳ͍

• ίϨΫγϣϯʹೖྗ༻ςϯιϧΛ௥Ճɻग़ྗςϯιϧͷ࣮ߦ࣌ʹfeed_dictͱͯ͠౉ ͢ύϥϝλΛࢦఆ • ίϨΫγϣϯʹग़ྗ༻ςϯιϧΛ௥Ճɻ༧ଌαʔϏεAPI࣮ߦ࣌ͷΦϖϨʔγϣϯ Λࢦఆ • ग़ྗ༻ςϯιϧʹ `tf.image.encode_jpeg`Λ࢖͏ͱInternal Server Error ͩͬͨͷͰtf.reshape(tf.squeeze(image, [0]), [1, -1]) ͱͨ͠ # Input sample_inputs = tf.placeholder(tf.float32, shape=(None, 1, dcgan.z_dim)) tf.add_to_collection('inputs', json.dumps({'sample_inputs': sample_inputs.name})) # Output sample_outputs = dcgan.sample_image_vectors(1, 1, inputs=sample_inputs[0]) tf.add_to_collection('outputs', json.dumps({'sample_outputs': sample_outputs.name})) ܇࿅ϓϩάϥϜΛ"1*ʹରԠͤ͞Δ

܇࿅ϓϩάϥϜΛδϣϒͱͯ͠ొ࿥ $ starchart train \ -m dcgan \ # MODEL_NAME -M trainer.task \ # MODULE_NAME -- \ --train_dir=TRAIN_PATH/model \ # YOUR_TRAIN_PARAMS --images_dir=TRAIN_PATH/images \ --data_dir=gs://$BUCKET_NAME/data/dcgan

δϣϒͷ࣮ߦΛ଴ͭ $ starchart state -m dcgan jobId: dcgan_20170125201233 (SUCCESSED) jobId: dcgan_20170125194440 (SUCCESSED) jobId: dcgan_20170125191521 (FAILED) • δϣϒIDͱεςʔλεΛ֬ೝ

ϞσϧΛެ։͢Δ $ starchart expose -m dcgan model v20170125194440 (default) Cloud ML & Storage v20170125201233

ϞσϧϑΝΠϧΛGit؅ཧ͢Δ . #"" dcgan $ #"" setup.py $ !"" trainer $ #"" __init__.py $ #"" dcgan.py $ !"" task.py !"" dcgan.json exposeͨ݁͠Ռ͕`Ϟσϧ໊.json`ʹอ ଘ͞ΕΔɻσϑΥϧτόʔδϣϯͷ੾ ସʹ࢖͏ͷͰ͜Ε΋Git؅ཧͱ͢Δ

ϞσϧϑΝΠϧΛGit؅ཧ͢Δ { "model": "MODEL_NAME", "versions": [ { "version": { "name": "projects/PROJECT_ID/models/MODEL_NAME/versions/v20170111170842", "deploymentUri": "gs://PROJECT_ID-ml/MODEL_NAME/20170111170842/model", "createTime": "2017-01-11T09:12:54Z", "job": { "jobId": "MODEL_NAME_20170111170842", "trainingInput": { "packageUris": [ "gs://PROJECT_ID-ml/MODEL_NAME/20170111170842/packages/trainer-0.0.0.tar.gz" ], "pythonModule": "trainer.task", "args": [ "--model_dir=gs://PROJECT_ID-ml/MODEL_NAME/20170111170842/model", "--train_dir=gs://PROJECT_ID-ml/MODEL_NAME/20170111170842/train", ], "region": "us-central1" }, "createTime": "2017-01-11T08:08:49Z", "startTime": "2017-01-11T08:13:55Z", "endTime": "2017-01-11T08:40:55Z", "state": "SUCCEEDED", "trainingOutput": { "consumedMLUnits": 0.45 } }, "isDefault": true } } ] } "1*όʔδϣϯʹඥͮ͘δϣϒ ΍࣮ߦ࣌ύϥϝλɺσϑΥϧτ όʔδϣϯ͔Ͳ͏͔֬ೝͰ͖Δ

༧ଌαʔϏε"1*Λ࢖͏ project = 'project-123456' model = 'dcgan' version = 'v20170125194440' credentials = GoogleCredentials.get_application_default() ml = discovery.build('ml', 'v1beta1', credentials=credentials) body = {'instances': [{'sample_inputs': np.zeros((1, 40)).tolist()}]} request = ml.projects().predict(name='projects/{}/models/{}/versions/{}'.format(project, model, version), body=body) try: response = request.execute() output = response['predictions'][0]['sample_outputs'] with tf.Session() as sess: image = sess.run(tf.image.encode_jpeg(tf.reshape(tf.constant(output, dtype=tf.uint8), [96, 96, 3]))) with file_io.FileIO('out.jpg', 'w') as f: f.write(image) except errors.HttpError as err: print(err._get_reason())

It works !!

ϞσϧͷσϑΥϧτόʔδϣϯΛมߋ͢Δ $ starchart apply -m dcgan model v20170125194440 Cloud ML & Storage v20170125201233 (default) • ϞσϧϑΝΠϧΛฤू͠ɺσϑΥϧτͱ͍ͨ͠όʔδϣϯͷ `isDefault` Λ true ʹɻ • ͜ͷ࣌఺ͷ܇࿅ϓϩάϥϜͱϞσϧϑΝΠϧΛPullRequestͱ͢Δ • ϨϏϡʔͰσϑΥϧτόʔδϣϯͷج४Λຬ͍ͨͯͨ͠ΒϚʔδͯ͠apply

4UBS$IBSUʹΑΔӡ༻ͷ͓͞Β͍ •Ϟσϧ͝ͱͷ܇࿅ϓϩάϥϜΛGit؅ཧ •train -> expose -> ϨϏϡʔ -> apply Λ܁Γฦ͢ Easy & Useful

·ͱΊ

·ͱΊ •ػցֶशΛαʔϏεར༻͢ΔͨΊʹػցֶशج൫Λݕ ౼ͨ͠ •Google Cloud MLʹΑΔߏஙɺӡ༻ΛߦͬͯΈͨ •ෆศͳ఺͸StarChartͰվળͨ͠ •ػցֶशͰαʔϏεվળ͠Α͏ʂʂ

͓ΘΓ

ϖύϘΧϨοδظੜืूத ʙ෱ԬͰ׆༂͍ͨ͠ʂ8FCΞϓϦέʔγϣϯΤϯδχΞʙ ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU