͸͡ΊͯͷΤοδਪ࿦ TensorFlow LiteͱTensorFlow.jsͰ͋ͦͿ 2019-10-05 ΦʔϓϯιʔεΧϯϑΝϨϯε2019৽ׁ ּݪ ޺ (@kasacchiful)

ࣗݾ঺հ • ּݪ ޺ (@kasacchiful) • ৽ׁࢢࡏॅ • ޷͖ͳݴޠ: Ruby • ίϛϡχςΟ: JAWS-UG৽ׁʗJaSST৽ׁ࣮ߦҕһձʗ ASTERਖ਼ձһʗSWANIIʗetc. • Pythonػցֶशษڧձin৽ׁ΍TensorFlow UG৽ׁʹ΋ࢀՃ

ࡢ೥ͷOSC৽ׁ https://speakerdeck.com/kasacchiful/osc2018niigata

ຊ೔ͷ಺༰ ͸͡ΊͯͷΤοδਪ࿦ TensorFlow Lite ͱ TensorFlow.js Ͱ͋ͦͿ • ػցֶशͷਪ࿦ΛΤοδσόΠεଆͰ࣮ߦ͢Δέʔε͕૿͖͍͑ͯͯ·͢ɻ • ࠓճ͸ɺϞόΠϧʗIoTσόΠε޲͚ͷϑϨʔϜϫʔΫͰ͋Δ ʮTensorFlow LiteʯͱJavaScriptϥΠϒϥϦͷʮTensorFlow.jsʯΛ࢖ͬͯ ৭ʑ༡ΜͰΈ·͢ɻ • ର৅: ػցֶशͷ৘ใऩूΛ໨తͱ͍ͯ͠Δਓ • Ϩϕϧ: ೖ໳ฤͰ͸ͳ͍

ֶशͱਪ࿦ ($!$#) "$" ( ) $ "$" $ ( ) ($!$#) " ֶश(܇࿅) ਪ࿦

ͳͥΤοδσόΠεଆͰ ਪ࿦ͳͷ͔

ҰൠతͳϓϩμΫτʹ͓͚Δ ਪ࿦ͷྲྀΕ 2. σʔλૹ৴ 3. αʔόଆͰਪ࿦ 4. ਪ࿦݁Ռड৴ 1. σʔλऔಘ 5. ਪ࿦݁Ռදࣔ

ΤοδσόΠεଆͰͷ ਪ࿦ͷྲྀΕ 2. σόΠεଆͰਪ࿦ 1. σʔλऔಘ 3. ਪ࿦݁Ռදࣔ

Ϟνϕʔγϣϯ • ϨΠςϯγ • αʔόͷԠ౴଴͕ͪෆཁ ɹίωΫςΟϏςΟ • Πϯλʔωοτ΁ͷ઀ଓ͕ ෆཁ • ফඅిྗ • ωοτϫʔΫ઀ଓ͸ిྗΛ ৯͏ • ϓϥΠόγʔ • σόΠε͔ΒσʔλΛऔಘ ͢Δඞཁ͕ͳ͍ • αʔόʹૹΔσʔλΛඞཁ ࠷খݶʹͰ͖Δ

ઌਐςΫϊϩδͷϋΠϓαΠΫϧ: 2019೥ @Ψʔτφʔ ग़యIUUQTXXXHBSUOFSDPNKQOFXTSPPNQSFTTSFMFBTFTQS

ϞσϧͷܰྔԽ • ֶशࡁϞσϧ͕਺ඦMB΍਺GBͩͱɺϞόΠϧΞϓϦʹಉࠝ͢Δͱμ΢ϯϩʔυʹ ͕͔͔࣌ؒΓ͗͢Δ͠ɺεϖοΫͷখ͞ͳσόΠεͰ͸ɺͦ΋ͦ΋ϝϞϦʹ৐Βͳ ͍Մೳੑ΋ • Ϟσϧ͕খ͚͞Ε͹ਪ࿦଎౓͸଎͍ɻਪ࿦଎౓͕஗͍ͱɺεϖοΫͷখ͞ͳσόΠ εͰ͸࢖͍෺ʹͳΒͳ͍ • ΤοδσόΠεͰ͸GPU͕࢖͑ͳ͍έʔε͕΄ͱΜͲͳͷͰɺͳ͓͞Βεϐʔυ͕ٻΊΒ ΕΔ • SqueezeNet΍MobileNet౳ɺܰྔͳϞσϧͰ࠷దԽ • ྔࢠԽ (Quantization) ʗৠཹ (Distillation) ʗࢬמΓ (Pruning) ͳͲ ਫ਼౓Λग़དྷΔ͚ͩམͱͣ͞ʹɺ଎౓޲্΍ϝϞϦ࢖༻ྔͳͲΛ཈͑Δ౳ ੑೳ޲্Λ໨ࢦͨ͢Ίʹɺݚڀ͕੝Μͳ෼໺

1. TensorFlow LiteͰ͋ͦͿ

No content

TensorFlow Liteͱ͸Կ͔ • σόΠε্Ͱͷਪ࿦ΛՄೳʹ͢ΔϑϨʔϜϫʔΫ • Android / iOS / Raspberry Pi / etc. • 2ͭͷϝΠϯίϯϙʔωϯτ • TensorFlow Lite interpreter (࣮ߦ؀ڥ) • TensorFlow Lite converter (ίϯόʔλ) • TensorFlowͰߏஙֶͨ͠शࡁϞσϧΛɺίϯόʔλͰม׵ • ͦͷࡍʹ֤छ࠷దԽॲཧʹΑΓܰྔԽ͞ΕΔ

Edge TPU USB Accelerator • USB઀ଓͷTPUσόΠε • Raspberry PiͰͷਪ࿦Λߴ଎ʹ • ໿1ສԁ͘Β͍ͰߪೖՄೳ • ϒʔεͰσϞ΍ͬͯ·͢

TensorFlow LiteΛ࢖༻͢Δࡍͷ ࡞ۀͷྲྀΕ 1. Ϟσϧߏஙͱֶश • TensorFlowͰߦ͍·͢ 2. ϞσϧΛίϯόʔτ • TensorFlow Lite ConverterΛ࢖༻͠·͢ 3. ίϯόʔτͨ͠ϞσϧΛσϓϩΠ

TensorFlow Liteίϯόʔλ # ެࣜͰ͸ɺPythonεΫϦϓτ্Ͱίϯόʔτ͢Δ͜ͱΛਪ঑͍ͯ͠Δ import tensorflow as tf model = tf.keras.models.Sequential() ... model.compile(...) model.fit(...) # Ϟσϧͷίϯόʔτ converter = tf.lite.TFLiteConverter.from_keras_model(model) tflite_model = converter.convert() # ϑΝΠϧʹॻ͖ग़͠ open("converted_model.tflite", "wb").write(tflite_model) Ҿ༻IUUQTXXXUFOTPSqPXPSHMJUFDPOWFSUQZUIPO@BQJ

PythonͰͷಡΈࠐΈͱਪ࿦ import numpy as np import tensorflow as tf # ϞσϧͷಡΈࠐΈ interpreter = tf.lite.Interpreter(model_path="converted_model.tflite") interpreter.allocate_tensors() # ೖग़ྗtensorΛऔಘ input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() # ϥϯμϜೖྗσʔλͰϞσϧΛςετ input_shape = input_details[0]['shape'] input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) interpreter.set_tensor(input_details[0]['index'], input_data) # ਪ࿦࣮ߦ interpreter.invoke() # ਪ࿦݁ՌΛऔಘ output_data = interpreter.get_tensor(output_details[0]['index']) print(output_data) Ҿ༻IUUQTXXXUFOTPSqPXPSHMJUFHVJEFJOGFSFODF

ࠓճͷTF Lite͋ͦͼͷ಺༰ • ෺ମݕग़ (Object Detection) • TensorFlow LiteͷWebαΠτʹͯެ։͞Ε͍ͯΔɺֶ शࡁϞσϧΛ࢖༻͠·͢ • ྔࢠԽ͞ΕͨCOCO SSD MobileNet v1ͷϞσϧ • https://www.tensorflow.org/lite/models/object_detection/overview? hl=ja • iPod Touch (ୈ6ੈ୅) Ͱ࣮ߦͯ͠Έ·͢

TensorFlow Lite σϞ

ެࣜαΠτʹαϯϓϧ৭ʑ https://www.tensorflow.org/lite/examples/?hl=ja

2. TensorFlow.jsͰ͋ͦͿ

No content

TensorFlow.jsͱ͸Կ͔ • JavaScript্Ͱಈ࡞͢ΔϑϨʔϜϫʔΫ • Webϒϥ΢β্Ͱར༻Ͱ͖Δ͠ɺNode.jsͰαʔόαΠυར༻΋Մೳ • Raspberry Pi΍obnizͰ΋ಈ࡞͢Δ • ਪ࿦͚ͩͰͳֶ͘श΋Մೳ • WebGLܦ༝ͰGPU΋࢖͑Δ • Node.jsͰ͋Ε͹ɺCUDAܦ༝ͰNVIDIA GPUΛ࢖༻͢Δ͜ͱ͕Մೳ • TensorFlowͰߏஙֶͨ͠शࡁϞσϧΛɺίϯόʔλͰม׵ͯ͠࢖༻͢ Δέʔε͕ଟ͍

TensorFlow.jsͰͷֶश // Define a model for linear regression. const model = tf.sequential(); model.add(tf.layers.dense({units: 1, inputShape: [1]})); // Prepare the model for training: Specify the loss and the optimizer. model.compile({loss: 'meanSquaredError', optimizer: 'sgd'}); // Generate some synthetic data for training. const xs = tf.tensor2d([1,2,3,4], [4,1]); const ys = tf.tensor2d([1,3,5,7], [4,1]); // Train the model using the data. model.fit(xs, ys).then(() => { // Use the model to do inference on a data point the model hasn't seen before: // Open the browser devtools to see the output model.predict(tf.tensor2d([5], [1,1]).print()); }); Ҿ༻5SZ5FOTPS'MPXKT IUUQTDPEFQFOJPQFO FEJUBCMFUSVFFEJUPST

TensorFlow.jsίϯόʔλ # Install (PythonͷϥΠϒϥϦ) pip install tensorflowjs # ίϚϯυϥΠϯ্Ͱίϯόʔτ tensorflowjs_converter --input_format keras \ path/to/model.h5 \ path/to/tfjs_target_dir Ҿ༻IUUQTXXXUFOTPSqPXPSHKTUVUPSJBMTDPOWFSTJPOJNQPSU@LFSBT

TensorFlow.jsίϯόʔλ # PythonεΫϦϓτ্Ͱ௚઀ίϯόʔτ import tensorflow as tf import tensorflowjs as tfjs model = tf.keras.models.Sequential() ... model.compile(...) model.fit(...) tfjs.converters.save_keras_model(model, tfjs_target_dir) Ҿ༻IUUQTXXXUFOTPSqPXPSHKTUVUPSJBMTDPOWFSTJPOJNQPSU@LFSBT

ίϯόʔτͨ͠ϞσϧϑΝΠϧ # த਎͸ɺ͍͔ͭ͘ʹ෼ׂ͞ΕͨϞσϧϑΝΠϧͷ࣮ଶͱɺmodel.json ls tfjs_target_dir/ group1-shard10of17 group1-shard13of17 group1-shard16of17 group1- shard2of17 group1-shard5of17 group1-shard8of17 group1-shard11of17 group1-shard14of17 group1-shard17of17 group1- shard3of17 group1-shard6of17 group1-shard9of17 group1-shard12of17 group1-shard15of17 group1-shard1of17 group1- shard4of17 group1-shard7of17 model.json

JavaScriptͰͷಡΈࠐΈ // model.jsonΛಡΈࠐΈ࣌ʹࢦఆ͢Ε͹ྑ͍ import * as tf from '@tensorflow/tfjs'; const model = await tf.loadLayersModel(‘./path/to/model.json'); // ਪ࿦ͷྫ const example = tf.fromPixels(webcamElement); const prediction = model.predict(example); Ҿ༻IUUQTXXXUFOTPSqPXPSHKTUVUPSJBMTDPOWFSTJPOJNQPSU@LFSBT

ࠓճͷTF.js͋ͦͼͷ಺༰ • PoseNetͰ࢟੎ਪఆ (Pose Estimation) • WebΧϝϥΛ࢖ͬͯɺWebϒϥ΢β্Ͱਪ࿦

PoseNet • ϦΞϧλΠϜ࢟੎ਪఆͰ͖ΔϞ σϧ • MobileNetϕʔεͳͷͰɺਪ࿦ಈ ࡞͕֓Ͷܰշ • ࠷ۙ͸ResNetϕʔε΋࢖͑Δ • TensorFlow.jsͷσϞͷ1ͭʹ͋ Γ·͢ • https://github.com/tensorflow/tfjs-models/tree/ master/posenet *NBHFTIUUQTNFEJVNDPNUFOTPSqPXSFBMUJNFIVNBOQPTFFTUJNBUJPOJOUIFCSPXTFSXJUIUFOTPSqPXKTEECDDE

PoseNetͰऔಘͰ͖Δ෦Ґ *NBHFTIUUQTNFEJVNDPNUFOTPSqPXSFBMUJNFIVNBOQPTFFTUJNBUJPOJOUIFCSPXTFSXJUIUFOTPSqPXKTEECDDE

TensorFlow.js σϞ

ެࣜαΠτʹσϞ৭ʑ https://www.tensorflow.org/js/demos/?hl=ja

·ͱΊ

·ͱΊ • ΤοδσόΠεଆͰͷਪ࿦͸ݱࡏ੝Μͳ෼໺ • TensorFlow Lite΍TensorFlow.jsΛ࢖ͬͯ༡Μ ͰΈ·͠ΐ͏