はじめてのエッジ推論 / osc2019niigata

はじめてのエッジ推論 / osc2019niigata

オープンソースカンファレンス2019新潟で発表した資料

82d6167c4d14393c2e20b37a74b363c5?s=128

kasacchiful

October 05, 2019
Tweet

Transcript

  1. ͸͡ΊͯͷΤοδਪ࿦ TensorFlow LiteͱTensorFlow.jsͰ͋ͦͿ 2019-10-05 ΦʔϓϯιʔεΧϯϑΝϨϯε2019৽ׁ ּݪ ޺ (@kasacchiful)

  2. ࣗݾ঺հ • ּݪ ޺ (@kasacchiful) • ৽ׁࢢࡏॅ • ޷͖ͳݴޠ: Ruby

    • ίϛϡχςΟ: JAWS-UG৽ׁʗJaSST৽ׁ࣮ߦҕһձʗ ASTERਖ਼ձһʗSWANIIʗetc. • Pythonػցֶशษڧձin৽ׁ΍TensorFlow UG৽ׁʹ΋ࢀՃ
  3. ࡢ೥ͷOSC৽ׁ https://speakerdeck.com/kasacchiful/osc2018niigata

  4. ຊ೔ͷ಺༰ ͸͡ΊͯͷΤοδਪ࿦ TensorFlow Lite ͱ TensorFlow.js Ͱ͋ͦͿ • ػցֶशͷਪ࿦ΛΤοδσόΠεଆͰ࣮ߦ͢Δέʔε͕૿͖͍͑ͯͯ·͢ɻ •

    ࠓճ͸ɺϞόΠϧʗIoTσόΠε޲͚ͷϑϨʔϜϫʔΫͰ͋Δ ʮTensorFlow LiteʯͱJavaScriptϥΠϒϥϦͷʮTensorFlow.jsʯΛ࢖ͬͯ ৭ʑ༡ΜͰΈ·͢ɻ • ର৅: ػցֶशͷ৘ใऩूΛ໨తͱ͍ͯ͠Δਓ • Ϩϕϧ: ೖ໳ฤͰ͸ͳ͍
  5. ֶशͱਪ࿦  ($!$#) "$"  ( ) $ "$" $

     ( )  ($!$#)   " ֶश(܇࿅) ਪ࿦
  6. ͳͥΤοδσόΠεଆͰ ਪ࿦ͳͷ͔

  7. ҰൠతͳϓϩμΫτʹ͓͚Δ ਪ࿦ͷྲྀΕ 2. σʔλૹ৴ 3. αʔόଆͰਪ࿦ 4. ਪ࿦݁Ռड৴ 1. σʔλऔಘ

    5. ਪ࿦݁Ռදࣔ
  8. ΤοδσόΠεଆͰͷ ਪ࿦ͷྲྀΕ 2. σόΠεଆͰਪ࿦ 1. σʔλऔಘ 3. ਪ࿦݁Ռදࣔ

  9. Ϟνϕʔγϣϯ • ϨΠςϯγ • αʔόͷԠ౴଴͕ͪෆཁ ɹίωΫςΟϏςΟ • Πϯλʔωοτ΁ͷ઀ଓ͕ ෆཁ •

    ফඅిྗ • ωοτϫʔΫ઀ଓ͸ిྗΛ ৯͏ • ϓϥΠόγʔ • σόΠε͔ΒσʔλΛऔಘ ͢Δඞཁ͕ͳ͍ • αʔόʹૹΔσʔλΛඞཁ ࠷খݶʹͰ͖Δ
  10. ઌਐςΫϊϩδͷϋΠϓαΠΫϧ: 2019೥ @Ψʔτφʔ ग़యIUUQTXXXHBSUOFSDPNKQOFXTSPPNQSFTTSFMFBTFTQS

  11. ϞσϧͷܰྔԽ • ֶशࡁϞσϧ͕਺ඦMB΍਺GBͩͱɺϞόΠϧΞϓϦʹಉࠝ͢Δͱμ΢ϯϩʔυʹ ͕͔͔࣌ؒΓ͗͢Δ͠ɺεϖοΫͷখ͞ͳσόΠεͰ͸ɺͦ΋ͦ΋ϝϞϦʹ৐Βͳ ͍Մೳੑ΋ • Ϟσϧ͕খ͚͞Ε͹ਪ࿦଎౓͸଎͍ɻਪ࿦଎౓͕஗͍ͱɺεϖοΫͷখ͞ͳσόΠ εͰ͸࢖͍෺ʹͳΒͳ͍ • ΤοδσόΠεͰ͸GPU͕࢖͑ͳ͍έʔε͕΄ͱΜͲͳͷͰɺͳ͓͞Βεϐʔυ͕ٻΊΒ

    ΕΔ • SqueezeNet΍MobileNet౳ɺܰྔͳϞσϧͰ࠷దԽ • ྔࢠԽ (Quantization) ʗৠཹ (Distillation) ʗࢬמΓ (Pruning) ͳͲ ਫ਼౓Λग़དྷΔ͚ͩམͱͣ͞ʹɺ଎౓޲্΍ϝϞϦ࢖༻ྔͳͲΛ཈͑Δ౳ ੑೳ޲্Λ໨ࢦͨ͢Ίʹɺݚڀ͕੝Μͳ෼໺
  12. 1. TensorFlow LiteͰ͋ͦͿ

  13. None
  14. TensorFlow Liteͱ͸Կ͔ • σόΠε্Ͱͷਪ࿦ΛՄೳʹ͢ΔϑϨʔϜϫʔΫ • Android / iOS / Raspberry

    Pi / etc. • 2ͭͷϝΠϯίϯϙʔωϯτ • TensorFlow Lite interpreter (࣮ߦ؀ڥ) • TensorFlow Lite converter (ίϯόʔλ) • TensorFlowͰߏஙֶͨ͠शࡁϞσϧΛɺίϯόʔλͰม׵ • ͦͷࡍʹ֤छ࠷దԽॲཧʹΑΓܰྔԽ͞ΕΔ
  15. Edge TPU USB Accelerator • USB઀ଓͷTPUσόΠε • Raspberry PiͰͷਪ࿦Λߴ଎ʹ •

    ໿1ສԁ͘Β͍ͰߪೖՄೳ • ϒʔεͰσϞ΍ͬͯ·͢
  16. TensorFlow LiteΛ࢖༻͢Δࡍͷ ࡞ۀͷྲྀΕ 1. Ϟσϧߏஙͱֶश • TensorFlowͰߦ͍·͢ 2. ϞσϧΛίϯόʔτ •

    TensorFlow Lite ConverterΛ࢖༻͠·͢ 3. ίϯόʔτͨ͠ϞσϧΛσϓϩΠ
  17. TensorFlow Liteίϯόʔλ # ެࣜͰ͸ɺPythonεΫϦϓτ্Ͱίϯόʔτ͢Δ͜ͱΛਪ঑͍ͯ͠Δ import tensorflow as tf model =

    tf.keras.models.Sequential() ... model.compile(...) model.fit(...) # Ϟσϧͷίϯόʔτ converter = tf.lite.TFLiteConverter.from_keras_model(model) tflite_model = converter.convert() # ϑΝΠϧʹॻ͖ग़͠ open("converted_model.tflite", "wb").write(tflite_model) Ҿ༻IUUQTXXXUFOTPSqPXPSHMJUFDPOWFSUQZUIPO@BQJ
  18. PythonͰͷಡΈࠐΈͱਪ࿦ import numpy as np import tensorflow as tf #

    ϞσϧͷಡΈࠐΈ interpreter = tf.lite.Interpreter(model_path="converted_model.tflite") interpreter.allocate_tensors() # ೖग़ྗtensorΛऔಘ input_details = interpreter.get_input_details() output_details = interpreter.get_output_details() # ϥϯμϜೖྗσʔλͰϞσϧΛςετ input_shape = input_details[0]['shape'] input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) interpreter.set_tensor(input_details[0]['index'], input_data) # ਪ࿦࣮ߦ interpreter.invoke() # ਪ࿦݁ՌΛऔಘ output_data = interpreter.get_tensor(output_details[0]['index']) print(output_data) Ҿ༻IUUQTXXXUFOTPSqPXPSHMJUFHVJEFJOGFSFODF
  19. ࠓճͷTF Lite͋ͦͼͷ಺༰ • ෺ମݕग़ (Object Detection) • TensorFlow LiteͷWebαΠτʹͯެ։͞Ε͍ͯΔɺֶ शࡁϞσϧΛ࢖༻͠·͢

    • ྔࢠԽ͞ΕͨCOCO SSD MobileNet v1ͷϞσϧ • https://www.tensorflow.org/lite/models/object_detection/overview? hl=ja • iPod Touch (ୈ6ੈ୅) Ͱ࣮ߦͯ͠Έ·͢
  20. TensorFlow Lite σϞ

  21. ެࣜαΠτʹαϯϓϧ৭ʑ https://www.tensorflow.org/lite/examples/?hl=ja

  22. 2. TensorFlow.jsͰ͋ͦͿ

  23. None
  24. TensorFlow.jsͱ͸Կ͔ • JavaScript্Ͱಈ࡞͢ΔϑϨʔϜϫʔΫ • Webϒϥ΢β্Ͱར༻Ͱ͖Δ͠ɺNode.jsͰαʔόαΠυར༻΋Մೳ • Raspberry Pi΍obnizͰ΋ಈ࡞͢Δ • ਪ࿦͚ͩͰͳֶ͘श΋Մೳ

    • WebGLܦ༝ͰGPU΋࢖͑Δ • Node.jsͰ͋Ε͹ɺCUDAܦ༝ͰNVIDIA GPUΛ࢖༻͢Δ͜ͱ͕Մೳ • TensorFlowͰߏஙֶͨ͠शࡁϞσϧΛɺίϯόʔλͰม׵ͯ͠࢖༻͢ Δέʔε͕ଟ͍
  25. TensorFlow.jsͰͷֶश // Define a model for linear regression. const model

    = tf.sequential(); model.add(tf.layers.dense({units: 1, inputShape: [1]})); // Prepare the model for training: Specify the loss and the optimizer. model.compile({loss: 'meanSquaredError', optimizer: 'sgd'}); // Generate some synthetic data for training. const xs = tf.tensor2d([1,2,3,4], [4,1]); const ys = tf.tensor2d([1,3,5,7], [4,1]); // Train the model using the data. model.fit(xs, ys).then(() => { // Use the model to do inference on a data point the model hasn't seen before: // Open the browser devtools to see the output model.predict(tf.tensor2d([5], [1,1]).print()); }); Ҿ༻5SZ5FOTPS'MPXKT IUUQTDPEFQFOJPQFO FEJUBCMFUSVFFEJUPST
  26. TensorFlow.jsίϯόʔλ # Install (PythonͷϥΠϒϥϦ) pip install tensorflowjs # ίϚϯυϥΠϯ্Ͱίϯόʔτ tensorflowjs_converter

    --input_format keras \ path/to/model.h5 \ path/to/tfjs_target_dir Ҿ༻IUUQTXXXUFOTPSqPXPSHKTUVUPSJBMTDPOWFSTJPOJNQPSU@LFSBT
  27. TensorFlow.jsίϯόʔλ # PythonεΫϦϓτ্Ͱ௚઀ίϯόʔτ import tensorflow as tf import tensorflowjs as

    tfjs model = tf.keras.models.Sequential() ... model.compile(...) model.fit(...) tfjs.converters.save_keras_model(model, tfjs_target_dir) Ҿ༻IUUQTXXXUFOTPSqPXPSHKTUVUPSJBMTDPOWFSTJPOJNQPSU@LFSBT
  28. ίϯόʔτͨ͠ϞσϧϑΝΠϧ # த਎͸ɺ͍͔ͭ͘ʹ෼ׂ͞ΕͨϞσϧϑΝΠϧͷ࣮ଶͱɺmodel.json ls tfjs_target_dir/ group1-shard10of17 group1-shard13of17 group1-shard16of17 group1- shard2of17

    group1-shard5of17 group1-shard8of17 group1-shard11of17 group1-shard14of17 group1-shard17of17 group1- shard3of17 group1-shard6of17 group1-shard9of17 group1-shard12of17 group1-shard15of17 group1-shard1of17 group1- shard4of17 group1-shard7of17 model.json
  29. JavaScriptͰͷಡΈࠐΈ // model.jsonΛಡΈࠐΈ࣌ʹࢦఆ͢Ε͹ྑ͍ import * as tf from '@tensorflow/tfjs'; const

    model = await tf.loadLayersModel(‘./path/to/model.json'); // ਪ࿦ͷྫ const example = tf.fromPixels(webcamElement); const prediction = model.predict(example); Ҿ༻IUUQTXXXUFOTPSqPXPSHKTUVUPSJBMTDPOWFSTJPOJNQPSU@LFSBT
  30. ࠓճͷTF.js͋ͦͼͷ಺༰ • PoseNetͰ࢟੎ਪఆ (Pose Estimation) • WebΧϝϥΛ࢖ͬͯɺWebϒϥ΢β্Ͱਪ࿦

  31. PoseNet • ϦΞϧλΠϜ࢟੎ਪఆͰ͖ΔϞ σϧ • MobileNetϕʔεͳͷͰɺਪ࿦ಈ ࡞͕֓Ͷܰշ • ࠷ۙ͸ResNetϕʔε΋࢖͑Δ •

    TensorFlow.jsͷσϞͷ1ͭʹ͋ Γ·͢ • https://github.com/tensorflow/tfjs-models/tree/ master/posenet *NBHFTIUUQTNFEJVNDPNUFOTPSqPXSFBMUJNFIVNBOQPTFFTUJNBUJPOJOUIFCSPXTFSXJUIUFOTPSqPXKTEECDDE
  32. PoseNetͰऔಘͰ͖Δ෦Ґ *NBHFTIUUQTNFEJVNDPNUFOTPSqPXSFBMUJNFIVNBOQPTFFTUJNBUJPOJOUIFCSPXTFSXJUIUFOTPSqPXKTEECDDE

  33. TensorFlow.js σϞ

  34. ެࣜαΠτʹσϞ৭ʑ https://www.tensorflow.org/js/demos/?hl=ja

  35. ·ͱΊ

  36. ·ͱΊ • ΤοδσόΠεଆͰͷਪ࿦͸ݱࡏ੝Μͳ෼໺ • TensorFlow Lite΍TensorFlow.jsΛ࢖ͬͯ༡Μ ͰΈ·͠ΐ͏