Slide 1

Slide 1 text

2020-07-29 JUNARA (JUNGO ARAKI) Flutter mlkit pluginͰFirebase ML kit ΧελϜϞσ ϧΛ࢖͏ ୈ21ճΏΔ͸ͪ.it: Flutterʹ͍ͭͯΏΔ͘࿩͢

Slide 2

Slide 2 text

• ߥ໦ ३ޗ • ෎தࢢࡏॅΤϯδχΞ • Ն͸ɺന۽Λ৯΂Δͷָ͕͠ΈͰ͢ • ओʹRuby on RailsΛ࢖ͬͨγεςϜ։ൃ • ͏͗͞ͷࣂҭه࿥ΞϓϦΛ࡞ΔͨΊɺFlutter Λ΍ͬͯ·͢ ࣗݾ঺հ

Slide 3

Slide 3 text

ຊ୊

Slide 4

Slide 4 text

FlutterͰ ը૾ೝ͍ࣝͨ͠

Slide 5

Slide 5 text

͏͗͞ͷϑϯͷը૾Λ ֶशͤͯ͞ ਺͍͑ͨ

Slide 6

Slide 6 text

Flutterͱ͍͑͹ Firebase

Slide 7

Slide 7 text

• ML Kit ͸ɺGoogle ͷػցֶशͷػೳΛ Android ΞϓϦ΍ iOS ΞϓϦͱͯ͠ఏڙ͢ΔϞόΠϧ SDK Ͱ͢ɻ • ҰൠతͳϢʔεέʔε • ςΩετͷೝࣝɺإݕग़ɺϥϯυϚʔΫͷೝࣝɺόʔίʔυͷεΩϟϯɺը૾ͷϥϕϧ෇͚ɺςΩετͷݴޠͷࣝผ ͳͲ • σόΠε / Ϋϥ΢υͷ྆ํʹରԠ • ML Kit ʹ͸ɺσόΠε·ͨ͸Ϋϥ΢υ্Ͱ࣮ߦͰ͖Δ API ͕༻ҙ͞Ε͍ͯ·͢ɻ • ΧελϜϞσϧΛσϓϩΠ • ಠࣗͷ TensorFlow Lite ϞσϧΛ͍ͭͰ΋૊ΈࠐΉ͜ͱ͕Ͱ͖·͢ɻϞσϧΛ Firebase ʹΞοϓϩʔυ͢Δ͚ͩ ͰɺϞσϧͷϗεςΟϯάͱΞϓϦʹର͢ΔαʔϏεॲཧ͸ Google ଆͰߦΘΕ·͢ɻ Firebase ML kit ͕࢖͑ͦ͏ʁ IUUQTpSFCBTFHPPHMFDPNEPDTNMLJU IMKB

Slide 8

Slide 8 text

Firebase ML kit ΧελϜϞσϧͰ Ͱ͖ͦ͏ͩͧ

Slide 9

Slide 9 text

• = TensorFlow Lite Ϟσϧ • TenforFlowͰֶशͨ͠Ϟσϧ͔Βग़ྗͨ͠tfliteόΠφϦϑΝΠϧͷ͜ͱ • → TensorFlow Lite ͷσϕϩούʔ ΨΠυ • TensorFlow Lite Ϟσϧ ϗεςΟϯά • ΞϓϦͷόΠφϦαΠζ͕ݮΔ • ୺຤্Ͱͷ ML ਪ࿦ • ࣗಈϞσϧ ϑΥʔϧόοΫ • ෳ਺ͷϞσϧιʔεΛࢦఆ͠·͢ɻΫϥ΢υʹϗετ͞Ε͍ͯΔϞσϧΛ࢖༻Ͱ͖ͳ͍৔߹͸ɺϩʔΧϧʹอଘ͞Ε͍ͯΔϞσϧ͕࢖༻͞Ε·͢ɻ • ࣗಈϞσϧߋ৽ • ΞϓϦ͕৽͍͠όʔδϣϯͷϞσϧΛࣗಈతʹμ΢ϯϩʔυ͢Δ ΧελϜϞσϧ ? IUUQTpSFCBTFHPPHMFDPNEPDTNMLJUVTFDVTUPNNPEFMT IMKB

Slide 10

Slide 10 text

ΧελϜϞσϧͷApp΁ ͷϗετΛFirebase͕Έ ͯ͘ΕΔͷ͸͏Ε͍͠

Slide 11

Slide 11 text

• TensorFlow Lite is designed to execute models efficiently on mobile and other embedded devices with limited compute and memory resources. TensorFlow models must be converted into this format before they can be used by TensorFlow Lite. • ϞόΠϧͰಈ࡞͠΍ͨ͘͢͠ϞσϧɻTensofFlow͔Βม׵ͯ͠࡞Δ • Use a pre-trained model • ͢Ͱʹม׵ࡁΈͷtfliteόΠφϦΛTensorFlow Hub͔Βೖख͔ͯͭ͠͏ • Re-train a model (transfer learning) or Train a custom model • https://codelabs.developers.google.com/codelabs/recognize-flowers-with-tensorflow-on-android/#0 (Ֆͷը૾Λ࢖ͬ ͯసҠֶशྫ) • https://www.tensorflow.org/lite/guide/get_started?hl=ja#2_convert_the_model_format TensorFlow Lite IUUQTXXXUFOTPSqPXPSHMJUF IMKB

Slide 12

Slide 12 text

TensorHub࢖͑͹ ಈ࡞ݕূͰ͖ͦ͏

Slide 13

Slide 13 text

• ΧελϜϞσϧΛ࢖͏ͳΒ mlkit ʢඇެࣜʣҰ୒ https://pub.dev/packages/mlkit • firebase_ml_vision ( https://pub.dev/packages/firebase_ml_vision ) ͸ެ͕ࣜͩɺΧελϜϞσϧ ʹ͸ඇରԠ FlutterͰFirebase ML kitΛ࢖͏ʹ͸ʁ

Slide 14

Slide 14 text

ௐ͔ࠪΒ TensorHub, Firebase,mlkit Λ࢖͑͹ݕূͰ͖ͦ͏ɹ

Slide 15

Slide 15 text

ݕূ͠Α͏

Slide 16

Slide 16 text

• FultterͰFirebase ML Kit ΧελϜϞσϧΛ࣮ࡍʹ͔ͭ͑Δ͔ • ৄࡉ • ΧελϜϞσϧΛFirebase ʹ deployͰ͖Δ͔ • TensorFlow Hub͔Β mobilenetͷϞσϧೖखͯ͠deploy • FlutterͰΧελϜϞσϧʹΑΔਪ࿦ΛϞόΠϧ୺຤্ͰͰ͖Δ͔ • mlkit repositoryͷexample appΛར༻ͯ͠ݕূ • + α • ϗετͨ͠࠷৽ͷΧελϜϞσϧ͕Flutterʹ൓ө͞ΕΔ͔ ݕূ͍ͨ͜͠ͱ

Slide 17

Slide 17 text

• mlkit exampleͰ࢖ΘΕ͍ͯΔ mobilenetΛར༻ • mobilenet͸ը૾෼ྨͷ୅දతͳϞσϧ • ࠓճͨΊͨ͠΋ͷ • mobilenet_v1_0.25_224_quantized • https://tfhub.dev/tensorflow/lite-model/mobilenet_v1_0.25_224_quantized/1/default/1 • ʢ͕࣌ؒ͋Ε͹ʣmobilenet_v1_1.0_224_quantized • https://tfhub.dev/tensorflow/coral-model/mobilenet_v1_1.0_224_quantized/1/default/1 ΧελϜϞσϧ

Slide 18

Slide 18 text

• Deployํ๏͸2छྨ • Firebaseίϯιʔϧܦ༝ • SDKར༻ • PythonͱNode͕͋Δ • ࠓճ͸PythonΛར༻ • https://github.com/firebase/quickstart-python/tree/master/machine-learning ΧελϜϞσϧΛFirebase ʹ deployͰ͖Δ͔ IUUQTpSFCBTFHPPHMFDOEPDTNMNBOBHFIPTUFENPEFMT IMFO

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

• mlkit sample ͷ example app ͷΧελϜϞσϧΛ࢖͍ͬͯΔՕॴͰݕূ • https://pub.dev/packages/mlkit • Android build͸ mlkit ͷreadmeҎ֎ͩͱɺԼهʹؾΛ͚ͭΕ͹OK • https://firebase.google.com/docs/ml-kit/android/use-custom-models?hl=ja • ಈ͔͘ʁ • managerͰϞσϧͷࢦఆͯ͠ɺinterpreterͰ࣮ߦ͢Δ • FirebaseModelInterpreter interpreter = FirebaseModelInterpreter.instance; • FirebaseModelManager manager = FirebaseModelManager.instance; • code • manager.registerRemoteModelSource Ͱ Firebaseʹొ࿥ͨ͠ custom model໊Λొ࿥͢Δ • interpreter.run( remoteModelName: _localModels[_currentModel], // ͜͜ʹ custom model ໊ͱొ࿥͢Δ • ػ಺ϞʔυͰಈ͘ʁ FlutterͰΧελϜϞσϧʹΑΔਪ࿦ΛϞόΠϧ୺຤্ͰͰ͖Δ͔

Slide 21

Slide 21 text

γϟϜೣ?

Slide 22

Slide 22 text

• mobilenet_v1_1.0_224_quantized ΧελϜϞσϧΛdeployͯ݁͠Ռ͕มΘΔ͔ʁ • ύϥϝʔλͷҧ͍ • mobilenet_v1_0.25_224_quantized ʢ< 1MBʣΑΓ ༰ྔ͕େ͖͍ʢ໿4MBʣ͕ਫ਼౓͕ߴ͍ ϗετͨ͠࠷৽ͷΧελϜϞσϧ͕Flutterʹ൓ө͞ΕΔ͔

Slide 23

Slide 23 text

໺΢αΪ !

Slide 24

Slide 24 text

• FirebaseͰϗετͰ͖ΔΧελϜϞσϧͷ༰ྔͷ্ݶ͸40MBͰ͢ɻ • ͨͱ͑͹ɺࣗવݴޠॲཧʢAlbert, Mobilebertʣ͸࢖͑ͳ͍Ͱ͢ʢ࢒೦ʣ • ผ్deployํ๏Λߟ͑Δ or Appͱڞʹ഑৴͢Δํ๏͕ඞཁͰ͢ɻ ͪΐͬͱ଍Γͳ͍ॴ

Slide 25

Slide 25 text

• FultterͰFirebase ML Kit ΧελϜϞσϧΛ࣮ࡍʹ͔ͭ͑Δ͔ →ɹ࢖͑Δʂ • ࢒λεΫ • TensorFlowͰฅͷը૾Λֶशͤ͞Δ && TensorFlow liteʹॻ͖ग़͢ ·ͱΊ