Share
Embed
15分でわかる、Flutter ML 世界 @FlutterGakkai
第3回 FlutterGakkai
15分でわかる、Flutter ML 世界
Jai-Chang Park (@Dreamwalker)
JaiChangPark
January 28, 2023
Transcript
48 slides
-
Park JaiChang (@Dreamwalker) • ᖃଐ: Dreamus Company • Flutter Seoul:
Organizer • GDG Golang Korea: Organizer • @jaichangpark • Github: JAICHANGPARK • Flutter / Android / Go / Rust ᥦᖖ෨ ᩲ3ỹ FlutterGakkai 2 -
ᶝ ! • Κ͔͡Ν᭳ዧ • FlutterML (Machine Learning)᧱ᠭ͞൱ၙͼĦ • FlutterMLϧόϰΤͻΚ͘ͽᑿᠭͩΐͫ͝Ħ
• ͻͥ͝ΛᘿΓΝ͝Ρ͝ΜΐͭΥ🥹 • ⬇ • Flutter x ML(Machine Learning)αΩώ (ͽͼΞ͖͖ͫͿ..) ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 4 -
Agenda 1. Flutter 2. ML 3. Flutter x ML ᩲ3ỹ
FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 5 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.1. FlutterͽͶ͖ Flutter is an
open source framework by Google for building beautiful, natively compiled, multi-platform applications from a single codebase. https://flutter.dev/ 7 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.1. FlutterͽͶ͖ ͼͮDart᜔Ͳ͵ͱĦ🎯 • Flutter
TeamΔൃᰮͽၞ͡ϜϲεϮϤϸε᜔ͺϮϸτΩϥΤວᅝͩ͟ͱg • ͯ͘ͳDart᜔͞Flutter φ⑲ϥ᠓ཏͺͽରͫΝξθΧ͞ີ͝͵ͱg • React NativeJS(JaveScript)Τᑿ͘gJaveScriptͯᥦମΤมͩͽ͖͡g • (Dart == Google && Flutter == Google)͞ᬤ͵͖Νᅪg • Dart᜔ᡯʑAoT(Ahead-of-Time) Compilerͼ͝͵ͱg • Dartφ⑲ϥ͞Flutterφ⑲ϥͱΓͽṐᅪͩAoTω⑲ϰφά⑲ϸ(Toolchain)ΤൃͫΝg • Dart VM(Virtual Machine)ᱠ᧱ặ 9 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.1. FlutterͽͶ͖ https://docs. fl utter.dev/resources/architectural-overview#architectural-layers
Architectural layers 1. Framework - Dart Lang 2. Engine - C/C++ - Rendering : Skia, Impeller 3. Embedder - Platform-speci fi c 10 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.1. FlutterͽͶ͖ Architectural layers 1.
Framework - Dart Lang 2. Engine - C/C++ - Rendering : Skia, Impeller 3. Embedder - Platform-speci fi c 11 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.1. FlutterͽͶ͖ • ϜϮψύϚή⑲ϥၙΤFlutter FrameworkᑿᠭͫΝͽͻͫ͘ΞΚ
͖ͫ͝Ħ • ᆚ͚fAndroid, iOSPlatformၙΤᑿ͖ͱ͖Ĉ • Platform Channel • FFI (Foreign Function Interface) 👇 12 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.2. Platform Channel 1. Hardwareၙ
• Bluetooth, BLE • Camera 2. ݖݶ • Permission https:// fl utter.dev/multi-platform/mobile 3. πϸκ⑲ᨼᏄďᲩĐ • Acc, Gyro, Mag 4. TCP/IP Socket 5. ͼͻg 13 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.2. Platform Channel • Kotlin
or Java on Android • Swift or Objective-C on iOS • C++ on Windows • Objective-C on macOS • C on Linux https://docs. fl utter.dev/development/platform-integration/platform- channels?tab=type-mappings-c-plus-plus-tab 14 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.2. Platform Channel Method Channel
• Asynchronous Method ! • ϜϮψύϚή⑲ϥͺᴢᙩͫΝͱΓͽ ᑿᠭͧΞΝφϨϒϰ Method Channel = Future Event Channel • Asynchronous! • ϜϮψύϚή⑲ϥEvent Streamďξύϯ⑲ϥĐΤᗀᙩͫ ΝͱΓͽᑿᠭͧΞΝφϨϒϰ Event Channel = Stream 15 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.2. Platform Channel Κ͔͡Ν᭳ዧ ͖Ζf
ီ͖ͩͩfᏝࡶͫͿgᶶֶͬΈᶶ᠓͔ΝĦ ᬰ͵ͫ͜͟͡͝Ħ 😡🤯 16 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1.2. Platform Channel ML ϧόϰᏣ᧣ͽ
Native Platform ࣮ḝͧΞΐͫg ͶΐΜfᬰ͵͜͡ͺͽርͳΐͫg 17 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 1. Flutter Recap (᠓ᛱ) 1.
FlutterGoogleFlutter Team͝ΛൃͩͱOpen Source UI Toolkit ͔Νg • Dart, Multi-Platform, Declarative UI 2. Architecture • Framework, Engine, Embedder 3. Platform Channel • Method Channel, Event Channel 18 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 2.1. Machine LearningͽͶ͖ AI Arti
fi cial Intelligence Machine Learning Deep Learning Χϰιϯοϥͺ ᱠ᧱ặઓ͖ 20 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 2.1. Machine LearningͽͶ͖ Data Input
Black Box Model ൃ Model ֶश Model Test Retraining Model Deploy User Feedback ᔭ 21 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 2.2. MLϮΩϛϮϯ • TensorFlow •
TensorFlow.js • TensorFlow Lite (TinyML) • Keras • Pytorch • Other ML API Service • Ex) Open AI 2.0 22 -
3. Flutter x ML 3.1. ᱠ᙮Flutter MLύϱϸώ 3.2. MLϧόϰΤ᧱ᠭͫΝ 3.3.
Holobooth-Flutter Forward Edition 23 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ Machine Learning
Model 1. ͫͽΚ͡ᥭΛΞͱϧόϰΤᑿ͘ 2. ᥦᐠᥭΝ 3. APIΤớᠭͫΝ 24 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ ͥͥĘᎈᬍ᠓ͼϢΩϸύ Κ͡ᥭΛΞͱMLϧόϰΤϫ⑲λ⑲࣮᩺͞ͽ
ᑿᠭ͟ΝΚ͘ͽͫΝͥͺ͔Νg 25 ConsoleΑͱใໝ ࣮ࡍͷΞϓϦԽ -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 26 https://www.tensor
fl ow.org/lite/android MobileΖEmbeddedỖ๕ͽᑿ͚Ν TensorFlow Lite MicrocontrollersΔᑿ͚ t fl ite model -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 28 🤔ͥΞΛͫΤᬰΝᶶ᠓͔͞Μΐͫ͝Ħ
ͫͽ͘ΐ͡ᥭΛΞͱϖψζ⑲νΤᑿᠭͫΝĦ https://pub.dev/ -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 29 Mar
14, 2019 JUN 3, 2020 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 30 fi
rebase_ml_vision 0.12.0+3 https://pub.dev/packages/ fi rebase_ml_vision OCT 21, 2021 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 31 This
package is now discontinued since these APIs are no longer available in the latest Firebase SDKs. As an alternative you can switch to Google's standalone ML Kit library via google_ml_kit for on-device vision APIs. For calling the Cloud Vision API from your app the recommended approach is using Firebase Authentication and Functions, which gives you a managed, serverless gateway to Google Cloud Vision APIs. For an example Functions project see the vision-annotate-images sample project. -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 32 google_ml_kit
0.13.0 Access᥀: 2023-01-27 https://pub.dev/packages/google_ml_kit -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 33 Google
ML Kit - google_ml_kit 1. Vision APIs • ը૾ʹϥϕϧΛ͚ͯը૾ʹϥϕϧΛ͚ɺόʔ ίʔυɺςΩετɺإɺΦϒδΣΫτΛݕग़͢Δ APIg • Face mesh Τ᩹͡ ͫᑿᠭ൱ၙ 2. Natural Language APIs • 58 ᜡ᜔ණ༁ΤᙣᎭͩ༁ͩfጿᙩΤᩮᚳͫΝᥦ ᜡ᜔ॲሑ APIg • ͫᑿᠭ൱ၙ -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 34 🤔ΰξτϥϧόϰ
ͻͫ͘Ξ͖͖ͫ͝Ħ ࿆ML KitϘϰώͧΞͱ ϧόϰΤᑿ͵͖ΝΚͫ͘͞… -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 35 https://pub.dev/packages/
fi rebase_ml_model_downloader fi rebase_ml_model_downloader FirebaseͽΧψϜϲ⑲ώͧΞͱΰξτϥϧόϰΤυΫϸϲ⑲ώ͟ΐͫg Access᥀ 2023-01-27 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 36 https://pub.dev/packages/t
fl ite_ fl utter https://pub.dev/packages/t fl ite -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 37 🤔
͔t fl te͵ͻͥ͝υΫϸϲ⑲ώ͟ΐͫĦ 1. ᥭᕧͩͱֶᘯϧόϰΤt fl iteϚΦΩϰͽมỐ • https://www.tensor fl ow.org/lite/guide/model_maker?hl=ja • Tensor fl ow Lite Model Maker 2. Tensor fl ow Hub • https://www.tensor fl ow.org/hub?hl=ko • TensorFlow Hub fύϱ⑲ϐϸεࡁΑຂֶᘯϧόϰϯϢνύϯͫg -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.1. ᱠ᙮Flutter MLύϱϸώ 39 🤔
Chat Bot(φϨψύϡψύ)Κ͘ͼμξϋϥĦĦ 1. Dialog fl ow • dialog fl ow_ fl utter 1.0.0 2. GPT3 from OpenAI • openai_gpt3_api • gpt_3_dart • fl utter_chatgpt_api • chat_gpt_sdk • chat_gpt_api • chat_gpt_ fl utter -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.2. MLϧόϰΤ᧱ᠭͫΝ 40 t fl
iteΤυΫϸϲ⑲ώͩᑿ͘ PackageΤᑿ͘ Assets Folder Firebase ML Custom Tensor fl ow Hub ᥦᐠᥭΝ Firebase_ml_mod el_downloader t fl ite & t fl ite_ fl utter t fl ite model google_ml_kit End End Start Webᑿ͘ TensorFlow.js ᥦᐠPackageᥭΝ End -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.2. MLϧόϰΤ᧱ᠭͫΝ 41 dependencies -
pubspec.yaml - PackageΤඦ୯ͽᱮ൰ ͟Νg ᆚ) PackageΤᑿ͘ -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.3. Holobooth-Flutter Forward Edition 42
https://photobooth. fl utter.dev/ Δͩf ͥΞ͚͖֮ΐͫ͝Ħ Google I/O 22 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.3. Holobooth-Flutter Forward Edition 43
https://holobooth. fl utter.dev/#/ -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.3. Holobooth-Flutter Forward Edition 44
1. Flutter 2. Firebase 3. TensorFlow.js detect the user’s face within the frame of the camera 4. MediaPipe estimates 468 3D face landmarks in real-time https://medium.com/ fl utter/how-its- made-holobooth-6473f3d018dd https://github.com/ fl utter/holobooth -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ 3.3. Holobooth-Flutter Forward Edition 45
https://github.com/ fl utter/holobooth Access᥀: 2023-01-27 -
ᩲ3ỹ FlutterGakkai 15ᐠΡ͝ΝfFlutter ML ᕴ ΐͺΓ 47 1. Flutter •
GoogleFlutter Team͝ΛൃͩͱOpen Source UI Toolkit͔Νg • Dart, Multi-Platform, Declarative UI 2. Flutter x ML • TensorFlow, PyTorch, ML API • KeyPoint 👉 Κ͡ᥭΛΞͱMLϧόϰΤϫ⑲λ⑲࣮᩺͞ͽᑿᠭ͟ΝΚ͘ͽͫΝͥͺ͔Νg • 1) ͫͽΚ͡ᥭΛΞͱϧόϰΤᑿ͘ 2) ᥦᐠᥭΝ 3) APIΤớᠭͫΝ • Flutter - MLઐ༻ύοέʔδ͕ଘࡏ, google_ml_kit (MLKit) 3. Flutter Forward Holobooth • Tensor fl ow.js, MediaPipe