MediaPipe と ML Kit ってどうちがうの？ / What is the difference between MediaPipe and ML Kit?

Slide 1

Slide 1 text

.FEJB1JQFͱ.-,JUͬͯͲ͏ ͕ͪ͏ͷʁ :VLJ"O[BJ !ZBO[N %FW'FTU5PLZP 1 2"TMJEP

Slide 2

Slide 2 text

:VLJ"O[BJ w (PPHMF%FWFMPQFS&YQFSUGPS"OESPJE w 9 UXJUUFS !ZBO[N w CMPHZBO[NCMPHTQPUDPN w V1IZDB*OD 2

Slide 3

Slide 3 text

"HFOEB w .FEJB1JQFͱ͸ w .-,JUͱ͸ w .FEJB1JQF4PMVUJPOTͱ.-,JUͷҧ͍ 3

Slide 4

Slide 4 text

.FEJB1JQF w 0QFOTPVSDFNBDIJOFMFBSOJOHGSBNFXPSL w POEFWJDF w DSPTTQMBUGPSN "OESPJE J04 8FC 1ZUIPO w .FEJB1JQF'SBNFXPSL w ޮ཰తͳPOEFWJDF.-QJQFMJOFΛߏங͢ΔͨΊʹ࢖༻͞ΕΔMPXMFWFMίϯϙʔ ωϯτ w .FEJB1JQF4PMVUJPOT w ΞϓϦέʔγϣϯʹػցֶशͷػೳΛ૊ΈࠐΉͨΊͷϥΠϒϥϦͱπʔϧ܈ 4 https://github.com/google-ai-edge/mediapipe

Slide 5

Slide 5 text

Slide 6

Slide 6 text

.-,JU w POEFWJDF.BDIJOF-FBSOJOH ػೳΛఏڙ͢Δ"OESPJE J04޲͚ͷ4%, w ఏڙ͢Δػೳ w 7JTJPO"1* w /BUVSBM-BOHVBHF"1* w IUUQTEFWFMPQFSTHPPHMFDPNNMLJUHVJEFT 6

Slide 7

Slide 7 text

.FEJB1JQF4PMVUJPOTͱ.-,JUͷҧ͍ 7

Slide 8

Slide 8 text

ରԠϓϥοτϑΥʔϜ 8 MediaPipe Solutions ML Kit w "OESPJE w J04 w 8FC w 1ZUIPO w "OESPJE w J04

Slide 9

Slide 9 text

ػೳ 9 Media Pipe Solutions ML Kit

Slide 10

Slide 10 text

ػೳ 10 MediaPipe Solutions ML Kit

Slide 11

Slide 11 text

Ϟσϧબ୒ͷ༗ແ 11 MediaPipe Solutions ML Kit w ར༻͢ΔϞσϧΛࣗ෼ͰબͿ w ਪ঑Ϟσϧ͕༻ҙ͞Ε͍ͯΔ w CVJMEJOͷϞσϧ͕࢖ΘΕΔ w Ұ෦ͷػೳͰ͸ΧελϜϞσϧͷར ༻͕Մೳ

Slide 12

Slide 12 text

Ϟσϧͷ഑෍ 12 MediaPipe Solutions ML Kit w ϞσϧΛΞϓϦ಺ʹ഑ஔ w CVJMEJOϞσϧ w ϥΠϒϥϦͱҰॹʹϞσϧΛΞϓϦʹCVOEMF w (PPHMFQMBZTFSWJDFTͰμ΢ϯϩʔυ w ΧελϜϞσϧʢରԠ͍ͯ͠ΔػೳͷΈʣ w ϞσϧΛΞϓϦ಺ʹ഑ஔ w 'JSFCBTF.-ʹϞσϧΛϗετ w (PPHMFQMBZTFSWJDFTͰμ΢ϯϩʔυ w fi SFCBTFNMNPEFMEPXOMPBEFSͰμ΢ϯϩʔυ

Slide 13

Slide 13 text

13 https://ai.google.dev/edge/mediapipe/solutions/vision/image_classi fi er#models MediaPipe Solutions

Slide 14

Slide 14 text

14 val baseOptions = BaseOptions.builder() .setModelAssetPath("efficientnet_lite0.tflite") .build() val options = ImageClassifier.ImageClassifierOptions .builder() .setBaseOptions(baseOptions) … .build() imageClassifier = ImageClassifier.createFromOptions(context, options) MediaPipe Solutions αϯϓϧίʔυ https://github.com/google-ai-edge/mediapipe-samples AGP, Gradle ͕ݹ͍ͷͰ஫ҙ implementation("com.google.mediapipe:tasks-vision:0.20230731")

Slide 15

Slide 15 text

15 ML Kit val options = ImageLabelerOptions.Builder() .setConfidenceThreshold(0.6f) .build() labeler = ImageLabeling.getClient(options) // bundled : About 5.7 MB size increase. implementation("com.google.mlkit:image-labeling:17.0.9") or // unbundled : About 200 KB size increase. // Model is dynamically downloaded via Google Play Services. implementation("com.google.android.gms:play-services-mlkit-image-labeling:16.0.8")

Slide 16

Slide 16 text

Ϟσϧͷҧ͍ 16 MediaPipe Solutions Image classi fi cation ML Kit Image Labeling w & ffi DJFOU/FU-JUF w *NBHF/FUͰֶशͨ͠छྨͷ Ϋϥε w IUUQTTUPSBHFHPPHMFBQJTDPN NFEJBQJQFUBTLTJNBHF@DMBTTJ fi FS MBCFMTUYU w छྨͷϥϕϧ w IUUQTEFWFMPQFSTHPPHMFDPN NMLJUWJTJPOJNBHFMBCFMJOH MBCFMNBQ

Slide 17

Slide 17 text

17 cup : 0.41 Cup : 0.98 Tableware : 0.95 Porcelain : 0.88

Slide 18

Slide 18 text

ઃఆͰ͖Δ߲໨ͷҧ͍ 18 MediaPipe Solutions Image classi fi cation ML Kit Image Labeling ImageClassifier.ImageClassifierOptions .builder() .setScoreThreshold(0.3f) .setMaxResults(3) .setRunningMode(RunningMode.LIVE_STREAM) .setDisplayNamesLocale("ja") .setCategoryAllowlist(…) or .setCategoryDenylist(…) ImageLabelerOptions.Builder() .setConfidenceThreshold(0.3f)

Slide 19

Slide 19 text

ΧελϜϞσϧ 19

Slide 20

Slide 20 text

20 https://www.kaggle.com/models/google/aiy/tfLite/vision-classi fi er-birds-v1/3

Slide 21

Slide 21 text

Slide 22

Slide 22 text

22 MediaPipe Solutions val baseOptions = BaseOptions.builder() .setModelAssetPath("vision_classifier_birds_v3.tflite") .build() val options = ImageClassifier.ImageClassifierOptions .builder() .setBaseOptions(baseOptions) … .build() imageClassifier = ImageClassifier.createFromOptions(context, options)

Slide 23

Slide 23 text

23 ML Kit https://developers.google.com/ml-kit/custom-models#model-compatibility

Slide 24

Slide 24 text

24 ML Kit https://developers.google.com/ml-kit/custom-models#model-compatibility

Slide 25

Slide 25 text

25 ML Kit (Bundled) val localModel = LocalModel.Builder() .setAssetFilePath("vision_classifier_birds_v3.tflite") // or .setAbsoluteFilePath(absolute file path to model file) // or .setUri(URI to model file) .build() val options = CustomImageLabelerOptions.Builder(localModel) .setConfidenceThreshold(0.3f) .setMaxResultCount(5) .build() labeler = ImageLabeling.getClient(options) implementation("com.google.mlkit:image-labeling-custom:17.0.3")

Slide 26

Slide 26 text

26 ML Kit (Bundled) val localModel = LocalModel.Builder() .setAssetFilePath("vision_classifier_birds_v3.tflite") // or .setAbsoluteFilePath(absolute file path to model file) // or .setUri(URI to model file) .build() val options = CustomImageLabelerOptions.Builder(localModel) .setConfidenceThreshold(0.3f) .setMaxResultCount(5) .build() labeler = ImageLabeling.getClient(options) 26 implementation("com.google.mlkit:image-labeling-custom:17.0.3")

Slide 27

Slide 27 text

27 build-in model vision_classi fi er_birds_v3.t fl ite

Slide 28

Slide 28 text

w 'JSFCBTF.-ʹϞσϧΛϗετ͢Δʢ'JSFCBTF͕ඞཁʣ w ΞϓϦαΠζΛ཈͑ΒΕΔ w ΞϓϦΛߋ৽ͤͣʹϞσϧΛߋ৽Ͱ͖Δ w Ϟσϧͷμ΢ϯϩʔυ͕ऴΘΔ·Ͱਪ࿦Ͱ͖ͳ͍ w (PPHMF1MBZ4FSWJDFTʹμ΢ϯϩʔυΛ͓೚ͤPS fi SFCBTFNM NPEFMEPXOMPBEFSͰμ΢ϯϩʔυ 28 ML Kit (Unbundled)

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

32 ML Kit (Unbundled : Google Play Services Ͱμ΢ϯϩʔυ) val remoteModel = CustomRemoteModel .Builder(FirebaseModelSource.Builder("vision_classifier_food").build()) .build() val downloadConditions = DownloadConditions.Builder().requireWifi().build() RemoteModelManager.getInstance() .download(remoteModel, downloadConditions) .addOnSuccessListener { val options = CustomImageLabelerOptions.Builder(remoteModel) .setConfidenceThreshold(0.1f) .setMaxResultCount(5) .build() labeler = ImageLabeling.getClient(options) } implementation("com.google.android.gms:play-services-mlkit-image-labeling-custom:16.0.0-beta5") implementation("com.google.mlkit:linkfirebase:17.0.0")

Slide 33

Slide 33 text

33 ML Kit (Unbundled : fi rebase-ml-modeldownloader Ͱμ΢ϯϩʔυ) val conditions = CustomModelDownloadConditions.Builder().requireWifi().build() FirebaseModelDownloader.getInstance() .getModel("vision_classifier_food", DownloadType.LOCAL_MODEL, conditions) .addOnSuccessListener { model -> model?.file?.let { val localModel = LocalModel.Builder() .setAbsoluteFilePath(it.absolutePath) .build() val customImageLabelerOptions = CustomImageLabelerOptions.Builder(localModel) .setConfidenceThreshold(0.1f) .setMaxResultCount(5) .build() labeler = ImageLabeling.getClient(customImageLabelerOptions) } } implementation("com.google.firebase:firebase-ml-modeldownloader:25.0.1")

Slide 34

Slide 34 text

Slide 35

Slide 35 text

.FEJB1JQF.PEFM.BLFS 35

Slide 36

Slide 36 text

.FEJB1JQF.PEFM.BLFS w ϕʔεͷϞσϧΛసҠֶश w Ұ෦ͷػೳ͕ରԠʢ0CKFDU %FUFDUJPO΍*NBHF $MBTTJ fi DBUJPOͳͲʣ w $PMBC͕༻ҙ͞Ε͍ͯΔ 36 https://ai.google.dev/edge/mediapipe/solutions/model_maker

Slide 37

Slide 37 text

37 https://ai.google.dev/edge/mediapipe/solutions/customization/image_classi fi er

Slide 38

Slide 38 text

38 w *NBHF$MBTTJ fi DBUJPOͷΧελϜϞσϧ w छྨຕͩͱ͜ͷఔ౓ͷਫ਼౓ɺ࠷௿ ຕ͸ཉ͍͠ w Yʢ΋ͬͱখ͍͍ͯ͘͞ʣͷ ຕYຕͰֶश࣌ؒ͸໿෼

Slide 39

Slide 39 text

·ͱΊ w 8FC΍1ZUIPOͰ࢖͍͍ͨˠ.FEJB1JQF4PMVUJPOT w ࢖͍͍ͨػೳ͕Ұํʹ͔͠ͳ͍ˠػೳ͕͋Δ΄͏ w Ϟσϧͷѻ͍Λ͓·͔͍ͤͨ͠ˠ.-,JU w 0CKFDU%FUFDUJPO *NBHF-BCFMJOHҎ֎ͰΧελϜϞσϧΛ࢖͍͍ͨˠ .FEJB1JQF4PMVUJPOT w 0CKFDU%FUFDUJPO *NBHF-BCFMJOHͷΧελϜϞσϧΛΞϓϦʹόϯυϧͨ͘͠ͳ ͍ˠ.-,JU w ͍ΖΜͳػցֶशػೳ͕؆୯ʹΞϓϦʹ૊ΈࠐΊΔΑ͏ʹͳͬͯΔ͔Βࢼͯ͠Έͯʂ 39

Slide 40

Slide 40 text

5IBOLZPV 40