Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ML Kit Introduction (for iOS)
Search
Elvis Lin
July 19, 2018
Programming
0
130
ML Kit Introduction (for iOS)
Introduce the basic concept of ML Kit and how to use it in iOS development
Elvis Lin
July 19, 2018
Tweet
Share
More Decks by Elvis Lin
See All by Elvis Lin
Protect Users' Privacy in iOS 14
elvismetaphor
0
40
Dubugging Tips and Tricks for iOS development
elvismetaphor
0
35
Strategies of Facebook LightSpeed project
elvismetaphor
0
50
Background Execution And WorkManager
elvismetaphor
2
470
作為一個跨平台的 Mobile App 開發者,從入門到放棄!?
elvismetaphor
2
410
Dependency Injection for testability of iOS app
elvismetaphor
1
1.3k
Briefly Introduction of Kotlin coroutines
elvismetaphor
1
250
MotionLayout Brief Introduction
elvismetaphor
1
310
Chapter 10. Pattern Matching with Regular Expressions
elvismetaphor
0
36
Other Decks in Programming
See All in Programming
Compiling Python to WebAssembly with py2wasm
syrusakbary
0
130
ここ1~2年くらいで 使えるようになった(主要ブラウザーの最新版 がすべて対応した ) ウェブの新機能について ランダムに喋る!
myzkyy
9
6.5k
品質とスピードを両立: TypeScriptの柔軟な型システムをバックエンドで活用する
kosui
4
870
15分間でふんわり理解するDocker @ Matsuriba MAX
ukwhatn
PRO
1
340
object-oriented-conference-2024
fuwasegu
6
1.8k
どうしてこうなった命名集 ~🔥編~ / OOC 2024 LT
pictiny
4
2.8k
WinUI 3デモ - "CommunityToolkit.Mvvm"NuGetパッケージ編
andrewkeepcoding
0
130
とにかくHTTP3をライトニングに話す / Anyway, I'll talk to Lightning about HTTP3.
seike460
PRO
0
120
フロントエンドパフォーマンス 入門
shouta2
7
1.5k
オブジェクト指向コードレビューの新しいアプローチ
akkie76
3
1.5k
脱・初心者!脱・マネコン!AWS CDKを使ってみませんか!?
har1101
0
110
Open Source Swift Workshop - Foundation and first party libraries
ikesyo
0
180
Featured
See All Featured
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
14
1.3k
Agile that works and the tools we love
rasmusluckow
323
20k
The MySQL Ecosystem @ GitHub 2015
samlambert
242
12k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
18
6.8k
Mobile First: as difficult as doing things right
swwweet
215
8.5k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
113
18k
jQuery: Nuts, Bolts and Bling
dougneiner
57
7.1k
Art, The Web, and Tiny UX
lynnandtonic
288
19k
Why You Should Never Use an ORM
jnunemaker
PRO
50
8.6k
WebSockets: Embracing the real-time Web
robhawkes
59
6.9k
Music & Morning Musume
bryan
39
5.5k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Transcript
ML Kit 使⽤用簡介 (iOS) Elvis Lin @Cocoahead Taipei 2018-07-19
關於我 • Elvis Lin • iOS 與 Android 永遠的初學者 •
Twitter: @elvismetaphor • Blog: https://blog.elvismetaphor.me
⼤大綱 • 什什麼是(我理理解的)機器學習 • 移動裝置上實作機器學習應⽤用的限制 • TensorFlow Lite 與 ML
Kit • 範例例
機器學習的應⽤用
機器學習 • 從資料中歸納出有⽤用的規則 • 訓練模型 • 使⽤用模型 • Mobile Application
Engineer 參參 與開發主要是在「使⽤用模型」 這個範圍
Data Result (Trained) Model
移動裝置上 實作機器學習應⽤用的限制 • 記憶體有限與儲存空間有限 • 計算能⼒力力不如⼤大型伺服器 • 電池容量量有限
移動裝置上 實作機器學習應⽤用的改良⽅方向 • 記憶體有限與儲存空間有限 —> 減少模型(Model)的體積 • 計算能⼒力力不如⼤大型伺服器 —> 降低演算法的複雜度
• 電池容量量有限 —> 降低演算法的複雜度
Google 推出的解決⽅方案 • TensorFlow Lite • ML Kit
Tensorflow Lite https://youtu.be/ByJnpbDd-zc
https://www.tensorflow.org/mobile/tflite/
轉換 Tensorflow 檔案的⼯工具 • Tensorflow converter • 轉成 Tensorflow Lite
格式 • Tensorflow-CoreML converter • 轉成 CoreML 格式 • https://github.com/tf-coreml/tf-coreml
ML Kit https://youtu.be/Z-dqGRSsaBs
Neural Networks API Metal
ML Kit • Cloud Vision API / Mobile Vision API
• Tensorflow Lite • 整合 Firebase,託管「客製化的模型」
ML Kit Base APIs • Image labeling • Text recognition
(OCR) • Face detection • Barcode scanning • Landmark detection • others……
託管客製化的模型 ⽬目前只⽀支援 Tensorflow Lite 格式
使⽤用 ML Kit
建立⼀一個 Firebase 專案
建立⼀一個 iOS app 然後下載設定檔 設定好 Bundle ID 下載 GoogleService-info.plist
新增 plist 檔案到專案 • 將 GoogleService-Info.plist 放到 <root>/<application_folder>/ 下
安裝 Firebase 函式庫 • 修改 Podfile,新增以下的內容 • cd <root> pod
install • 打改 <project_name>.xcworkspace pod 'Firebase/Core' pod 'Firebase/MLVision' pod 'Firebase/MLVisionTextModel' pod 'Firebase/MLVisionFaceModel' pod 'Firebase/MLVisionBarcodeModel' pod 'Firebase/MLVision' pod 'Firebase/MLVisionLabelModel'
掃描 Barcode (Local) let barcodeDetector: VisionBarcodeDetector = Vision.vision().barcodeDetector(options: options)
let visionImage = VisionImage(image: pickedImage) barcodeDetector.detect(in: visionImage) { (barcodes, error) in guard error == nil, let barcodes = barcodes, !barcodes.isEmpty else { self.dismiss(animated: true, completion: nil) self.resultView.text = "No Barcode Detected" return } for barcode in barcodes { // handle the detected barcode } }
第1步:初始化 Detector let barcodeDetector: VisionBarcodeDetector = Vision.vision().barcodeDetector(options: options) let
visionImage = VisionImage(image: pickedImage)
第2步:取得結果 barcodeDetector.detect(in: visionImage) { (barcodes, error) in guard error ==
nil, let barcodes = barcodes, !barcodes.isEmpty else { self.dismiss(animated: true, completion: nil) self.resultView.text = "No Barcode Detected" return } for barcode in barcodes { // handle the detected barcode } }
⽀支援的 Barcode 格式 • Code 128 (FORMAT_CODE_128) • Code 39
(FORMAT_CODE_39) • Code 93 (FORMAT_CODE_93) • Codabar (FORMAT_CODABAR) • EAN-13 (FORMAT_EAN_13) • EAN-8 (FORMAT_EAN_8) • ITF (FORMAT_ITF) • UPC-A (FORMAT_UPC_A) • UPC-E (FORMAT_UPC_E) •QR Code (FORMAT_QR_CODE) • PDF417 (FORMAT_PDF417) • Aztec (FORMAT_AZTEC) • Data Matrix (FORMAT_DATA_MATRIX)
辨識⽂文字 (Local) lazy var textDetector: VisionTextDetector = Vision.vision().textDetector() func
runTextRecognition(with image: UIImage) { let visionImage = VisionImage(image: image) textDetector.detect(in: visionImage) { (features, error) in if let error = error { print("Received error: \(error)") } self.processResult(from: features, error: error) } }
辨識⽂文字 (Cloud) Lazy var cloudTextDetector: VisionCloudTextDetector = Vision.vision().cloudTextDetector() func
runCloudTextRecognition(with image: UIImage) { let visionImage = VisionImage(image: image) cloudTextDetector.detect(in: visionImage) { (features, error) in if let error = error { print("Received error: \(error)") } self.processCloudResult(from: features, error: error) } }
補充資料 • ML Kit 簡介 (for Android) https://blog.elvismetaphor.me/ml-kit-fundamentals-for- android-6444e2db0fdb •
ML Kit 簡介 (for iOS) https://blog.elvismetaphor.me/ml-kit-fundamentals-for- ios-cb705044e69b
參參考資料 • https://youtu.be/Z-dqGRSsaBs • https://codelabs.developers.google.com/codelabs/mlkit-ios/ • https://github.com/firebase/quickstart-ios/tree/master/ mlvision • https://www.appcoda.com.tw/ml-kit/
None