CoreML3ͷΦϯσόΠετϨʔχϯάͰͭ͘Δ฼Իਪఆ Inference of vowel by on-device training of CoreML3 Naruki Chigira @ Mirrativ, inc. _naru_jpn 

σϞͰ࢖͏ΞϓϦͷίʔυ https://github.com/naru-jpn/InferenceVowel 

฼Իਪఆ /a/, /i/, /u/, /e/, /o/ ͷਪఆ 

ΦϯσόΠετϨʔχϯά https://developer.apple.com/jp/machine-learning/core-ml/ …ϢʔβʔσʔλΛ࢖༻ͨ͠ϞσϧͷΞοϓσʔτΛΦϯσό ΠεͰ࣮ߦͰ͖ΔΑ͏ʹͳͬͨͨΊɺϓϥΠόγʔΛ৵֐͢Δ ͜ͱͳ͘ɺϢʔβʔͷߦಈʹ߹ΘͤͯϞσϧΛΞοϓσʔτ͠ ͍ͯ͘͜ͱ͕Ͱ͖·͢ɻ 

No content

ํ਑ 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio let captureSession = AVCaptureSession() … captureSession.startSession() 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio ϑʔϦΤม׵ Fourier Transformation ϝϧϑΟϧλόϯΫ Mel Filter Bank ཭ࢄίαΠϯม׵ Discrete Cosine Transformation 

ϑʔϦΤม׵ Fourier Transformation (ϩά)ϝϧϑΟϧλόϯΫ (Log) Mel Filter Bank ཭ࢄίαΠϯม׵ Discrete Cosine Transformation Import Accelerate // ԻΛप೾਺͝ͱʹ෼ղ vDSP_fft_zrip(fftsetup, … // ਓ͕ฉ͖෼͚΍͍͢௿ԻΛΑΓৄࡉʹ vDSP_mmul(melFilerBank, … // ΑΓগͳ͍σʔλ਺Ͱදݱ vDSP_DCT_Execute(dctsetup, … 

ϑʔϦΤม׵ Fourier Transformation (ϩά)ϝϧϑΟϧλόϯΫ (Log) Mel Filter Bank ཭ࢄίαΠϯม׵ Discrete Cosine Transformation Import Accelerate // ԻΛप೾਺͝ͱʹ෼ղ vDSP_fft_zrip(fftsetup, … // ਓ͕ฉ͖෼͚΍͍͢௿ԻΛΑΓৄࡉʹ vDSP_mmul(melFilerBank, … // ΑΓগͳ͍σʔλ਺Ͱදݱ vDSP_DCT_Execute(dctsetup, … 1024ݸͷσʔλ 16ݸͷσʔλ ~25μs 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb apple/coremltools updatable_nearest_neighbor_classifier.ipynb Jupyter notebook ্Ͱkۙ๣๏Λར༻ͯ͠෼ ྨΛ͢ΔϞσϧΛ࡞੒͢Δαϯϓϧ 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ֶशʹ࢖͏σʔλͷ४උ let inputValue = snapshot.featureValue let outputValue = MLFeatureValue( string: snapshot.vowel.identifier ) let dataPointFeatures: [String: MLFeatureValue] = [ "input": inputValue, "label": outputValue ] let provider = try! MLDictionaryFeatureProvider( dictionary: dataPointFeatures ) 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ֶशͷ࣮ߦ let batchProvider = MLArrayBatchProvider( array: featureProviders ) let updateTask = try! MLUpdateTask( forModelAt: UpdatableKNN.urlOfModelInThisBundle, trainingData: batchProvider, configuration: nil, completionHandler: completion ) updateTask.resume() 

લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ਪ࿦ͷ࣮ߦ let inputs = snapshots.compactMap({ $0.featureValue.multiArrayValue }).map({ UpdatableKNNInput(input: $0) }) let model = try! UpdatableKNN(contentsOf: updatedModelUrl) // ฼Ի͝ͱͷ֬཰͕ग़ྗͱͯ͠ฦͬͯ͘Δ let outputs = try model.predictions(inputs: inputs) 100~200μs 

σϞ