Share
Embed
Transcript
16 slides
-
CoreML3ͷΦϯσόΠετϨʔχϯάͰͭ͘ΔԻਪఆ Inference of vowel by on-device training of CoreML3 Naruki
Chigira @ Mirrativ, inc. _naru_jpn -
લॲཧ Preprocessing ֶश / ਪ Training / Inference Իσʔλ Audio
let captureSession = AVCaptureSession() … captureSession.startSession() -
લॲཧ Preprocessing ֶश / ਪ Training / Inference Իσʔλ Audio
ϑʔϦΤม Fourier Transformation ϝϧϑΟϧλόϯΫ Mel Filter Bank ࢄίαΠϯม Discrete Cosine Transformation -
ϑʔϦΤม Fourier Transformation (ϩά)ϝϧϑΟϧλόϯΫ (Log) Mel Filter Bank ࢄίαΠϯม Discrete
Cosine Transformation Import Accelerate // ԻΛप͝ͱʹղ vDSP_fft_zrip(fftsetup, … // ਓ͕ฉ͖͚͍͢ԻΛΑΓৄࡉʹ vDSP_mmul(melFilerBank, … // ΑΓগͳ͍σʔλͰදݱ vDSP_DCT_Execute(dctsetup, … -
ϑʔϦΤม Fourier Transformation (ϩά)ϝϧϑΟϧλόϯΫ (Log) Mel Filter Bank ࢄίαΠϯม Discrete
Cosine Transformation Import Accelerate // ԻΛप͝ͱʹղ vDSP_fft_zrip(fftsetup, … // ਓ͕ฉ͖͚͍͢ԻΛΑΓৄࡉʹ vDSP_mmul(melFilerBank, … // ΑΓগͳ͍σʔλͰදݱ vDSP_DCT_Execute(dctsetup, … 1024ݸͷσʔλ 16ݸͷσʔλ ~25μs -
લॲཧ Preprocessing ֶश / ਪ Training / Inference Իσʔλ Audio
https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb apple/coremltools updatable_nearest_neighbor_classifier.ipynb Jupyter notebook ্Ͱkۙ๏Λར༻ͯ͠ ྨΛ͢ΔϞσϧΛ࡞͢Δαϯϓϧ -
લॲཧ Preprocessing ֶश / ਪ Training / Inference Իσʔλ Audio
https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ֶशʹ͏σʔλͷ४උ let inputValue = snapshot.featureValue let outputValue = MLFeatureValue( string: snapshot.vowel.identifier ) let dataPointFeatures: [String: MLFeatureValue] = [ "input": inputValue, "label": outputValue ] let provider = try! MLDictionaryFeatureProvider( dictionary: dataPointFeatures ) -
લॲཧ Preprocessing ֶश / ਪ Training / Inference Իσʔλ Audio
https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ֶशͷ࣮ߦ let batchProvider = MLArrayBatchProvider( array: featureProviders ) let updateTask = try! MLUpdateTask( forModelAt: UpdatableKNN.urlOfModelInThisBundle, trainingData: batchProvider, configuration: nil, completionHandler: completion ) updateTask.resume() -
લॲཧ Preprocessing ֶश / ਪ Training / Inference Իσʔλ Audio
https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ਪͷ࣮ߦ let inputs = snapshots.compactMap({ $0.featureValue.multiArrayValue }).map({ UpdatableKNNInput(input: $0) }) let model = try! UpdatableKNN(contentsOf: updatedModelUrl) // Ի͝ͱͷ͕֬ग़ྗͱͯ͠ฦͬͯ͘Δ let outputs = try model.predictions(inputs: inputs) 100~200μs