CoreML3のオンデバイストレーニングでつくる母音推定

2bfb16e53b154afc9f141f831b4aaaf2?s=47 naru-jpn
January 25, 2020

 CoreML3のオンデバイストレーニングでつくる母音推定

2bfb16e53b154afc9f141f831b4aaaf2?s=128

naru-jpn

January 25, 2020
Tweet

Transcript

  1. CoreML3ͷΦϯσόΠετϨʔχϯάͰͭ͘Δ฼Իਪఆ Inference of vowel by on-device training of CoreML3 Naruki

    Chigira @ Mirrativ, inc. _naru_jpn
  2. σϞͰ࢖͏ΞϓϦͷίʔυ https://github.com/naru-jpn/InferenceVowel

  3. ฼Իਪఆ /a/, /i/, /u/, /e/, /o/ ͷਪఆ

  4. ΦϯσόΠετϨʔχϯά https://developer.apple.com/jp/machine-learning/core-ml/ …ϢʔβʔσʔλΛ࢖༻ͨ͠ϞσϧͷΞοϓσʔτΛΦϯσό ΠεͰ࣮ߦͰ͖ΔΑ͏ʹͳͬͨͨΊɺϓϥΠόγʔΛ৵֐͢Δ ͜ͱͳ͘ɺϢʔβʔͷߦಈʹ߹ΘͤͯϞσϧΛΞοϓσʔτ͠ ͍ͯ͘͜ͱ͕Ͱ͖·͢ɻ

  5. None
  6. ํ਑

  7. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

  8. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

    let captureSession = AVCaptureSession() … captureSession.startSession()
  9. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

    ϑʔϦΤม׵ Fourier Transformation ϝϧϑΟϧλόϯΫ Mel Filter Bank ཭ࢄίαΠϯม׵ Discrete Cosine Transformation
  10. ϑʔϦΤม׵ Fourier Transformation (ϩά)ϝϧϑΟϧλόϯΫ (Log) Mel Filter Bank ཭ࢄίαΠϯม׵ Discrete

    Cosine Transformation Import Accelerate // ԻΛप೾਺͝ͱʹ෼ղ vDSP_fft_zrip(fftsetup, … // ਓ͕ฉ͖෼͚΍͍͢௿ԻΛΑΓৄࡉʹ vDSP_mmul(melFilerBank, … // ΑΓগͳ͍σʔλ਺Ͱදݱ vDSP_DCT_Execute(dctsetup, …
  11. ϑʔϦΤม׵ Fourier Transformation (ϩά)ϝϧϑΟϧλόϯΫ (Log) Mel Filter Bank ཭ࢄίαΠϯม׵ Discrete

    Cosine Transformation Import Accelerate // ԻΛप೾਺͝ͱʹ෼ղ vDSP_fft_zrip(fftsetup, … // ਓ͕ฉ͖෼͚΍͍͢௿ԻΛΑΓৄࡉʹ vDSP_mmul(melFilerBank, … // ΑΓগͳ͍σʔλ਺Ͱදݱ vDSP_DCT_Execute(dctsetup, … 1024ݸͷσʔλ 16ݸͷσʔλ ~25μs
  12. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

    https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb apple/coremltools updatable_nearest_neighbor_classifier.ipynb Jupyter notebook ্Ͱkۙ๣๏Λར༻ͯ͠෼ ྨΛ͢ΔϞσϧΛ࡞੒͢Δαϯϓϧ
  13. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

    https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ֶशʹ࢖͏σʔλͷ४උ let inputValue = snapshot.featureValue let outputValue = MLFeatureValue( string: snapshot.vowel.identifier ) let dataPointFeatures: [String: MLFeatureValue] = [ "input": inputValue, "label": outputValue ] let provider = try! MLDictionaryFeatureProvider( dictionary: dataPointFeatures )
  14. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

    https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ֶशͷ࣮ߦ let batchProvider = MLArrayBatchProvider( array: featureProviders ) let updateTask = try! MLUpdateTask( forModelAt: UpdatableKNN.urlOfModelInThisBundle, trainingData: batchProvider, configuration: nil, completionHandler: completion ) updateTask.resume()
  15. લॲཧ Preprocessing ֶश / ਪ࿦ Training / Inference Ի੠σʔλ Audio

    https://github.com/apple/coremltools/blob/master/examples/updatable_models/updatable_nearest_neighbor_classifier.ipynb // ਪ࿦ͷ࣮ߦ let inputs = snapshots.compactMap({ $0.featureValue.multiArrayValue }).map({ UpdatableKNNInput(input: $0) }) let model = try! UpdatableKNN(contentsOf: updatedModelUrl) // ฼Ի͝ͱͷ֬཰͕ग़ྗͱͯ͠ฦͬͯ͘Δ let outputs = try model.predictions(inputs: inputs) 100~200μs
  16. σϞ