Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CoreMLでアイドル顔認識アプリを作ろう

kenmaz
September 17, 2017

 CoreMLでアイドル顔認識アプリを作ろう

iOSDC Japan 2017 での発表資料です。
https://iosdc.jp/2017/

kenmaz

September 17, 2017
Tweet

More Decks by kenmaz

Other Decks in Programming

Transcript

  1. ओͳ7/3FRVFTUͷαϒΫϥε ༻్ 7/%FUFDU'BDF3FDUBOHMFT3FRVFTU إͷۣܗݕग़ 7/%FUFDU'BDF-BOENBSLT3FRVFTU إͷύʔπݕग़ 7/%FUFDU3FDUBOHMFT3FRVFTU ۣܗݕग़ 7/%FUFDU5FYU3FDUBOHMFT3FRVFTU ςΩετۣܗݕग़

    7/5SBDL0CKFDU3FRVFTU ෺ମݕग़ 7/%FUFDU)PSJ[PO3FRVFTU ਫฏݕग़ 7/*NBHF3FHJTUSBUJPO3FRVFTU ը૾ϨδετϨʔγϣϯ ʢҐஔ߹Θͤʣ 7/%FUFDU#BSDPEFT3FRVFTU όʔίʔυݕग़ 7/$PSF.-3FRVFTU ΧελϜͷ$PSF.-Λ༻͍ͨݕग़
  2. ओͳ7/3FRVFTUͷαϒΫϥε ༻్ 7/%FUFDU'BDF3FDUBOHMFT3FRVFTU إͷۣܗݕग़ 7/%FUFDU'BDF-BOENBSLT3FRVFTU إͷύʔπݕग़ 7/%FUFDU3FDUBOHMFT3FRVFTU ۣܗݕग़ 7/%FUFDU5FYU3FDUBOHMFT3FRVFTU ςΩετۣܗݕग़

    7/5SBDL0CKFDU3FRVFTU ෺ମݕग़ 7/%FUFDU)PSJ[PO3FRVFTU ਫฏݕग़ 7/*NBHF3FHJTUSBUJPO3FRVFTU ը૾ϨδετϨʔγϣϯ ʢҐஔ߹Θͤʣ 7/%FUFDU#BSDPEFT3FRVFTU όʔίʔυݕग़ 7/$PSF.-3FRVFTU ΧελϜͷ$PSF.-Λ༻͍ͨݕग़ ·ͣ͜ΕΒΛ࢖͑ͳ͍͔ݕ౼ ཁ݅ʹ߹Θͳ͚Ε͹ $PSF.-Λબ୒
  3. ᶃ܇࿅ࡁΈͷ$PSF.-ϞσϧΛ࢖͏ ໊শ αΠζ ༻్ ࣝผྫ .PCJMF/FU .# ෺ମೝࣝ ໦ɺಈ෺ɺ৯΂෺ɺंɺਓͳͲ 4RVFF[F/FU

    .# 3FT/FU .# *ODFQUJPOW .# 7(( .# 1MBDF(PPH-F/FU .# γʔϯೝࣝ ۭߓɺ৸ࣨɺ৿ྛɺԊ؛ "QQMFެࣜఏڙ IUUQTEFWFMPQFSBQQMFDPNNBDIJOFMFBSOJOH
  4. ໊শ ༻్ $BS3FDPHOJUJPO ंͷϒϥϯυˍϞσϧࣝผ :0-0 ෺ମೝࣝ "HF/FU ਓؒͷإͷ೥ྸਪఆ (FOEFS/FU ਓؒͷإͷੑผਪఆ

    &NPUJPO/FU ਓؒͷإͷײ৘ਪఆ )&% Τοδநग़ 'PPE ྉཧͷࣝผ 0YGPSE Ֆͷछྨͷࣝผ -PDBUJPO/FU ࣸਅͷࡱӨ஍ਪఆ ͦͷଞͷެ։$PSF.-Ϟσϧ IUUQTHJUIVCDPNMJLFEBO"XFTPNF$PSF.-.PEFMT
  5. ໊শ ༻్ $BS3FDPHOJUJPO ंͷϒϥϯυˍϞσϧࣝผ :0-0 ෺ମೝࣝ "HF/FU ਓؒͷإͷ೥ྸਪఆ (FOEFS/FU ਓؒͷإͷੑผਪఆ

    &NPUJPO/FU ਓؒͷإͷײ৘ਪఆ )&% Τοδநग़ 'PPE ྉཧͷࣝผ 0YGPSE Ֆͷछྨͷࣝผ -PDBUJPO/FU ࣸਅͷࡱӨ஍ਪఆ ͦͷଞͷެ։$PSF.-Ϟσϧ IUUQTHJUIVCDPNMJLFEBO"XFTPNF$PSF.-.PEFMT ཁ݅ʹ߹͏΋ͷ͕ͳ͚Ε͹ ➔ࣗ෼Ͱ$PSF.-ϞσϧΛੜ੒͢Δ
  6. ը૾ೝࣝΛ࢖ͬͨΞϓϦͷ։ൃཁ݅ ࢖͑ͦ͏ͳֶशࡁΈ $PSF.-Ϟσϧ͕͋Δʁ ࢖͑ͦ͏ͳ1ZUIPO࣮૷ͷ ֶशࡁΈϞσϧ͕͋Δʁ ࣌ؒͱͱڭࢣσʔλͱ Ϡϧؾ͸͋Δʁ 7JTJPOͰ࣮૷Մೳʁ ᶃ7JTJPOϑϨʔϜϫʔΫΛ࢖͏ ᶄֶशࡁΈ$PSF.-ϞσϧΛ࢖͏

    ᶅ$PSF.-5PPMTͰ $PSF.-Ϟσϧʹม׵ͯ͠࢖͏ ᶆػցֶशͷίʔυΛ1ZUIPOͰ࣮૷ ˍ܇࿅ͯ͠ɺ$PSF.-5PPMTͰ $PSF.-Ϟσϧʹม׵ͯ͠࢖͏ :&4 /0 ➔ΞΠυϧإࣝผΞϓϦ
  7. $ scrapy runspider momospider.py import scrapy from crawler.items import MomoItem

    class MomoSpider(scrapy.Spider): name = "momo" start_urls = [“https://momo.com/blog/“] def parse(self, response): image_url = response.css('#main #content .artwork img::attr(src-swap)').extract_first() member_name = response.css('#title h1::text').extract_first() item = MomoItem() item['image_urls'] = [image_url] item['member_name'] = member_name return item
  8. ࣝผϞσϧ ΞΠυϧإࣝผϞσϧ ϝϯόʔ ֬཰ ΕΈ  ͸ͳ͜  ͓͍͠ 

    ͋ʔΓͦ  ΋΋͜  ਪ࿦ ग़ྗ֤ϝϯόʔͷ֬཰ ೖྗɿY3(#ͷإը૾ ϥϕϧ෇͖ڭࢣը૾ ͸ͳ͜ ೖྗ
  9. ࣝผϞσϧ ΞΠυϧإࣝผϞσϧ ϝϯόʔ ֬཰ ΕΈ  ͸ͳ͜  ͓͍͠ 

    ͋ʔΓͦ  ΋΋͜  ਪ࿦ ग़ྗ֤ϝϯόʔͷ֬཰ ೖྗɿY3(#ͷإը૾ ϥϕϧ෇͖ڭࢣը૾ ͸ͳ͜ ೖྗ ɹֶश
  10. ࣝผϞσϧ ΞΠυϧإࣝผϞσϧ ϝϯόʔ ֬཰ ΕΈ  ͸ͳ͜  ͓͍͠ 

    ͋ʔΓͦ  ΋΋͜  ਪ࿦ ग़ྗ֤ϝϯόʔͷ֬཰ ˠը૾ͷଟΫϥε෼ྨ໰୊ ೖྗɿY3(#ͷإը૾ ϥϕϧ෇͖ڭࢣը૾ ͸ͳ͜ ೖྗ ɹֶश
  11. $PSF.-Ϟσϧ΁ͷม׵ )%' ܗࣜ ɾωοτϫʔΫߏ଄ ɾֶशࡁΈͷॏΈ 1 import coremltools 2 coreml_model

    = coremltools.converters.keras.convert( 3 'model.h5' 4 input_names = 'image', 5 image_input_names = 'image', 6 class_labels = 'labels.txt') 7 coreml_model.save(‘idle_classifer.mlmodel’)
  12. $PSF.-Ϟσϧͷݕূ 1 import coremltools 2 img = load(‘hanako.png’) 3 model

    = coremltools.models.MLModel('Momomind.mlmodel') 4 res = model.predict(img) 5 print(res) # => ͸ͳ͜ 0.8888 ͸ͳ͜  ͸ͳ͜  ਖ਼͘͠ม׵Ͱ͖͍ͯΔ͜ͱ͕ݕূͰ͖ͨ
  13. ϑϨʔϜը૾औಘˍϓϨϏϡʔදࣔ let session = AVCaptureSession() let device = AVCaptureDevice.default(for: AVMediaType.video)!

    let deviceInput = try! AVCaptureDeviceInput(device: device) session.addInput(deviceInput) let previewLayer = AVCaptureVideoPreviewLayer(session: session) view.layer.insertSublayer(previewLayer, at: 0) let queue = DispatchQueue(label: “xxx") let output = AVCaptureVideoDataOutput() output.setSampleBufferDelegate(self, queue: queue) session.addOutput(output) func captureOutput(..., didOutput sampleBuffer: CMSampleBuffer, ..) { let buffer: CVImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) ... } ᶃΧϝϥೖྗ ᶄϓϨϏϡʔදࣔ ᶅΩϟϓνϟग़ྗ ᶆϑϨʔϜը૾औಘ
  14. إҐஔݕग़ let buffer: CVImageBuffer = ... let inputImage = CIImage(cvImageBuffer:

    buffer) //ᶃإۣܗऔಘϦΫΤετ let handler = VNImageRequestHandler(ciImage: inputImage) handler.perform([ VNDetectFaceRectanglesRequest(completionHandler: { (req, err) in //ᶄإۣܗͷऔಘ let req = request as! VNDetectFaceRectanglesRequest, let faces = req.results as! [VNFaceObservation], let face = faces.first.boundingBox, let rect = face.boundingBox, //ᶅإ෦෼ͷ੾Γൈ͖ let faceImage: CIImage = cropImage(from inputImage, rect: rect)
  15. إࣝผ //ᶃֶशࡁΈCoreMLϞσϧͷϩʔυ let idleClassifier = IdleClassifier() let model = VNCoreMLModel(for:

    idleClassifier.model) //ᶄVision+CoreMLͰਪ࿦Λ࣮ߦ let image: CIImage = ... let handler = VNImageRequestHandler(ciImage: image) try handler.perform([ VNCoreMLRequest(model: model) {(req, err) in //ᶅࣝผ݁Ռͷऔಘ let res = req.results!.first as! VNClassificationObservation let name = res.identifier let probability = Int(res.confidence * 100) print(“ࣝผ݁Ռ \(name):\(probability)%”)
  16. ͸ͳ͜ ࠶ੜ੒ͨ͠Ϟσϧͷಈతஔ׵ μ΢ϯϩʔυ NMNPEFM NMNPEFMD class MLModel { class func

    compileModel(at modelURL: URL) throws -> URL ίϯύΠϧ ୺຤಺ JOPVUͷ࢓༷มߋͳͲ ☓ ࢀߟɿ)PUTXBQQJOH$PSF.-NPEFMTPOUIFJ1IPOF
  17. ࢀߟจݙ w ͍Β͢ͱ΍ʮΞΠυϧʯͷݕࡧ݁Ռ w .BDIJOF-FBSOJOH"QQMF w ਂ૚ֶश%FFQ-FBSOJOHʢۙ୅Պֶࣾʣ w ਂ૚ֶशʢػցֶशϓϩϑΣογϣφϧγϦʔζʣ w

    θϩ͔Β࡞Δ%FFQ-FBSOJOH w 5FOTPSqPXCFHJOOFST w DJGBS w ("/ (FOFSBUJWF"EWFSTBSJBM/FUXPSLTBS9JW w )PUTXBQQJOH$PSF.-NPEFMTPOUIFJ1IPOF w إೝࣝҐஔ߹Θͤͷॏཁੑ