Vision Framework 入門

potatotips #42 https://potatotips.connpass.com/event/60015/
※公開されている情報を元に構成されています

Transcript

1. 7JTJPO'SBNFXPSLೖ໳
   !NPUPLJFF
   QPUBUPUJQT
   

   View Slide

2. ࣗݾ঺հ
   w HJUIVC UXJUUFS!NPUPLJFF
   w ϝϧΧϦι΢κ΢J04ΤϯδχΞ
   w ϝϧΧϦΧ΢ϧνʔϜ
   w J04%$+BQBOελοϑ
   

   View Slide

3. 7JTJPO'SBNFXPSLͱ͸
   

   View Slide

4. $PSF.-ͱ7JTJPOͷҧ͍͸ʁ
   

   View Slide

5. ΞϓϦέʔγϣϯ
   7JTJPO /-1
   $PSF.-
   "DDFMFSBUF .14
   IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΛࢀߟʹ࡞੒
   

   View Slide

6. ΞϓϦέʔγϣϯ
   7JTJPO /-1

   /BUVSBM-BOHVBHF1SPDFTTJOH
   $PSF.-
   "DDFMFSBUF .14
   %PNBJO4QFDJpD
   ಛఆͷλεΫʹಛԽͨ͠ը૾ղੳͷͨΊͷϑϨʔϜϫʔΫ
   

   View Slide

7. ΞϓϦέʔγϣϯ
   7JTJPO /-1

   /BUVSBM-BOHVBHF1SPDFTTJOH
   $PSF.-
   "DDFMFSBUF .14
   7JTJPO͸ը૾ղੳʹಛԽ͍ͯ͠Δ
   %PNBJO4QFDJpD
   

   View Slide

8. ΞϓϦέʔγϣϯ
   7JTJPO /-1

   /BUVSBM-BOHVBHF1SPDFTTJOH
   $PSF.-
   "DDFMFSBUF .14
   %PNBJO4QFDJpD
   7JTJPO͸$PSF.-͕ϕʔεʹͳ͍ͬͯΔ
   

   View Slide

9. ΞϓϦέʔγϣϯ
   7JTJPO /-1

   /BUVSBM-BOHVBHF1SPDFTTJOH
   $PSF.-
   "DDFMFSBUF .14
   %PNBJO4QFDJpD
   ྡΓʹ͋Δ/-1͸'PVOEBUJPOʹ૊Έࠐ·Ε͍ͯͯɺ
   /4-JOHVJTUJD5BHHFSͳͲͰ࢖ΘΕ͍ͯΔͱͷ͜ͱ
   

   View Slide

10. ࠓճ΍Γ͔ͨͬͨͷ͸ɺ
    7JTJPOͰόʔίʔυΛݕग़͢Δ͜ͱ
    

    View Slide

11. Barcode Detection
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΑΓ
    

    View Slide

12. w 7JTJPOΛ࢖ͬͯόʔίʔυͷݕग़Λ͍ͨ͠
    w ݁࿦͔Βݴ͏ͱ7JTJPO.-&YBNQMFͱ͍͏88%$
    ͷηογϣϯͰ΋࢖ΘΕ͍ͯͨαϯϓϧΛݩʹػց
    ֶशͷ஌͕ࣝͳͯ͘΋͢ΜͳΓ7JTJPO͸ಈ͔͢͜ͱ
    ͕Ͱ͖ͨ
    w ͨͩ͠ͱ͍͏࿩Λ͢Δ
    

    View Slide

13. ࣮૷ํ๏
    

    View Slide

14. IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΑΓ
    Analyzing an Image
    The
    Machinery
    The Results
    The Asks
    

    View Slide

15. IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΑΓ
    Analyzing an Image
    The
    Machinery
    The Results
    The Asks
    Requests
    VNDetectBarcodesRequest
    VNDetectFaceLandmarksRequest
    VNDetectFaceRectanglesRequest
    …
    Observations
    VNClassificationObservation
    VNDetectedObjectObservation
    VNFaceObservation
    …
    RequestHandler
    VNImageRequestHandler
    

    View Slide

16. w ը૾͔ΒόʔίʔυΛݕग़
    w 7/%FUFDU#BSDPEFT3FRVFTUΛ࢖ͬͯɺը૾͔Β
    όʔίʔυΛಡΈऔΔ
    

    View Slide

17. Analyzing an Image
    The
    Machinery
    The Results
    The Asks
    Requests
    VNDetectBarcodesRequest
    VNDetectFaceLandmarksRequest
    VNDetectFaceRectanglesRequest
    …
    Observations
    VNClassificationObservation
    VNDetectedObjectObservation
    VNFaceObservation
    …
    RequestHandler
    VNImageRequestHandler
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΑΓ
    

    View Slide

18. let handler = VNImageRequestHandler(ciImage: ciImage, orientation: orientation)
    try! handler.perform([barcodeClassificationRequest])
    

    View Slide

19. w ը૾͔Βόʔίʔυͷ*4#/ίʔυΛऔΓग़͢
    

    View Slide

20. Analyzing an Image
    The
    Machinery
    The Results
    The Asks
    Requests
    VNDetectBarcodesRequest
    VNDetectFaceLandmarksRequest
    VNDetectFaceRectanglesRequest
    …
    Observations
    VNClassificationObservation
    VNDetectedObjectObservation
    VNFaceObservation
    …
    RequestHandler
    VNImageRequestHandler
    IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΑΓ
    

    View Slide

21. lazy var barcodeClassificationRequest: VNDetectBarcodesRequest = {
    return VNDetectBarcodesRequest(completionHandler: { request, error in
    guard let observations = request.results as? [VNBarcodeObservation] else {
    return
    }
    let rects = observations.map { $0.boundingBox }
    DispatchQueue.main.async { [weak self] in
    for rect in rects {
    let view = UIView(frame: convertedRect(rect: rect, to: self!.imageView.frame.size))
    view.backgroundColor = .green
    self?.imageView.addSubview(view)
    }
    }
    let _ = observations.flatMap { observation -> CIBarcodeDescriptor? in
    if let desc = observation.barcodeDescriptor as? CIBarcodeDescriptor {
    return desc
    } else {
    return nil
    }
    }
    })
    }()
    

    View Slide

22. lazy var barcodeClassificationRequest: VNDetectBarcodesRequest = {
    return VNDetectBarcodesRequest(completionHandler: { request, error in
    guard let observations = request.results as? [VNBarcodeObservation] else {
    return
    }
    let rects = observations.map { $0.boundingBox }
    DispatchQueue.main.async { [weak self] in
    for rect in rects {
    let view = UIView(frame: convertedRect(rect: rect, to: self!.imageView.frame.size))
    view.backgroundColor = .green
    self?.imageView.addSubview(view)
    }
    }
    let _ = observations.flatMap { observation -> CIBarcodeDescriptor? in
    if let desc = observation.barcodeDescriptor as? CIBarcodeDescriptor {
    return desc
    } else {
    return nil
    }
    }
    })
    }()
    

    View Slide

23. ͱΓ͋͑ͣͰ͖ͨ
    

    View Slide

24. ͕ɺ
    

    View Slide

25. όʔίʔυ͕Ͳ͜ʹ͋Δ͔͸Θ͔͚ͬͨͲɺ
    ಺༰ʢ&"/ίʔυʣ͸෼͔Βͳ͔ͬͨʜ
    

    View Slide

26. lazy var barcodeClassificationRequest: VNDetectBarcodesRequest = {
    return VNDetectBarcodesRequest(completionHandler: { request, error in
    guard let observations = request.results as? [VNBarcodeObservation] else {
    return
    }
    let rects = observations.map { $0.boundingBox }
    DispatchQueue.main.async { [weak self] in
    for rect in rects {
    let view = UIView(frame: convertedRect(rect: rect, to: self!.imageView.frame.size))
    view.backgroundColor = .green
    self?.imageView.addSubview(view)
    }
    }
    let _ = observations.flatMap { observation -> CIBarcodeDescriptor? in
    if let desc = observation.barcodeDescriptor as? CIBarcodeDescriptor {
    return desc
    } else {
    return nil
    }
    }
    })
    }()
    

    View Slide

27. w $*#BS$PEF%FTDSJQUPSͰ&"/ίʔυ͕औಘͰ͖Δ͸
    ͣɺͱࢥ͍͕ͬͯͨɺͦΜͳ΋ͷ͸ͳ͔ͬͨ
    w $*#BS$PEF%FTDSJQUPS͸/40CKFDUͷαϒΫϥεͰ
    ಛʹԿ΋࣮૷͞Ε͍ͯͳ͍
    

    View Slide

28. IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZXXEDΑΓ
    Framework Barcode Support
    Several frameworks support various barcodes types
    AVFoundation.framework Vision.framework CoreImage.framework
    Detection of barcodes

    during capture
    Detection of barcodes

    after capture
    Rendering barcodes
    CIBarcodeDescriptor
    

    View Slide

29. IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPODPSFJNBHFDJCBSDPEFEFTDSJQUPS
    

    View Slide

30. w ࣮૷͸$*#BS$PEF%FTDSJQUPSͷαϒΫϥεʹҕͶΒ
    Ε͍ͯͯɺ͍·ཉ͍͠΋ͷ&"/ίʔυʹରԠͨ͠
    ΋ͷ͸ͳ͔ͬͨ
    IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPODPSFJNBHFDJCBSDPEFEFTDSJQUPS
    

    View Slide

31. w $*23$PEF%FTDSJQUPS
    w $*"[UFD$PEF%FTDSJQUPS
    w $*1%'$PEF%FTDSJQUPS
    w $*%BUB.BUSJY$PEF%FTDSJQUPS
    ରԠ͍ͯ͠Δ̎࣍ݩίʔυ
    

    View Slide

32. Framework Barcode Support
    Several frameworks support various barcodes types
    AVFoundation.framework Vision.framework CoreImage.framework
    Detection of barcodes

    during capture
    Detection of barcodes

    after capture
    Rendering barcodes
    CIBarcodeDescriptor
    

    View Slide

33. ·ͱΊ
    w 7JTJPOΛ࢖͏ͱόʔίʔυ͕ը૾ͷதʹ͋Δͱ͍͏
    ͜ͱ͸෼͔Δ
    w ը໘্ͷͲ͜ʹ͋Δ͔ͷ࠲ඪ͸औΕΔͷͰɺؤுͬ
    ͯ&"/ίʔυม׵·Ͱ͸Ͱ͖Δ͸ͣ
    w IUUQTEFWFMPQFSBQQMFDPNWJEFPTQMBZ
    XXED UJNF
    

    View Slide

34. ͓ΘΓ
    

    View Slide