$30 off During Our Annual Pro Sale. View Details »

Vision.framework - 商品画像からのテキスト検出と並列化実装への試み

Vision.framework - 商品画像からのテキスト検出と並列化実装への試み

2023.05.22 DeNA×STORES×ラクマ iOS Meetup!!

h.crane

May 22, 2023
Tweet

More Decks by h.crane

Other Decks in Technology

Transcript

  1. ʙ Vision.framework ͷ׆༻ࣄྫ ʙ


    ঎඼ը૾͔ΒͷςΩετݕग़ͱ


    ฒྻԽ࣮૷΁ͷࢼΈ
    2023.05.22 DeNA×STORES×ϥΫϚ iOS Meetup!!

    Hiromu Tsuruta @ Rakuten Rakuma

    View Slide

  2.                   

                      
    ・Rakuten Group, Inc
              
              ・Rakuma
       
       ・iOS Developer
       ・Developer Relations
                     

                      
    ・Twitter:@hcrane14

    View Slide

  3. Vision.framework ͱ͸ 🤔

    View Slide

  4. Quote: https://developer.apple.com/documentation/vision

    View Slide

  5. Apply computer vision algorithms to perform a variety
    of tasks on input images and video.

    The Vision framework performs face and face landmark
    detection, text detection, barcode recognition, image
    registration, and general feature tracking. Vision also
    allows the use of custom Core ML models for tasks like
    classi
    fi
    cation or object detection.

    View Slide

  6. View Slide

  7. Vision.framework ͷྺ࢙

    View Slide

  8. iOS11: Vision.framework ͷొ৔ɻCore ML ͱ૊Έ߹Θͤͯจࣈೝࣝɻ

    iOS13: จࣈೝ͕ࣝՄೳʹͳΔɻͨͩ͠ɺӳޠͷΈɻ

    iOS14: 7ͭͷݴޠʹରԠ͢Δɻ೔ຊޠ͸ະରԠɻ

    iOS16: 14ͭͷݴޠʹରԠ͢Δɻ೔ຊޠʹ΋ରԠɻ
    ※จࣈೝࣝʹؔͯ͠ͷΈ

    View Slide

  9. Vision.framework ͷ׆༻ྫ

    View Slide

  10. PayPayϑϦϚ
    Quote:
    fi
    ndy-code.io/events/dtOzzlRgwookF?fr=event_20230208

    View Slide

  11. ϥΫϚͰͷ׆༻ྫ

    View Slide

  12. ※·ͩϦϦʔε͸͞Ε͍ͯ·ͤΜ

    View Slide

  13. Vision.framework ͷ࣮૷

    View Slide

  14. VNDetectFaceRectanglesRequest
    VNDetectFaceLandmarksRequest
    VNDetectRectanglesRequest
    VNDetectBarcodesRequest
    VNDetectTextRectanglesRequest


    Request
    VNImageRequestHandler

    VNSequenceRequestHandler


    VNFaceObservation

    VNImageAlignmentObservation

    VNRectangleObservation

    VNBarcodeObservation


    RequestHandler Observation

    View Slide

  15. Example
    ֤ Observation ͷ਌ΫϥεͰ͋Δ VNObservation ͷ഑ྻ͔͠औಘͰ͖ͳ͍

    View Slide

  16. Helper Extension
    ը૾͔Βೝࣝͨ͠ςΩετͷ݁Ռ ؍ଌͨ͠ςΩετͷީิΛऔಘ

    View Slide

  17. .topCandidates ͷڍಈ

    View Slide

  18. Image Model
    Quote: https://www.rakuteneagles.jp/news/detail/202300285435.html

    View Slide

  19. .topCandidates(1)

    View Slide

  20. .topCandidates(4)

    View Slide

  21. .topCandidates(4)
    ߴ
    ௿
    ਫ਼౓

    View Slide

  22. Vision.framework ͷ


    ࣮૷ʹ͓͍ͯͷ՝୊ 👀

    View Slide

  23. ϥΫϚͰ͸
    ࠷େ̍̌ຕͷ
    ը૾Λग़඼Ͱ͖Δ
    ը૾ʹΑͬͯ͸ɺςΩ
    ετ͕ฦͬͯ͘Δͷʹ
    0.5ඵ͔͔ͬͨΓ͢Δ
    ͜ͱ΋͋Δ

    View Slide

  24. ॲཧΛฒྻԽ͢Δ

    View Slide

  25. Recognizer


    TaskGroup


    Task
    Task
    Finish
    ɾ
    ɾ
    ɾ
    AsyncStream
    Task
    Start

    View Slide

  26. ※ίʔυ͸Ұ෦Λলུ͍ͯ͠·͢
    Request ͱ RequestHandler Λ AsyncThrowingStream Λ࢖ͬͯɺasync ʹରԠͤ͞Δ

    View Slide

  27. ฦ͖ͬͯͨ஋Λ AsyncThrowingStream ʹྲྀ͢
    ※ίʔυ͸Ұ෦Λলུ͍ͯ͠·͢

    View Slide

  28. ※ίʔυ͸Ұ෦Λলུ͍ͯ͠·͢
    TaskGroup ͰॲཧΛฒྻ࣮ߦ͢Δ

    View Slide

  29. ※ίʔυ͸Ұ෦Λলུ͍ͯ͠·͢
    ݺͼग़͠ݩͷؔ਺͸εοΩϦʂ

    View Slide

  30. ϦϦʔεָ͓͠Έʹʂ

    View Slide

  31. Vision.framework ͸͍͍ͧʂʂ

    View Slide

  32. Fin

    View Slide