Vision.framework - 商品画像からのテキスト検出と並列化実装への試み

Slide 1

Slide 1 text

ʙ Vision.framework ͷ׆༻ࣄྫ ʙ ঎඼ը૾͔ΒͷςΩετݕग़ͱ ฒྻԽ࣮૷΁ͷࢼΈ 2023.05.22 DeNA×STORES×ϥΫϚ iOS Meetup!! Hiromu Tsuruta @ Rakuten Rakuma

Slide 2

Slide 2 text

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　・Rakuten Group, Inc 　　　　　　　　　　　　　　　　　　　　・Rakuma 　　　　　　・iOS Developer 　　　・Developer Relations 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　・Twitter：@hcrane14

Slide 3

Slide 3 text

Vision.framework ͱ͸ 🤔

Slide 4

Slide 4 text

Quote: https://developer.apple.com/documentation/vision

Slide 5

Slide 5 text

Apply computer vision algorithms to perform a variety of tasks on input images and video. The Vision framework performs face and face landmark detection, text detection, barcode recognition, image registration, and general feature tracking. Vision also allows the use of custom Core ML models for tasks like classi fi cation or object detection.

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Vision.framework ͷྺ࢙

Slide 8

Slide 8 text

iOS11: Vision.framework ͷొ৔ɻCore ML ͱ૊Έ߹Θͤͯจࣈೝࣝɻ iOS13: จࣈೝ͕ࣝՄೳʹͳΔɻͨͩ͠ɺӳޠͷΈɻ iOS14: 7ͭͷݴޠʹରԠ͢Δɻ೔ຊޠ͸ະରԠɻ iOS16: 14ͭͷݴޠʹରԠ͢Δɻ೔ຊޠʹ΋ରԠɻ ※จࣈೝࣝʹؔͯ͠ͷΈ

Slide 9

Slide 9 text

Vision.framework ͷ׆༻ྫ

Slide 10

Slide 10 text

PayPayϑϦϚ Quote: fi ndy-code.io/events/dtOzzlRgwookF?fr=event_20230208

Slide 11

Slide 11 text

ϥΫϚͰͷ׆༻ྫ

Slide 12

Slide 12 text

※·ͩϦϦʔε͸͞Ε͍ͯ·ͤΜ

Slide 13

Slide 13 text

Vision.framework ͷ࣮૷

Slide 14

Slide 14 text

VNDetectFaceRectanglesRequest VNDetectFaceLandmarksRequest VNDetectRectanglesRequest VNDetectBarcodesRequest VNDetectTextRectanglesRequest  … Request VNImageRequestHandler  VNSequenceRequestHandler  … VNFaceObservation  VNImageAlignmentObservation  VNRectangleObservation  VNBarcodeObservation  … RequestHandler Observation

Slide 15

Slide 15 text

Example ֤ Observation ͷ਌ΫϥεͰ͋Δ VNObservation ͷ഑ྻ͔͠औಘͰ͖ͳ͍

Slide 16

Slide 16 text

Helper Extension ը૾͔Βೝࣝͨ͠ςΩετͷ݁Ռ ؍ଌͨ͠ςΩετͷީิΛऔಘ

Slide 17

Slide 17 text

.topCandidates ͷڍಈ

Slide 18

Slide 18 text

Image Model Quote: https://www.rakuteneagles.jp/news/detail/202300285435.html

Slide 19

Slide 19 text

.topCandidates(1)

Slide 20

Slide 20 text

.topCandidates(4)

Slide 21

Slide 21 text

.topCandidates(4) ߴ ௿ ਫ਼౓

Slide 22

Slide 22 text

Vision.framework ͷ ࣮૷ʹ͓͍ͯͷ՝୊ 👀

Slide 23

Slide 23 text

ϥΫϚͰ͸ ࠷େ̍̌ຕͷ ը૾Λग़඼Ͱ͖Δ ը૾ʹΑͬͯ͸ɺςΩ ετ͕ฦͬͯ͘Δͷʹ 0.5ඵ͔͔ͬͨΓ͢Δ ͜ͱ΋͋Δ

Slide 24

Slide 24 text

ॲཧΛฒྻԽ͢Δ

Slide 25

Slide 25 text

Recognizer TaskGroup Task Task Finish ɾ ɾ ɾ AsyncStream Task Start

Slide 26

Slide 26 text

※ίʔυ͸Ұ෦Λলུ͍ͯ͠·͢ Request ͱ RequestHandler Λ AsyncThrowingStream Λ࢖ͬͯɺasync ʹରԠͤ͞Δ