WWDC2025セッション共有: Visionフレームワークによるドキュメントの読み込み

in Swift愛好会スピンオフ WWDC25セッション要約会@DeNA (2025/06/26)

Transcript

1. WWDC2025ηογϣϯڞ༗: VisionϑϨʔϜϫʔΫʹΑΔυΩϡϝϯτͷ ಡΈࠐΈ in SwiftѪ޷ձεϐϯΦϑ WWDC25ηογϣϯཁ໿ձ@DeNA (2025/06/26) ʹʔ͞Μ(@ni_san2000)
2. ࣗݾ঺հ • ʹʔ͞Μ(@ni_san2000) • ۀ຿ͰiOSΞϓϦΛ࡞͍ͬͯ·͢(໿3೥) • Apple৴ऀͳͷͰجௐߨԋ͸ੲ͔ΒݟͯΔ • macOSͷωʔϛϯάετʔϦʔ • ΫϨΠάɾϑΣσϦΪͷύϧΫʔϧ • झຯ • Χϝϥ / ؍༿২෺ / ίʔώʔ / eεϙʔπ • ࠷ۙࣸਅίϯςετͱ͔ڵຯ͋Γ·͢ 2
3. ࿩͢ηογϣϯʹ͍ͭͯ • VisionϑϨʔϜϫʔΫʹΑΔυΩϡϝϯτͷಡΈࠐΈ • “Foundation ModelsͰ͸ͳ͍”ɺAIؔ࿈ͷ͓࿩Ͱ͢ 3 ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
4. VisionϑϨʔϜϫʔΫʹ͍͓ͭͯ͞Β͍ • ಈը૾ʹಛԽͨ͠AIϞσϧͷػೳΛఏڙ͢ΔϑϨʔϜϫʔΫ • ը૾ೝࣝ, ෺ମݕग़, إೝࣝͳͲ • .mlmodelΛཁ͢ΔػೳΛखܰʹAPIͱͯ͠ར༻Ͱ͖Δ • iOS18Ͱ͸31ݸͷAPIΛఏڙ͍ͯͨ͠ 4 ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQNBDIJOFMFBSOJOHNPEFMT
5. VisionϑϨʔϜϫʔΫʹ͍͓ͭͯ͞Β͍ CalculateImageAestheticsScoresRequest ClassifyImageRequest CoreMLRequest DetectAnimalBodyPoseRequest DetectBarcodesRequest DetectContoursRequest DetectDocumentSegmentationRequest DetectFaceCaptureQualityRequest DetectFaceLandmarksRequest DetectFaceRectanglesRequest DetectHorizonRequest DetectHumanBodyPose3DRequest DetectHumanBodyPoseRequest DetectHumanHandPoseRequest DetectHumanRectanglesRequest DetectRectanglesRequest DetectTextRectanglesRequest DetectTrajectoriesRequest GenerateAttentionBasedSaliencyIma GenerateForegroundInstanceMaskRequest GenerateImageFeaturePrintRequest GenerateOpticalFlowRequest GeneratePersonInstanceMaskRequest GeneratePersonSegmentationRequest RecognizeAnimalsRequest RecognizeTextRequest TrackHomographicImageRegistration TrackObjectRequest TrackOpticalFlowRequest TrackRectangleRequest TrackTranslationalImageRegistrationRequest 5 RecognizeDocumentRequest /FX DetectLensSmudgeRequest /FX
6. ͜Ε·ͰͷυΩϡϝϯτೝࣝ • ݟ͍͑ͯΔจࣈ͸ೝࣝͰ͖Δ • ͚Ͳɺจষߏ੒΍දͷίϯςΩετ͸ࣦΘΕΔ 6 ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
7. xOS26͔ΒͰ͖ΔΑ͏ʹͳΔ͜ͱ • υΩϡϝϯτ͔Βจॻͷߏ੒Λཧղͯ͠ཁૉͷೝ͕ࣝͰ͖Δ • ஈམɾՕ৚ॻ͖΍දͷίϯςΩετೝࣝ • ಛఆͷϑΥʔϚοτʹଈͨ͠ςΩετͷࣝผ • QRίʔυͷऔΓग़͠ 7
8. υΩϡϝϯτ͔Βจॻͷߏ੒Λཧղͯ͠ཁૉͷೝ͕ࣝͰ͖Δ ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
9. ᶃ RequestΫϥεͷ༻ҙ (RecognizeDocumentRequest) ᶄ ը૾Λ౉͢ ᶅ DocumentObservationΛड͚औΔ ᶆ document͔Β֤ཁૉΛऔಘ ᶇ That’s it !
10. DocumentObservationͷݕग़ߏ੒ • Container͸List, Table, Text, BarcodesͳͲͷཁૉΛ࣋ͭ
    • $FMM΍*UFNT͸$POUBJOFSͷཁૉ΋͍࣋ͬͯΔͷͰɺ͞Βʹ಺แͰ͖Δ 10 ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED ☝
11. • ࣝผϑΥʔϚοτ͸10छྨ • ೔ຊޠ͸·ͩNot supported • දࣔ͢ΔࡍʹText AttributeΛࢦఆ͢Δඞཁ͕ͳ͘ͳΓͦ͏ ςΩετΛࣝผ͢ΔDataDetector 11 આ໌ 5ZQF ิ଍ ΧϨϯμʔ CalendarEvent ϝʔϧΞυϨε EmailAddress ϑϥΠτ൪߸ FlightNumber 63- Link ଌఆ஋ ୯Ґ෇͖ Measurement %JNFOTJPOͰදݱͰ͖Δ୯Ґ ֹۚ ௨՟෇͖ MoneyAmount -PDBMF$VSSFODZͰදݱͰ͖Δ୯Ґ ࢧ෷͍ঢ়گ PaymentIdentifier 6OJ fi FE
    1BZNFOUT
    *OUFSGBDF
    61* ి࿩൪߸ PhoneNumber ॅॴ PostalAddress FH
    TUSFFU
    DJUZ
    TUBUF ഑ૹ௥੻൪߸ ShipmentTrackingNumber ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
12. ֤ϑΥʔϚοτͱͯ͠σʔλΛऔಘ • switch-case͢Δ͚ͩ • ͲͷϑΥʔϚοτ͔͸ςΩετ୯ҐͰࣝผࡁΈ 12
13. υΩϡϝϯτ͔Βจॻͷߏ੒Λཧղͯ͠ཁૉͷೝ͕ࣝͰ͖Δ ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
14. • VisionϑϨʔϜϫʔΫͷ৽نAPI • υΩϡϝϯτͷ಺༰Λߏ଄తʹ෼ੳͰ͖Δ • Container͸Table, List, Cell, TextͳͲͷཁૉΛ͍࣋ͬͯΔ • Cell΍ListͷItem͸ೖΕࢠͰContainerΛ֨ೲͰ͖Δ • Text͸֨ೲ͞Εͨจࣈ͔ΒϑΥʔϚοτΛೝࣝͰ͖Δ • DataDetector.Match.SemanticDetails.XXXXXʹଟ਺ͷ৽نσʔλߏ଄ RecognizeDocumentRequest·ͱΊ 14
15. ࢒ΓͷVisionؔ࿈ͷ৽ཁૉ̎ͭ ͓·͚ 15
16. • DetectLensSmudgeRequest / SmudgeObservation͕௥Ճ • ೖྗͨ͠ը૾͕ԚΕ͍ͯΔ͔Λ൑ผ͢Δ • ᮢ஋Λઃ͚Δ͜ͱͰ෼ੳΤϥʔΛະવʹ๷͛Δ Ϩϯζද໘ͷԚΕ΍ಶΓΛݕग़͢ΔObservation 16 ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
17. • खͷؔઅҐஔΛݕग़͢ΔϞσϧ͕ߋ৽͞Εͨ • ਪ࿦ͷਫ਼౓ͱ଎౓͕޲্ • WWDC2021Ͱൃද͞ΕͨϞσϧ͔ΒΞοϓσʔτ HandPose Detectionͷ෼ੳϞσϧߋ৽ 17 ႄႨჭğIUUQTEFWFMPQFSBQQMFDPNKQWJEFPTQMBZXXED
18. • VisionϑϨʔϜϫʔΫͷػೳ௥Ճ • υΩϡϝϯτͷ಺༰Λߏ଄తʹ෼ੳͰ͖Δ • ςΩετͷ಺༰Λ෼ྨͰ͖Δ • ͦͷଞʹ΋2ͭͷVisionϑϨʔϜϫʔΫͷΞοϓσʔτ • DetectLensSmudgeRequest / SmudgeObservation • HandPose Detectionͷ෼ੳϞσϧߋ৽ ͓͠·͍ 18