OCRを使ってゲームのアイテムをデータ化する

プロトタイプを製品にする技術
OCRを使ってゲームのアイテムをデータ化する

Transcript

1. ϓϩτλΠϓΛ੡඼ʹ͢Δٕज़ LJTIJLBXB
   LBUTVNJ !LJTIJLBXBLBUTVNJ!IBDIZEFSNJP LJTIJLBXBLBUTVNJ 0$3Λ࢖ͬͯήʔϜͷΞΠςϜΛσʔλԽ͢Δ
3. ϓϩτλΠϓΛ੡඼ʹ͢Δٕज़ r ໿ݸͷΞΠςϜΛਖ਼֬ʹσʔλԽ͢Δ
   r ̍ͭʹ͖ͭඵҎ಺ʹಡΈऔΕΔ
   r ήʔϜ಺ͷݴޠ͕೔ຊޠͱӳޠͷͲͪΒͰ΋ಡΈऔΕΔ
4. ϓϩτλΠϓΛ੡඼ʹ͢Δٕज़ r ໿ݸͷΞΠςϜΛਖ਼֬ʹσʔλԽ͢Δ
   r ̍ͭʹ͖ͭඵҎ಺ʹಡΈऔΕΔ
   r ήʔϜ಺ͷݴޠ͕೔ຊޠͱӳޠͷͲͪΒͰ΋ಡΈऔΕΔ ਫ਼౓
5. ϓϩτλΠϓΛ੡඼ʹ͢Δٕज़ r ໿ݸͷΞΠςϜΛਖ਼֬ʹσʔλԽ͢Δ
   r ̍ͭʹ͖ͭඵҎ಺ʹಡΈऔΕΔ
   r ήʔϜ಺ͷݴޠ͕೔ຊޠͱӳޠͷͲͪΒͰ΋ಡΈऔΕΔ ਫ਼౓ ଎౓
6. ϓϩτλΠϓΛ੡඼ʹ͢Δٕज़ r ໿ݸͷΞΠςϜΛਖ਼֬ʹσʔλԽ͢Δ
   r ̍ͭʹ͖ͭඵҎ಺ʹಡΈऔΕΔ
   r ήʔϜ಺ͷݴޠ͕೔ຊޠͱӳޠͷͲͪΒͰ΋ಡΈऔΕΔ ਫ਼౓ ଎౓ ॊೈੑ
7. 4BNQMF
   $PEF HJUIVCDPNLJTIJLBXBLBUTVNJ$BNFSB0$3
10. େ͖͞ ৭ छྨ ޮՌςΩετ
11. 4UFQ
    
    3BX
    0$3 actor OCRRunner { private var busy = false func process(_ cgImage: CGImage) async -> [RecognizedTextObservation]? { guard !busy else { return nil } busy = true defer { busy = false } var request = RecognizeTextRequest() request.recognitionLanguages = [ Locale.Language(identifier: "ja-JP"), Locale.Language(identifier: "en-US"), ] request.recognitionLevel = .accurate request.usesLanguageCorrection = false return try? await request.perform(on: cgImage) } }
13. 4UFQ
    
    4UBCJMJUZ
    'JMUFS
14. let windowSize: Int = 5 let minHits: Int = 3 func updateStability(with results: [RecognizedTextObservation]) { let textsThisFrame = Set( results.compactMap { (observation) -> String? in let raw = observation.topCandidates(1).first?.string ?? "" let t = raw.trimmingCharacters(in: .whitespacesAndNewlines) return t.isEmpty ? nil : t } ) recentTextSets.append(textsThisFrame) if recentTextSets.count > windowSize { recentTextSets.removeFirst(recentTextSets.count - windowSize) } var counts: [String: Int] = [:] for set in recentTextSets { for t in set { counts[t, default: 0] += 1 } } let stable = counts.filter { $0.value >= minHits } stableTexts = Set(stable.keys) stableLines = stable .map { StableLine(text: $0.key, hits: $0.value) } .sorted { $0.hits == $1.hits ? $0.text < $1.text : $0.hits > $1.hits } } ϑϨʔϜ಺ͰճҎ্ग़ݱͨ͠
    ςΩετΛ࠾༻͢Δɻ
16. Ԍ߈ܸྗ্ঢ Ԍ߈ܸྗ্ঢ ཕ߈ܸྗ্ঢ Ԍ߈ܸྗ্ঢ ཕ߈ܸྗ্ঢ ग़ܸ࣌ͷ෢ثʹ
    ੟߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ੟߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ཕ߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ཕ߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ཕ߈ܸྗΛ෇Ճ ϦϯάόοϑΝʹΑΔ҆ఆੑͷ޲্ /ϑϨʔϜத.ճҎ্ग़ͨςΩετΛ࠾༻͢Δ
17. Ԍ߈ܸྗ্ঢ Ԍ߈ܸྗ্ঢ ཕ߈ܸྗ্ঢ Ԍ߈ܸྗ্ঢ ཕ߈ܸྗ্ঢ ग़ܸ࣌ͷ෢ثʹ
    ੟߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ੟߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ཕ߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ཕ߈ܸྗΛ෇Ճ ग़ܸ࣌ͷ෢ثʹ
    ཕ߈ܸྗΛ෇Ճ ϦϯάόοϑΝʹΑΔ҆ఆੑͷ޲্ /ϑϨʔϜத.ճҎ্ग़ͨςΩετΛ࠾༻͢Δ
18. 4UFQ
    
    30*
    3FHJPO
    PG
    *OUFSFTU
19. ΨΠυ࿮ͷൣғ͚ͩಡΈऔΔ ؔ܎ͳ͍ςΩετΛಡ·ͳ͍ɾ଎౓޲্
20. static let roiOnScreen = CGRect(x: 0.08, y: 0.30, width: 0.84, height: 0.32) static let visionROI: NormalizedRect = { let r = roiOnScreen return NormalizedRect(x: r.minX, y: 1 - r.maxY, width: r.width, height: r.height) }() func process( _ cgImage: CGImage, roi: NormalizedRect ) async -> [RecognizedTextObservation]? { ... request.regionOfInterest = roi ... } 6*ͷ࠲ඪͷ7JTJPOGSBNFXPSLͷ
    ࠲ඪʹม׵ͯ͠ηοτ ΨΠυ࿮ͷൣғ͚ͩಡΈऔΔ ؔ܎ͳ͍ςΩετΛಡ·ͳ͍ɾ଎౓޲্
22. 4UFQ
    
    .BTUFS
    .BUDIJOH
23. Ϛελʔσʔλͱর߹
24. func bestMatch(for input: String) -> (master: Master, distance: Int)? { let n = normalize(input) var best: (Master, Int)? for (key, master) in normalizedKeys { if abs(key.count - n.count) > 5 { continue } let d = levenshtein(n, key) if best == nil || d < best!.1 { best = (master, d) } } return best } // डཧ: ڑ཭ ≤ max(1, |master| × 0.3) ≒ score ≥ 0.70 let threshold = max(1, Int(Double(master.textJa.count) * 0.3)) guard match.distance <= threshold else { return nil } ฤूڑ཭
    -FWFOTIUFJO
    %JTUBODF ͰҰக౓Λ൑ఆ Ϛελʔσʔλͱর߹
26. ΞϧΰϦζϜ )BNNJOH
    ڑ཭ ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ ࠷଎
    903
    
    QPQDPVOU ɻಉ͡௕͞ͷ
    
    จࣈྻͰʮҟͳΔҐஔͷ਺ʯΛ਺͑Δ ࠾༻͠ͳ͔ͬͨཧ༝ 0$3
    ͷจࣈ௕͸
    GSBNF
    ͝ͱʹ༳ΕΔͨΊద༻ෆՄɻ
    จࣈͰ΋௕͕͞ҧ͏ͱ࢖͑ͳ͍ ΞϧΰϦζϜ OHSBN
    +BDDBSE
    ྨࣅ౓ ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ ଎͍
    ू߹ԋࢉ ɻจࣈ
    /HSBN
    ू߹Λ࡞Γ
    c"ˬ#c
    
    c"˫#c
    Λܭࢉ ࠾༻͠ͳ͔ͬͨཧ༝ จࣈॱংΛࣺͯΔͨΊʮ߈ܸྗ্ঢʯͱʮ্ঢ߈ܸྗʯΛ۠ผͰ͖ͳ͍ ΞϧΰϦζϜ %BNFSBV-FWFOTIUFJO ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ -FWFOTIUFJO
    ͱ΄΅ಉ౳
    Θ͔ͣʹ஗͍ ɻ-FWFOTIUFJO
    
    ྡ઀
    
    จࣈͷೖΕସ͑Λίετ
    
    Ͱڐ༰ ࠾༻͠ͳ͔ͬͨཧ༝ 0$3
    Ͱ͸
    USBOTQPTJUJPO
    ΑΓ७ਮͳஔ׵ޡΓ͕େ൒ͰɺԸܙ͕ബ͍ ΞϧΰϦζϜ +BSP8JOLMFS ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ -FWFOTIUFJO
    ΑΓఆ਺͕খ͍͞ɻҰகจࣈ਺
    
    USBOTQPTJUJPO
    
    ڞ௨
    QSF fi Y
    Ճ఺Ͱྨࣅ౓Λग़͢ ࠾༻͠ͳ͔ͬͨཧ༝ ୹͍ਓ໊ɾॅॴ޲͚ʹ࠷దԽ͞Εͨؔ਺Ͱɺʙ
    จࣈͷ
    FGGFDU
    จͰ͸
    -FWFOTIUFJO
    ͱͷ͕ࠩग़ʹ͍͘ ͦͷଞͷর߹ΞϧΰϦζϜ
27. ͦͷଞͷর߹ΞϧΰϦζϜ ΞϧΰϦζϜ 4ZN4QFMM ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ ࣄલܭࢉͰ࣮࣭
    0 
    MPPLVQɻNBTUFS
    ΛʮFEJU
    ≤
    L
    ͷશมܗʯʹల։ͨࣙ͠ॻΛ
    QSFCVJME
    ͠ɺೖྗ΋ల։ͯ͠
    IBTI
    িಥΛݕग़ ࠾༻͠ͳ͔ͬͨཧ༝ ڑ཭
    ≤
    
    ·Ͱ͔͠Ҿ͚ͣ௕͞มಈʹऑ͍ɻࣙॻల։Ͱ
    0 -
    ͷϝϞϦ๲ு͕͋Γɺ
    ݅ن໛Ͱ͸Ըܙ͕๡͍͠ ΞϧΰϦζϜ #,USFF ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ ฏۉ
    0 MPH
    /
    ఔ౓ͷۙ๣୳ࡧɻจࣈྻۭؒʹ
    NFUSJD
    USFF
    Λߏங͠ɺڑ཭
    E
    Ҏ಺ͷ΋ͷΛ໦ͰߜΔ ࠾༻͠ͳ͔ͬͨཧ༝ ಺෦Ͱ݁ہ
    -FWFOTIUFJO
    ΛݺͿɻ
    ݅ن໛Ͱ͸ΠϯσοΫεߏஙίετ͕ԸܙΛ্ճΔ ΞϧΰϦζϜ 4PVOEFY
    
    .FUBQIPOF
    Իӆ
    IBTI ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ ଎͍
    ఆ਺࣌ؒ ɻൃԻྨࣅੑͰಉ஋ΫϥεΛ࡞ΓɺϋογϡҰகͰൺֱ ࠾༻͠ͳ͔ͬͨཧ༝ ӳޠԻӆ޲͚ͷࢉ๏Ͱ͋Γɺ೔ຊޠ
    
    $+,
    ʹ͸ద༻Ͱ͖ͳ͍ ΞϧΰϦζϜ จ຺ϞσϧຒΊࠐΈڑ཭
    #&35
    ౳ ܭࢉྔ
    
    ଎౓ͱ࢓૊Έ େ෯ʹ஗͍
    ਺े
    NT
    
    ΫΤϦ ɻจࣈྻΛߴ࣍ݩϕΫτϧʹຒΊࠐΈɺDPTJOF
    ڑ཭Ͱྨࣅ౓Λܭࢉ ࠾༻͠ͳ͔ͬͨཧ༝ ϦΞϧλΠϜಈըʹ͸ॏ͗͢ΔɻϞσϧ͕
    NBTUFS
    ͷ
    EPNBJO
    ޠኮɺಛʹήʔϜ଄ޠΛ஌Βͳ͍
28. ೖྗσʔλΛΩϨΠʹ͢Δࡉ͔͍޻෉ r Χˠྗ̍ͭΧλΧφͷΧΛ׽ࣈͷྗ ͔ͪΒ ʹஔ͖׵͑Δ
    ‣ ߈ܸʮྗʯͳͲܾ·ͬͨύλʔϯʹ͍ͭͯ
    r શ֯ɾ൒֯Λଗ͑Δ
    r ۭനΛআڈ͢Δ Ϛονϯάͷલʹਖ਼نԽͯ͠ϊΠζΛআڈ͢Δ
29. ೝࣝ଎౓Λ޲্ͤ͞Δࡉ͔͍޻෉ 0xD5D5EBF7EAD5D5EB ݩը૾ 9×8 grayscale 8×8 bit pattern 64-bit hash E)BTIΛϋϛϯάڑ཭Ͱൺֱͯ͠ྨࣅͷϑϨʔϜΛແࢹ͢Δ
31. ·ͱΊ r Ϛελʔσʔλʢਖ਼ղͷఆٛʣΛ੔͑Δ
    r ϑϨʔϜ୯ҐͰޡೝࣝΛϑΟϧλʔ͢Δ
    ‣ .VMUJGSBNF
    $POTFOTVT
    r ࣄલʹೖྗΛΫϦʔϯʹ͢Δ
    ‣ ςΩετͷਖ਼نԽ
    ‣ Α͋͘ΔޡೝࣝΛஔ׵
    r ڍಈΛܾఆ࿦తɾ؍ଌՄೳʹ͢Δ ߴ͍඼࣭Ͱ࠶ݱੑͷ͋Δ݁ՌΛग़ྗ͢ΔͨΊͷ޻෉
32. 3FTPVSDFT r IUUQTHJUIVCDPNLJTIJLBXBLBUTVNJ$BNFSB0$3
    r IUUQTHJUIVCDPNLJTIJLBXBLBUTVNJ3FMJD'PSHF
    r IUUQTBQQTBQQMFDPNVTBQQSFMJDGPSHFJE
    r IUUQTSFMJDGPSHFQBHFTEFW