e-book title normalization

at YAPC::Hakodate 2024

Transcript

1. $ whoami • @nasa9084 • Hakodate - Sapporo - Asaka - Sapporo • Go/Kubernetes • builderscon 2024 Organizer • Kubernetes SIG Docs Localization Subproject (Japanese)
2. Context:
7. Can't we share a single account? • ALL contents will be shared • Sometimes I don't want to share some • Reading state will be shared • Everyone can buy any on Amazon
8. MEH.
10. Amazon Kids • can create multiple pro fi les • can share contents YOU WANT TO for each pro fi le • each pro fi le can have each reading state
12. Meh.
13. "
    '&8
    .*/65&4
    -"5&3
15. e-book title Normalization @nasa9084 2024.10.06 YAPC::Hakodate
16. ະདྷ೔ه(1) (֯઒ίϛοΫεɾΤʔε)
17. ະདྷ೔ه(1) (֯઒ίϛοΫεɾΤʔε) γϦʔζ ר਺ Ϩʔϕϧ
18. .+\(\d+\) \(.*コミックス?.*\) γϦʔζ ר਺ Ϩʔϕϧ
19. ໊୳ఁίφϯʢ̔̑ʣ (গ೥αϯσʔίϛοΫε)
20. ໊୳ఁίφϯʢ̔̑ʣ (গ೥αϯσʔίϛοΫε) શ֯
21. ໊୳ఁίφϯʢ̔̑ʣ (গ೥αϯσʔίϛοΫε) ൒֯ ൒֯
22. .+[(（][0-9０-９]+[)）] \(.*コミックス?.*\)
23. ϘϘϘʔϘɾϘʔϘϘ 8 (δϟϯϓίϛοΫεDIGITAL)
24. .+[(（]?[0-9０-９]+[)）]? \(.*コミックス?.*\)
25. ̼̾̽̾ɹ̎ר (·Μ͕λΠϜKRίϛοΫε)
26. .+[(（]?[0-9０-９]+巻?[)）]? \(.*コミックス?.*\)
27. ̼̾̽̾ɹ̍רʲAmazon.co.jpݶఆඳ͖ԼΖ͠ಛయ෇͖ʳ (·Μ͕λΠϜKRίϛοΫε)
28. .+[(（]?[0-9０-９]+巻?[)）]?(【Amazon\.co\.jp限定描 き下ろし特典付き】)? \(.*コミックス?.*\)
29. ΞϦεͱଂ࿡(6)ʲಛయϖʔύʔ෇͖ʳ (RYU COMICS) ΞϦεͱଂ࿡(9)ʲిࢠݶఆಛయϖʔύʔ෇͖ʳ (RYU COMICS)
30. .+[(（]?[0-9０-９]+巻?[)）]?(【(Amazon\.co\.jp|電子)? 限定描き下ろし)?特典(ペーパー)?付き】)? \(.*コミック ス?.*\)
31. Q: Can't we use "any" for the special o ff ers like ʲ.+ʳ?
32. A: NO. e.g. ʲਪ͠ͷࢠʳ
33. ͓͔͠ͳసੜXVI ӄ๳͸ࠇͯ͘΄Ζۤ͘ʲిࢠॻ੶ݶఆॻ͖ԼΖ͠SS෇͖ʳ (TOϒοΫεϥϊϕ) ר਺͕ϩʔϚ਺ࣈ ר਺ͷޙʹ෭୊ ৽͍͠ಛయͷύλʔϯ
34. .+[(（]?[0-9０-９IVX]+巻?[)）]?( [^ 【]+)?(【(Amazon\.co\.jp|電子(書籍)?)?限定[書描]き下 ろし)?(SS|特典(ペーパー)?)付き】)? \(.*コミック ス?.*\)
35. ͓લɺλψΩʹͳΒͶʔ͔ʁ: 6ʲΠϥετಛయ෇ʳ (comic POOL) ίϩϯ༗Γ ৽ಛయ ӳޠͰcomic
36. MONOQLO (ϞϊΫϩ) 2024೥ 02݄߸ [ࡶࢽ]
37. .+[(（]?([0-9０-９IVX]+巻?|\d{4}年 ?\d{1,2}月 号)[)）]?( [^【]+)?(【(Amazon\.co\.jp|電子(書籍)?)?(限 定)?([書描]き下ろし)?(SS|(イラスト)?特典(ペーパー)?) 付き】)? \(.*(コミックス?|comics?).*\)\[雑誌\]
38. GetNavi 2024೥3.5݄߸[ࡶࢽ]
39. ΫϏΩϦαΠΫϧ ੨৭αϰΝϯͱٔݴݣ͍ ΫϏγϝϩϚϯνετ ਓࣦؒ֨ɾྵ࡚ਓࣝ ΫϏπϦϋΠεΫʔϧ ٔݴݣ͍ͷఋࢠ (ߨஊࣾจݿ) αΠίϩδΧϧʢ্ʣ 㙽௻໦ᆇีͷٔݴࡴ͠ (ߨஊࣾจݿ) αΠίϩδΧϧʢԼʣɹӪ͔ΕऀͷখӖ (ߨஊࣾจݿ) ώτΫΠϚδΧϧɹࡴፋحज़ͷ೏ٶܑຓ (ߨஊࣾจݿ) ωίιΪϥδΧϧʢதʣɹ੺͖੐ࡋ͂ͅɽᒵͳΔछ (ߨஊࣾจݿ) ωίιΪϥδΧϧʢԼʣɹ੨৭αϰΝϯͱٔݴݣ͍ (ߨஊࣾจݿ) ωίιΪϥδΧϧʢ্ʣɹेࡾ֊ஈ (ߨஊࣾจݿ) ΩυφϓΩσΟϯάɹ੨৭αϰΝϯͱٔݴݣ͍ͷ່ (ߨஊࣾϊϕϧε)
40. 5)064"/%4
    
    :&"34
    -"5&3
41. Normalize replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(re placeAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(repl aceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replac eAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(replaceAll(prop("Name"), "̍", "1"), "̎", "2"), "̏", "3"), "̐", "4"), "̑", "5"), "̒", "6"), "̓", "7"), "̔", "8"), "̕", "9"), "̌", "0"), "ᶃ", "1"), "ᶄ", "2"), "ᶅ", "3"), "ᶆ", "4"), "ᶇ", "5"), "ᶈ", "6"), "ᶉ", "7"), "ᶊ", "8"), "ᶋ", "9"), "ᶌ", "10"), "ʢ", "("), "ʣ", ")"), "ɹ", " "), "ୈҰू", "ୈ1ू"), "ୈೋू", "ୈ2ू"), "ୈࡾू", "ୈ3ू"), "ୈ࢛ू", "ୈ4ू"), "ୈޒू", "ୈ5ू"), "ୈ࿡ू", "ୈ6ू"), "ୈࣣू", "ୈ7ू"), "ୈീू", "ୈ8ू"), "ୈ۝ू", "ୈ9ू"), "ୈेू", "ୈ10ू"), "ˌ", "#")
42. Remove special part replaceAll(replaceAll(prop("normalized"), "ʲ(((((ిࢠ(ॻ੶)?|σδλϧ)൛?|Amazon\.co\.jp)(ݶఆ)?)?((((Π ϥετ)?ಛయ)?(ίϥϘ)?(ϖʔύʔ)?)|[ॻ|ඳ]͖ԼΖ͠(SS)?|͓·͚|ΧϥʔϨγϐ)(ಛయ)?෇͖? ?)|(୯࿩|Χϥʔϖʔδ૿ྔ) ൛|ిࢠ(୯ߦຊ|൛)|ແྉ)(ʗڞ௨ϖʔύʔ෇͖)?ʳ|\[(ిࢠॻ੶൛|ࡶࢽ)\]|(௨ৗ|ಛ૷)൛", ""), "^ +| +$", "")
43. Extract Brand match(prop("rmSpecial"), "\(([^()]*(จݿ|৽ॻ|ίϛοΫε?|[Cc]omics?|COMICS?|ϒοΫε|ϊϕϧζ|BOOKS?|ΧυΧ ϫ|֯઒|ߨஊࣾ|ίϧΫ|ΞεΩʔυϫϯΰ|MFC|GANMA!|ΪλʔɾϚΨδϯ)[^()]*)\)")
44. Extract Episode Number match(prop("rmBrand"), "(^\d+ )|(^\d+: ([^ ]+)?)|(^#\d+)|(^\dרʮ[^ʮʯ]+ʯ)|(\([্தԼ]\))|([্தԼ]$)| (ୈ?\d+[רू]:?(( ʮ[^ʯ]+ʯଞ?)|( ʮ[^ʯ]+ ଞʯ)|( ʰ[^ʱ]+ ଞʱ))?)|(\(\d+\) ? ～ .+$|(\(\d+\))|(ʲ\d+ʳ)| (:? ?(Vol\.)?\d+ *$)|(\d{4}೥ ?(\d\.)?\d{1,2}݄߸(\(vol.\d+\))?$)|(ୈ.෦ ?ʮ[^ʮʯ]+[0-9IVX]+ʯ( ?ຊ޷͖ ͷԼႥ্ ୈ[Ұೋࡾ࢛ޒ࿡ࣣീ۝े]෦)?$)|([IVX]+[ ～ ].+$)|( [^ ]+ฤ( \d+)?)$)")
45. Grouping... let(src, prop("rmEpisode"), ifs(test(src, "(ଓɾ)?.෺ޠ(\(.\))?"), "෺ޠγϦʔζ", test(src, "ΫϏΩϦαΠ Ϋϧ|ΫϏγϝϩϚϯνετ|ΫϏπϦϋΠεΫʔϧ|αΠίϩδΧϧ|ώτΫΠϚδΧϧ|ωίιΪϥδΧϧ|ΩυφϓΩσΟϯά"), " ٔݴγϦʔζ", test(src, "^ྵ࡚..ͷਓؒ.+"), "ਓؒγϦʔζ")
47. - fi n-