Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Missspell Detection
Search
bk
February 10, 2020
Science
180
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Missspell Detection
bk
February 10, 2020
Transcript
44 slides
ฤूڑʹΑΔจࣈྻޡදهݕ ϨʔϕϯγϡλΠϯڑͱδϟϩɾΟϯΫϥʔڑ
࣍ 1. ՝……………………………………p.3-10 2. ࡞ͬͨͷ……………………………p.11-16 3. ฤूڑ………………………………p.17-39 4. ݁Ռ……………………………………p.40-41 5.·ͱΊ…………………………………p.42
6.ࢀߟจݙ………………………………p.43
՝
՝ ϒϥϯυͷࡏݿ
՝ flea ख࡞ۀͰग़
՝ flea
՝ GUCCI Tote Bag Black Leather flea ग़লྗԽ
՝ GUCCHI Tote Bag Black Leather flea
՝ GUCCHI Tote Bag Black Leather flea • ग़औΓফ͠ •
ग़ऀධՁԼ • ΞΧϯτఀࢭ ϒϥϯυ໊ޡදهͷ ϖφϧςΟ
՝ AIͰͳΜͱ͔ͯ͠ Python ࣗવݴޠॲཧ
࡞ͬͨͷ
࡞ͬͨͷ ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ
ɾɾɾ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ࡞ͬͨͷ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ਖ਼ϒϥϯυ໊Ϧετ GUCCI VUITTON ɾɾɾ ɾɾɾ ɾɾɾ ࡞ͬͨͷ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ਖ਼ϒϥϯυ໊Ϧετ GUCCI VUITTON ɾɾɾ ɾɾɾ ɾɾɾ ૯ͨΓ ࣅͨ୯ޠΛग़ྗ ࡞ͬͨͷ
ग़λΠτϧϦετ GUCCHI Tote Bag Black Leather ɾɾɾ ɾɾɾ ɾɾɾ ɾɾɾ
୯ޠʹղ ग़୯ޠϦετ GUCCHI Tote Bag Black Leather ਖ਼ϒϥϯυ໊Ϧετ GUCCI VUITTON ɾɾɾ ɾɾɾ ɾɾɾ ૯ͨΓ ࣅͨ୯ޠΛग़ྗ ࡞ͬͨͷ
ฤूڑ
ฤूڑ 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance) 2. δϟϩɾΟϯΫϥʔڑ (Jaro-Winkler Distance) GUCCHI
GUCCI
ฤूڑ 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance) 2. δϟϩɾΟϯΫϥʔڑ (Jaro-Winkler Distance) GUCCHI
GUCCI 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance)
ฤूڑʢϨʔϕϯγϡλΠϯڑʣ ͋Δจࣈྻ ൺֱ͢Δจࣈྻ จࣈΛૢ࡞ͯ͠Ұகͤ͞Δ
͋Δจࣈྻ ൺֱ͢Δจࣈྻ จࣈΛૢ࡞ͯ͠Ұகͤ͞Δ ૢ࡞ ஔ আ ૠೖ ૢ࡞ճ=ڑ ฤूڑʢϨʔϕϯγϡλΠϯڑʣ
ஔ ݩͷจࣈྻ G U T T I ൺֱ͢Δจࣈྻ G U
C C I ஔ ૢ࡞ճ = ڑ = 2 ฤूڑʢϨʔϕϯγϡλΠϯڑʣ
ஔ আ ૠೖ GUTTI GUCCI GUCCHI GUCCI GUCI GUCCI ฤूճʢڑʣ
2 1 1 ݩͷจࣈྻ ൺֱ͢Δจࣈྻ ฤूํ๏ ฤूڑʢϨʔϕϯγϡλΠϯڑʣ
ฤूڑ 1. ϨʔϕϯγϡλΠϯڑ (Levenshtein Distance) 2. δϟϩɾΟϯΫϥʔڑ (Jaro-Winkler Distance) GUCCHI
GUCCI
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ δϟϩڑɿ จࣈྻͷ෦తͳҰக߹͍ΛଌΔ ͕େ͖͍ํ͕ڑ͕͍ۙ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) m m m m s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ δϟϩڑɿ จࣈྻͷ෦తͳҰக߹͍ΛଌΔ ͕େ͖͍ํ͕ڑ͕͍ۙ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
mɿ۠ؒͷҰகจࣈ max(|s1 |, |s2 |) 2 − 1 ݩͷจࣈྻɿGCCUHI →
6 ൺֱ͢ΔจࣈྻɿGUCCI → 5 max(6,5) 2 − 1 = 2 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
mɿ۠ؒͷҰகจࣈ ݩͷจࣈྻ G C C U H I ൺֱ͢Δจࣈྻ G
U C C I ۠ؒͰҰகจࣈΛݕࡧ Ұகจࣈ͕͋ΕΧϯτ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
mɿ۠ؒͷҰகจࣈ ݩͷจࣈྻ G C C U H I ൺֱ͢Δจࣈྻ G
U C C I m = 5 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) t s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ จࣈྻͷ෦తͳҰக߹͍ΛଌΔ ͕େ͖͍ํ͕ڑ͕͍ۙ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
tɿҰகจࣈͷஔ ݩͷจࣈྻ G C C U H I ൺֱ͢Δจࣈྻ G
U C C I Ұகͨ͠จࣈΛநग़ ݩͷจࣈྻ G C C U I ൺֱ͢Δจࣈྻ G U C C I ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
tɿҰகจࣈͷஔ ݩͷจࣈྻ G C C U I ൺֱ͢Δจࣈྻ G U
C C I t = 2 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ ಉҰͷจࣈྻʹ͢ΔҝʹԿจࣈஔ͢Δͷ͔
Dj = 1 3 * ( m |s1 | +
m |s2 | + m − t 2 m ) s1, s2 ɿจࣈྻͷ͞ mɿ۠ؒͷҰகจࣈ tɿҰகจࣈͷஔ = 1 3 * ( 5 6 + 5 5 + 5 − 2 2 5 ) = 79 90 = 0.8777... ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Djw = Dj + l * 1 10 * (1
− Dj ) Dj ɿJaro Distance lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ δϟϩɾΟϯΫϥʔڑɿ ઌ಄จࣈͷҰகॏΈΛ͚ͭͯධՁ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Djw = Dj + l * 1 10 * (1
− Dj ) Dj ɿJaro Distance lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ l δϟϩɾΟϯΫϥʔڑɿ ઌ಄จࣈͷҰகॏΈΛ͚ͭͯධՁ ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ ݩͷจࣈྻ G C C U H I
ൺֱ͢Δจࣈྻ G U C C I l = 1 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
Djw = Dj + l * 1 10 * (1
− Dj ) Dj ɿJaro Distance lɿઌ಄͔ΒͷҰகจࣈʢl <= 4ʣ = 79 90 + 1 * 1 10 * (1 − 79 90 ) = 801 900 = 0.89 ฤूڑʢδϟϩɾΟϯΫϥʔڑʣ
* https://github.com/ztane/python-Levenshtein/ **https://github.com/nap/jaro-winkler-distance Levenshteinɿখ͍͞΄Ͳ͍ۙ Jaro-Winklerɿେ͖͍΄Ͳ͍ۙ ݩͷจࣈྻ ൺֱ͢Δ จࣈྻ *Levenshtein **Jaro-Winkler
GUCCHI GUCCI 1 0.97 GUTTI 2 0.79 GCCUHI 3 0.89 άον༟ࡾ 5 0.00 ฤूڑ
ݩͷจࣈྻ ൺֱ͢Δ จࣈྻ *Levenshtein **Jaro-Winkler GUCCHI GUCCI 1 0.97 GUTTI
2 0.79 GCCUHI 3 0.89 άον༟ࡾ 5 0.00 Jaro-WinklerҰக͢Δจࣈ͕ ଘࡏ͍ͯ͠Δ͜ͱΛධՁ͍ͯ͠Δɻ LevenshteinͱJaro-WinklerͰ ۙ͞ͷॱং͕ҟͳΔɻ ฤूڑ * https://github.com/ztane/python-Levenshtein/ **https://github.com/nap/jaro-winkler-distance
݁Ռ
.py ͳΜ͔ಈ͍ͯΔ͔Βྑ͠ ݁Ռ * https://github.com/bk-18/Misspelled-Brand-Name-Detector
·ͱΊ • ग़࣌ͷϒϥϯυ໊ޡදهͱ͍͏՝ • Ϧετ૯ͨΓʹΑΔޡදهݕ • ϨʔϕϯγϡλΠϯڑ • δϟϩɾΟϯΫϥʔڑ
ࢀߟจݙ • ̎ͭͷจࣈྻͷྨࣅΛԽɹϨʔϕϯγϡλΠϯڑͱδϟϩɾΟ ϯΫϥʔڑͷղઆ, ਓೳͰ͋ͦͿ, http://nkdkccmbr.hateblo.jp/entry/ 2016/08/18/102727 • ฤूڑ (Levenshtein
Distance), naoyaͷͯͳμΠΞϦʔ, https:// naoya-2.hatenadiary.org/entry/20090329/1238307757 • จࣈྻྨࣅධՁ ϨʔϕϯγϡλΠϯڑ / δϟϩɾΟϯΫϥʔڑ, ਓೳͯ͠ΈΔ, http://grahamian.hatenablog.com/entry/word_similarity • Yaoshu Wang(B) , Jianbin Qin, and Wei Wang,: Efficient Approximate Entity Matching Using Jaro-Winkler Distance, Univeristy of New South Wales, http://qinjianbin.com/files/wise2017-wang.pdf
ENJOY! ENJAY! EMJOY! ENJOI! ENZYOI!
More Decks by bk
See All by bk
Befriending Kurtosis with R
bk_18
1
1k
tidy_rpart
bk_18
1
1.8k
dotdotdot_in_predict_function
bk_18
1
1.2k
Introduction_of_GoogleAnalytics_with_R
bk_18
2
1k
web scraping with polite package
bk_18
2
860
start-salesforce-with-r
bk_18
0
940
About Missing Values
bk_18
1
430
Other Decks in Science
See All in Science
現代AI入門: 大規模言語モデルから思考するシステムへ — Test-Time Scaling, Loop, Harness, RSI
stillpedant
PRO
49
25k
JSAI2026企画セッションKS-14 インタビュー集『⼈⼯知能と哲学と四つの問い』が提起する⼈⼯知能のこれからの課題 趣旨説明 / JSAI2026 Special Session: A Collection of Interviews, “Artificial Intelligence, Philosophy, and Four Questions”
ykiyota
0
500
JPA2026_NetworkSymposium_Overview
junkashihara
0
130
Toward Causal Scientific Discovery with AI
sshimizu2006
0
250
不動産業界における業界特化のデータ整備とAI活用 ─Vertical DataとVertical AI─
estie
1
1.2k
Visual Linear Algebra - Lecture at Shosen Grande
hiranabe
0
730
HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps
tomoaki0705
0
180
知能とはなにか -ヒトとAIのあいだ-
tagtag
PRO
0
270
Wet Active Matter
rajeshrinet
0
280
Засідання студентського наукового гуртка «Садово-паркове господарство»
mariiabarkan12
0
100
AWS WAFのAIトラフィック収益化機能を考えてみる_20260626
masakiokuda
1
140
勾配流から乖離した学習ダイナミクス
hanbao
1
1.2k
Featured
See All Featured
Effective software design: The role of men in debugging patriarchy in IT @ Voxxed Days AMS
baasie
1
610
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
35
2.9k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
47
8.4k
Game over? The fight for quality and originality in the time of robots
wayneb77
1
330
WCS-LA-2024
lcolladotor
0
1k
Writing Fast Ruby
sferik
630
63k
Why Our Code Smells
bkeepers
PRO
340
58k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
32
3.1k
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
marketingsoph
0
470
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
659
62k
The Cult of Friendly URLs
andyhume
79
7.1k