Similarity metrics for Japanese kanji

An overview of my PhD work on calculating the similarity between Japanese kanji, presented to the Melbourne Maths and Science meetup.

Transcript

1. Similarity Metrics for Japanese Kanji Lars Yencken / 99designs Maths and Science Meetup, 30th Nov 2012
2. Linguistics Computer Science Computational Linguistics
3. Relative difficulty of languages
4. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
5. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish
6. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
7. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
8. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
9. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
10. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
11. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
12. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
13. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu Exceptionally difficult for native English speakers ‭‭‭‭‭‭ 2200 class hours Arabic, Cantonese, Mandarin, Japanese, Korean
14. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu Exceptionally difficult for native English speakers ‭‭‭‭‭‭ 2200 class hours Arabic, Cantonese, Mandarin, Japanese, Korean
15. 持
16. 持 /mo(tsu)/ "to carry"
17. 持 挂 拝
18. distance(持, 挂) = ???
19. The space of kanji
21. dog dough log
22. 持 挂 拝 土
23. Approaches
24. Compare images
25. 持挂
27. Compare components
28. 
     㣡
    ౔
    ੇ ኱
    ౔
    ੇ
29. Compare strokes
30. P R O S P E R I T Y P R O P E R T I E S
31. P R O S P E R I T Y P R O P E R T I E S distance: 6
32. 
    3, 11a, 2a, 2a 3, 11a, 2a, 2a, 2a distance: 1
33. Compare trees
34.         
            
    
35.         
            
     tree edit distance
36. So what works?
39. Thanks!