Similarity metrics for Japanese kanji

Transcript

1. Similarity Metrics for Japanese Kanji Lars Yencken / 99designs Maths

   and Science Meetup, 30th Nov 2012

2. Linguistics Computer Science Computational Linguistics

3. Relative difficulty of languages

4. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

   STATE

5. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

   STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish

6. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

   STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

7. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

   STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

8. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

   STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

9. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

   STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

10. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

    STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

11. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

    STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

12. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

    STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

13. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

    STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu Exceptionally difficult for native English speakers ‭‭‭‭‭‭ 2200 class hours Arabic, Cantonese, Mandarin, Japanese, Korean

14. DIFFICULTY OF LEARNING LANGUAGES FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF

    STATE Closely related to English ‭‱575-600 class hours Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese, Romanian, Spanish, Swedish Significant linguistic and/or cultural differences ‭‭‭ 1100 class hours Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian, Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew, Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian, Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik), Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai, Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu Exceptionally difficult for native English speakers ‭‭‭‭‭‭ 2200 class hours Arabic, Cantonese, Mandarin, Japanese, Korean

15. 持

16. 持 /mo(tsu)/ "to carry"

17. 持 挂 拝

18. distance(持, 挂) = ???

19. The space of kanji

20. None

21. dog dough log

22. 持 挂 拝 土

23. Approaches

24. Compare images

25. 持挂

26. None

27. Compare components

28. 
     㣡
    ౔
    ੇ ኱
    ౔
    ੇ

29. Compare strokes

30. P R O S P E R I T Y

    P R O P E R T I E S

31. P R O S P E R I T Y

    P R O P E R T I E S distance: 6

32. 
    3, 11a, 2a, 2a 3, 11a, 2a, 2a,

    2a distance: 1

33. Compare trees

34.         
    

            
    

35.         
    

            
     tree edit distance

36. So what works?

37. None
38. None

39. Thanks!