Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Similarity metrics for Japanese kanji

Lars Yencken
November 30, 2012

Similarity metrics for Japanese kanji

An overview of my PhD work on calculating the similarity between Japanese kanji, presented to the Melbourne Maths and Science meetup.

Lars Yencken

November 30, 2012
Tweet

More Decks by Lars Yencken

Other Decks in Science

Transcript

  1. Similarity Metrics for
    Japanese Kanji
    Lars Yencken / 99designs
    Maths and Science Meetup, 30th Nov 2012

    View Slide

  2. Linguistics
    Computer
    Science
    Computational Linguistics

    View Slide

  3. Relative difficulty
    of languages

    View Slide

  4. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE

    View Slide

  5. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish

    View Slide

  6. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  7. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  8. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  9. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  10. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  11. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  12. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu

    View Slide

  13. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
    Exceptionally difficult for
    native English speakers
    ‭‭‭‭‭‭ 2200 class hours
    Arabic, Cantonese, Mandarin, Japanese, Korean

    View Slide

  14. DIFFICULTY OF LEARNING LANGUAGES
    FOREIGN SERVICE INSTITUTE, US DEPARTMENT OF STATE
    Closely related to English
    ‭‱575-600 class hours
    Afrikaans, Danish, Dutch, French, Italian, Norwegian, Portuguese,
    Romanian, Spanish, Swedish
    Significant linguistic and/or
    cultural differences
    ‭‭‭ 1100 class hours
    Albanian, Amharic, Armenian, Azerbaijani, Bengali, Bosnian, Bulgarian,
    Burmese, Croatian, Czech, Estonian, Finish, Georgian, Greek, Hebrew,
    Hindi, Hungarian, Icelandic, Khmer, Lao, Latvian, Lithuanian,
    Macedonian, Mongolian, Nepali, Pashto, Persian (Dari, Farsi, Tajik),
    Polish, Russian, Serbian, Sinhalese, Slovak, Slovenian, Tagalog, Thai,
    Turkish, Ukranian, Urdu, Uzbek, Vietnamese, Xhosa, Zulu
    Exceptionally difficult for
    native English speakers
    ‭‭‭‭‭‭ 2200 class hours
    Arabic, Cantonese, Mandarin, Japanese, Korean

    View Slide


  15. View Slide


  16. /mo(tsu)/ "to carry"

    View Slide

  17. 持 挂

    View Slide

  18. distance(持, 挂) = ???

    View Slide

  19. The space of kanji

    View Slide

  20. View Slide

  21. dog
    dough
    log

    View Slide





  22. View Slide

  23. Approaches

    View Slide

  24. Compare images

    View Slide

  25. 持挂

    View Slide

  26. View Slide

  27. Compare components

    View Slide



  28. 㣡 ౔ ੇ
    ኱ ౔ ੇ

    View Slide

  29. Compare strokes

    View Slide

  30. P R O S P E R I T Y
    P R O P E R T I E S

    View Slide

  31. P R O S P E R I T Y
    P R O P E R T I E S
    distance: 6

    View Slide



  32. 3, 11a, 2a, 2a
    3, 11a, 2a, 2a, 2a
    distance: 1

    View Slide

  33. Compare trees

    View Slide














  34. View Slide














  35. tree edit distance

    View Slide

  36. So what works?

    View Slide

  37. View Slide

  38. View Slide

  39. Thanks!

    View Slide