the world of characters

Transcript

1. 1จࣈͷੈք
   @orisano
   

   View Slide

2. Έͳ͞Μ
   จࣈΛ਺͑ΒΕ·͢ΑͶʁ
   

   View Slide

3. a
   

   View Slide

4. a => 1
   

   View Slide

5. ͋
   

   View Slide

6. ͋ => 1
   

   View Slide

7. 佛
   

   View Slide

8. 佛 => 1
   

   View Slide

9. View Slide

10. => 1
    

    View Slide

11. View Slide

12. => 1
    

    View Slide

13. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔
    ͫ͗
    ͢
    L̠ͨͧͩ͘
    G̴̻͈͍͔̹
    ̑͗̎̅͛
    ́
    Ǫ̵̹̻̝̳
    ͂̌ ̌͘! ͖̬̰̙̗
    ̿̋ ͥ
    ͥ̂ͣ̐́́͜͞
    

    View Slide

14. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔
    ͫ͗
    ͢
    L̠ͨͧͩ͘
    G̴̻͈͍͔̹
    ̑͗̎̅͛
    ́
    Ǫ̵̹̻̝̳
    ͂̌ ̌͘! ͖̬̰̙̗
    ̿̋ ͥ
    ͥ̂ͣ̐́́͜͞ => 6
    

    View Slide

15. Έͳ͞Μ
    όΠτ਺Λ਺͑ΒΕ·͔͢ʁ
    (UTF-8)
    

    View Slide

16. a
    

    View Slide

17. a => 1
    

    View Slide

18. ͋
    

    View Slide

19. ͋ => 3
    

    View Slide

20. 佛
    

    View Slide

21. 佛 => 4
    

    View Slide

22. View Slide

23. => 4
    

    View Slide

24. View Slide

25. => 18
    

    View Slide

26. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔
    ͫ͗
    ͢
    L̠ͨͧͩ͘
    G̴̻͈͍͔̹
    ̑͗̎̅͛
    ́
    Ǫ̵̹̻̝̳
    ͂̌ ̌͘! ͖̬̰̙̗
    ̿̋ ͥ
    ͥ̂ͣ̐́́͜͞
    

    View Slide

27. Z͑ͫ̓ͪ̂ͫ̽ ̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔
    ͫ͗
    ͢
    L̠ͨͧͩ͘
    G̴̻͈͍͔̹
    ̑͗̎̅͛
    ́
    Ǫ̵̹̻̝̳
    ͂̌ ̌͘! ͖̬̰̙̗
    ̿̋ ͥ
    ͥ̂ͣ̐́́͜͞ => 143
    

    View Slide

28. ͋ͳ͕ͨࢥ͏1จࣈ͸
    Ͳ͏਺͑Δ΂͖͔ʁ
    

    View Slide

29. byte਺Ͱ͸਺͑ΒΕͳ͍
    

    View Slide

30. Unicode͸จࣈू߹
    จࣈͱ਺஋͕ରԠ͢Δ
    

    View Slide

31. ͋ => 3042
    

    View Slide

32. => 1F914
    

    View Slide

33. ͜ͷ਺஋ͷ͜ͱΛ
    ίʔυϙΠϯτ
    ͱݺͿ
    

    View Slide

34. ͜ͷίʔυϙΠϯτΛ
    byteྻͰදݱ͢Δํ๏Λ
    ΤϯίʔσΟϯάͱ͍͏
    

    View Slide

35. UTF-8ͱ͔UTF-16ͱ͔͸
    ΤϯίʔσΟϯάͷҰछ
    

    View Slide

36. ͱΓ͋͑ͣ
    ίʔυϙΠϯτΛ਺͑Ε͹
    ղܾʁ
    

    View Slide

37. ͍͍͑
    

    View Slide

38. =>
    1F468 + 200D + 1F469 +
    200D + 1F466
    

    View Slide

39. ࣮͸ෳ਺ͷίʔυϙΠϯτͰ
    ҰͭͷจࣈʹͳͬͨΓ͢Δ
    

    View Slide

40. ਓ͕ؒೝ͍ࣝͯ͠Δ̍จࣈ͸
    ॻهૉ(Grapheme cluster)
    ͱݺ͹Ε͍ͯΔ
    

    View Slide

41. Ͳ͏΍Ε͹
    ίʔυϙΠϯτͷྻ͔Β
    ॻهૉΛऔΓग़ͤΔ͔
    

    View Slide

42. ίʔυϙΠϯτ͕ؒ
    ॻهૉڥքʹͳΔ͔Ͳ͏͔ͷ
    ݫີͳϧʔϧ͕͋Δ
    

    View Slide

43. UAX #29
    Unicode Text Segmentation
    

    View Slide

44. View Slide

45. ͜ΕΛJSͰ࣮૷ͯ͠·ͨ͠
    github.com/orisano/graphemesplit
    

    View Slide

46. ৄ͘͠͸ UAX #29 Λݟͯ
    http://unicode.org/reports/tr29/
    

    View Slide

47. ݟ஌Β͵ਓʹʓจࣈͱ
    ݴΘΕͨͱ͖ʹ͸
    ͪΌΜͱ֬ೝ͠Α͏ʂ
    

    View Slide

48. 1 byte?
    1 codepoint?
    1 grapheme cluster?
    

    View Slide