Upgrade to Pro — share decks privately, control downloads, hide ads and more …

future-of-i18n

kazupon
September 07, 2018

 future-of-i18n

kazupon

September 07, 2018
Tweet

More Decks by kazupon

Other Decks in Technology

Transcript

  1. The future of i18n
    (ϘΫ͕ߟ͑Δi18nͷະདྷ)
    2018.09.07
    builderscon tokyo 2018
    @kazupon

    View Slide

  2. Who are You?

    View Slide

  3. kazupon
    CTO
    Vue.js Core Team
    Vue.js japan user group organizer
    @kazu_pon kazupon

    View Slide

  4. View Slide

  5. Ticket sales, Coming soon ...

    View Slide

  6. Introduction

    View Slide

  7. Background
    • I've experienced a lot of
    different i18n what
    happened.
    • String encoding
    • Multi Languages
    • and more ….
    • I've developed, maintained
    as OSS developer.
    App
    or
    Service

    vue-i18n

    View Slide

  8. Come into
    one's mind for
    the future of
    i18n ...


    Welcome Feedback!

    View Slide

  9. Today about talking

    View Slide

  10. Agenda
    • The first half
    • About i18n
    • Current of i18n
    • Problems of i18n
    • The second half
    • Next generation i18n

    View Slide

  11. Today about not to talk
    • Design and Process of
    software development
    for globalization
    • Process of localization
    • See the book!
    https://book.impress.co.jp/books/1117101057

    View Slide

  12. About i18n

    View Slide

  13. i18n
    l10n

    View Slide

  14. What is i18n?
    • Internationalization






    • Generalize products in the format that doesn't
    depend on specific language, region and
    culture
    i nternationalizatio n
    18 letters
    i18n

    View Slide

  15. What is l10n?
    • Localization






    • Specialize products in the format that depend
    on specific language, region and culture
    l ocalizatio n
    10 letters
    l10n

    View Slide

  16. i18n <-> l10n
    i18n + l10n = g11n (globalization)
    11 letters
    Base
    ja
    en fr
    l10n
    i18n
    g11n

    View Slide

  17. FYI: Numeronym
    • A numeronym is a number-based word.
    l10n
    i18n g11n
    https://en.wikipedia.org/wiki/Numeronym
    World Wide Web Consortium
    w3c
    a11y k8s d11n

    View Slide

  18. Joke: How to call i18n?
    i18n (͍͍΍Μ: iiyann)
    https://twitter.com/Konboi/status/1031431327692681216

    View Slide

  19. By the way,
    Do you support i18n
    at your app ?

    View Slide

  20. Are you Serious?

    View Slide

  21. Importance of i18n
    • hard to support software maintainance after
    release …
    • Leak easily, when software has became
    large scale.
    • Need to re-design the UI layouts.
    • Database, and other systems are also
    affected …

    View Slide

  22. • And also hard to localization …
    • The translator fix the source code.

    -> Really? It’s impossible …
    • Even if i18n had been fixed with translator,
    there is the possibility of breaking app ...
    Importance of i18n

    Breaking …

    View Slide

  23. If cannot support i18n…
    • Can not keep usability of product …


    What’s mean?

    View Slide

  24. NG Case
    https://twitter.com/frsyuki/status/1030189494371115008

    View Slide

  25. Current of i18n

    View Slide

  26. Platform
    • Built-in i18n and
    provide for
    application
    • Windows / Mac
    OS / Linux (*nix)
    • iOS / Android
    • Web

    View Slide

  27. Development Environment
    • Built-in i18n to Programming languages
    • ex: datetime, string encoding
    • Built-in i18n to Application Framework
    • ex: Ruby on Rails

    View Slide

  28. Contents of i18n

    View Slide

  29. Display of resource

    View Slide

  30. App
    Resource interpolation
    i18n lib
    Resource
    Catalog
    configure
    en_US.json
    ja_JP.json
    read
    select
    interpolate

    View Slide

  31. Format of Resource Catalog
    • .pot, .po (gettext)
    • JSON
    • YAML
    • .strings (macOS / iOS)
    • XML base (Android, Java, .NET)
    • And more …

    View Slide

  32. Resource Catalog Structure
    • Flat key pairs
    • Hierarchical

    View Slide

  33. Message Formatting
    • ICU (International Components for Unicode)
    Message Format

    http://userguide.icu-project.org/formatparse/messages

    View Slide

  34. ICU
    • Plural

    View Slide

  35. ICU
    • Date & Time

    View Slide

  36. ICU
    • Number

    View Slide

  37. DateTime

    View Slide

  38. Built-in programming languages
    • Support in most programming languages
    • e.g. Ruby
    • DateTime class
    • Time class
    • Date class

    View Slide

  39. Character Code

    View Slide

  40. Character encoding
    • Unicode have almost becoming defaults.

    https://unicode.org/
    • Web:

    ex, html


    • programming languages:

    ex, python (v3)

    View Slide

  41. Collation
    • Database

    e.g. MySQL

    View Slide

  42. Collation
    • ECMAScript i18n API

    View Slide

  43. Emoji

    View Slide

  44. Skin tone
    http://std.dkuug.dk/JTC1/SC2/WG2/docs/n4599.pdf
    Reference from: “WG2 N4599 Skin tone modifier symbols” Unicode Consortium,

    View Slide

  45. Font

    View Slide

  46. Web Fonts

    View Slide

  47. Google Fonts
    https://fonts.google.com/

    View Slide

  48. Standardization

    View Slide

  49. It’s almost standardized
    • character encoding: ISO/IEC 10646
    • datetime: ISO8601
    • language tags: BCP 47 (IETF)
    • countries/states/subdivisions : ISO3166
    • phone number: E.164
    • currency: ISO4217
    • and more …

    View Slide

  50. Libraries

    View Slide

  51. Enhanced with external libs
    • Phone Number

    libphonenumber (C/C++): 

    https://github.com/googlei18n/libphonenumber
    • Countries

    i18n-iso-countries(Node.js): 

    https://github.com/michaelwittig/node-i18n-iso-countries
    • DateTime

    spacetime (JavaScript):

    https://github.com/spencermountain/spacetime
    • and more …

    View Slide

  52. UI libraries

    View Slide

  53. Date Picker

    View Slide

  54. Calendar

    View Slide

  55. Emoji Picker

    View Slide

  56. g11n is easy to start!
    due to i18n fruitful
    environments.

    View Slide

  57. However …

    View Slide

  58. https://qiita.com/yugui/items/55f2529c5a731badeff7

    View Slide

  59. Hard to perfect …

    View Slide

  60. Problems of i18n

    View Slide

  61. Languages
    • One country have multi languages (e.g. india)
    • Dialect (e.g. bt-pt)
    • Middle name (e.g. es)
    • Honorifix
    • and more …

    View Slide

  62. Feature of product
    • Law
    • Tax (payment, order)
    • Accounts (tax, audit)
    • GDPR
    • Culture
    • Foods (tastes, … etc)
    • and more …

    View Slide

  63. UI
    • LTR (most)
    • RTL (arabia, …etc)
    • Vertical line (japanese, … etc)

    View Slide

  64. Database
    • Collation
    • Attribute of records dependent on country or
    culture
    • e.g. Middle name
    • Contents management for languages
    • and more …

    View Slide

  65. • I/F Miss-match of i18n per library
    • Resource catalog management
    Server
    Client / Server
    Client
    i18n lib
    i18n lib
    e.g. API or BFF model
    Resource catalog
    (errors, etc)
    Resource catalog
    (client specific)
    Client
    i18n lib
    Resource catalog
    (client specific)

    View Slide

  66. Client / Server
    • UI placeholder and contents management
    https://developer.kaizenplatform.com/entry/axross/2018-07-02
    Reference: kaizen platform engineer blog

    View Slide

  67. Edge case of project
    • Grouping of Prefectures and Countries

    (no standardization)

    View Slide

  68. Edge case of project
    • Convert csv to shift_jis from utf-8
    • e.g. Excel

    View Slide

  69. Edge case of project
    • Convert to ISO8601 from un-standardized
    datetime string

    View Slide

  70. i18n fatigue …

    View Slide

  71. By the way,
    What state of l10n?

    View Slide

  72. Translation
    • Translation have improved dramatically with
    machine learning!
    https://arxiv.org/pdf/1609.08144.pdf
    Reference:

    Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

    View Slide

  73. Translation process
    • Out-sourcing model
    • Crowd-sourcing model
    Translator
    Software
    Company
    Translation
    Company
    Software
    Company
    Translator
    Software
    Company
    Translator
    Software
    Company
    Translator
    Translator …
    out-sourcing crowd-sourcing

    Translator

    View Slide

  74. i18n-able content service
    https://www.contentful.com/r/knowledgebase/localization/

    View Slide

  75. Support i18n integration
    • Import / Export resource catalog
    Localization Services
    .po
    .json
    .resx

    Import
    Export

    View Slide

  76. Support i18n integration
    • API
    https://medium.com/@jamuhl/vue-js-from-internationalization-i18n-to-localization-l10n-and-back-again-c3e5f7cc5e71
    Reference: vue.js: from internationalization (i18n) to localization (l10n) and back again

    View Slide

  77. We need the
    NEW i18n
    environments !


    View Slide

  78. Q & A

    View Slide

  79. Next
    generation
    i18n

    View Slide

  80. What kind of i18n do
    we want?

    View Slide

  81. I thought in App dev position
    • I don't want to worry about key naming of
    resource catalog …
    • I don't want to worry about resource catalog
    management …
    • I don't want to effort pseudo localization test …
    • I don't want to worry about choice of i18n
    library …

    View Slide

  82. I want to focus on
    App development !!

    View Slide

  83. Next generation i18n

    View Slide

  84. ✨DX✨
    (Developer Experience)

    View Slide

  85. Need to provide DX
    to developer!

    View Slide

  86. However,
    How can do you
    provide DX?

    View Slide

  87. So,
    I think that some
    factors are necessary

    View Slide

  88. Required factors for DX
    • Usability
    • Maintainability
    • Testability
    • Extendability
    • Integrity

    View Slide

  89. Usability
    • i18n takes time and effort …
    • Need to be the simply and easily to use it

    View Slide

  90. Maintainability
    • Just like UI, frequently occur the specification
    changing.
    • e.g. add/change/delete the resource

    View Slide

  91. Testability
    • Hard to i18n test …
    • e.g. Test per language
    • e.g. Setup language environment

    View Slide

  92. Extendability
    • need to support edge-case of i18n
    • e.g. String encoding:
    • utf-8 -> shift-jis
    • e.g. Decision Shortcut key:
    • en: enter
    • ja: shift + enter

    View Slide

  93. Integrity
    • Need to integrate l10n tools or services
    • e.g. continuous localization

    View Slide

  94. So, how should such i18n be
    provided with these factors?

    View Slide

  95. I thought about it!

    View Slide

  96. • SaaS for i18n
    • i18n framework

    View Slide

  97. SaaS for i18n?
    • Provide some SaaS of development
    environments!
    Need the i18n as a Service!
    source code
    management
    ci code coverage
    platform environment error tracking

    View Slide

  98. i18n framework?
    Resource Catalog
    Message
    Formarting
    Custom X
    CLI Testing tools
    UI supporting
    tools

    View Slide

  99. Ideally,
    Saas is best!

    View Slide

  100. However,
    We’re not an entrepreneur,
    but a developer.

    View Slide

  101. I talk about i18n
    framework after this!

    View Slide

  102. i18n framework
    • Requirement
    • Resolve the current problems
    • Provide 5 factors

    View Slide

  103. What kind of framework
    do we provide?

    View Slide

  104. By the way,
    regarding to framework …

    View Slide

  105. We have the image of full-stack!

    View Slide

  106. Is i18n framework
    the best
    full-stack framework?

    View Slide

  107. No,
    I think so not best.

    View Slide

  108. Because …
    • We don’t need all the things!
    • High learning cost…
    Resource Catalog
    Message
    Formarting
    Custom X
    CLI Testing tool
    UI supporting
    tool

    View Slide

  109. I think i18n framework
    should be like
    progressive framework.

    View Slide

  110. What’s
    Progressive Framework?
    https://docs.google.com/presentation/d/1WnYsxRMiNEArT3xz7xXHdKeH1C-jT92VxmptghJb5Es/edit#slide=id.p
    From the below slide:

    View Slide

  111. Advocate
    Evan You
    Vue.js creator

    View Slide

  112. Framework

    View Slide

  113. “In development, framework must resolve the
    inherent complexity of application.”

    View Slide

  114. “However, framework have instrumental complexity.”

    View Slide

  115. “Framework complexity is the price that we
    pay in handling application complexity.”

    View Slide

  116. Insufficient metaphor

    View Slide

  117. Overkill metaphor

    View Slide

  118. “Need to select the right framework.”

    View Slide

  119. However,
    In Business, requests are
    constantly changing …

    View Slide

  120. Progressive
    Framework

    View Slide

  121. “Framework that can solve problems in respond
    to progressive request changes of applications.”
    install libs and tools when need to resolve

    View Slide

  122. We are usually doing it …
    What is different?

    View Slide

  123. •Define problem regions and
    provide resolve ways.
    •Support the least official libs
    and tools possible.

    View Slide

  124. In Vue.js …

    View Slide

  125. View Slide

  126. Vue.js Core Vue Router Vuex Vue CLI
    If necessary, use progressively libs and tools

    View Slide

  127. Also in i18n …

    View Slide

  128. Just like it,
    I believe we can compose.

    View Slide

  129. Problem regions
    • i18n Core
    • String encoding
    • Date & Time
    • and more …
    • Resource interpolating
    • Message syntax format (ICU or other?)
    • Interpolating system

    View Slide

  130. Problem regions
    • Resource catalog system
    • Resource structure
    • Resource bundling
    • Resource format
    • Resource management
    • CLI

    View Slide

  131. Problem regions
    • Large scale supporting system
    • UI layouting tool
    • Test utilities
    • Integration supprting
    • … and more!

    View Slide

  132. i18n
    Core
    Resource
    Catalog
    System
    Resource
    Management
    Resource
    Interpolation
    Large
    Scale
    Supporting
    System

    View Slide

  133. Resource
    Catalog
    System
    Resource
    Management
    Resource
    Interpolation
    Large
    Scale
    Supporting
    System
    ? ? ? ?
    ?
    i18n
    Core

    View Slide

  134. Cases
    • small-case: e.g. client only
    • i18n Core
    • Resource interpolation
    • Resource catalog system
    • large-case: e.g. client / server
    • Above + …
    • Resource management
    • Large scale supporting system

    View Slide

  135. Architecture

    View Slide

  136. Plugin System
    • Enhance the features available
    • Add the methods or functions
    i18n Core
    i18n plugin-x
    (ISO3166)
    App
    (EC Service)
    i18n plugin-1
    (E.164)

    View Slide

  137. Technology

    View Slide

  138. View Slide

  139. What’s WebAssembly?
    • WebAssembly (abbreviated WASM) is a binary
    instruction format for a stack-based virtual
    machine.
    • WASM is designed as a portable target for
    compilation of high-level languages like C/C++/
    Rust, enabling deployment on the web for client and
    server applications.
    https://webassembly.org/
    Reference From top of website

    View Slide

  140. This is it!

    View Slide

  141. WASM implementation
    i18n Core
    &
    Plugins
    WASM VM WASM VM
    WASM VM
    Load Load
    Run
    Run
    module

    View Slide

  142. Tools

    View Slide

  143. CLI
    • Resource Catalog Management tool
    • Resource Interpolate optimization
    • and more …
    i18n add zh
    i18n extract ja.json, fr.json
    i18n convert en.csv en.json
    i18n compile

    View Slide

  144. Editor
    • Provide the extension of editor
    • Like the IntelliSense of VSCode
    Easy resource key inputing !

    View Slide

  145. UI layout
    • Like the Storybook? (NOTE: My image)
    https://github.com/joscha/storybook-addon-i18n-tools
    Reference From `I18n Toolbox Storybook Add-on`

    View Slide

  146. Test utilities
    • Pesodu Localization?
    https://medium.com/netflix-techblog/pseudo-localization-netflix-12fff76fbcbe
    Reference From `Pseudo Localization @ Netflix`

    View Slide

  147. Integration

    View Slide

  148. • Support continuous localization
    App
    Hook API for l10n service
    i18n Core Localization Service
    hook
    Call
    Notify
    Translate
    Translator

    View Slide

  149. TBD
    of
    i18n framework

    View Slide

  150. TBD
    • WASM is still MVP…
    • NONE-JavaScript environments don't
    support …
    • The Implementation of plugin system is
    difficult …
    • How to distribution i18n framework
    • Library implemented with WebAssembly

    (Should we use existing package manager?)

    View Slide

  151. TBD
    • Should we need to re-implement the bellow?
    • date & time
    • String encoding
    • and more …

    View Slide

  152. Conclusions

    View Slide

  153. Conclusions
    • About i18n
    • Current of i18n
    • Problems of i18n
    • Next generation i18n

    View Slide

  154. Q & A

    View Slide

  155. Welcome
    feedback & patron!
    https://www.patreon.com/kazupon

    View Slide

  156. Thank You!

    View Slide