Slide 1
Slide 1 text
The future of i18n
(ϘΫ͕ߟ͑Δi18nͷະདྷ)
2018.09.07
builderscon tokyo 2018
@kazupon
Slide 2
Slide 2 text
Who are You?
Slide 3
Slide 3 text
kazupon
CTO
Vue.js Core Team
Vue.js japan user group organizer
@kazu_pon kazupon
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
Ticket sales, Coming soon ...
Slide 6
Slide 6 text
Introduction
Slide 7
Slide 7 text
Background
• I've experienced a lot of
different i18n what
happened.
• String encoding
• Multi Languages
• and more ….
• I've developed, maintained
as OSS developer.
App
or
Service
vue-i18n
Slide 8
Slide 8 text
Come into
one's mind for
the future of
i18n ...
Welcome Feedback!
Slide 9
Slide 9 text
Today about talking
Slide 10
Slide 10 text
Agenda
• The first half
• About i18n
• Current of i18n
• Problems of i18n
• The second half
• Next generation i18n
Slide 11
Slide 11 text
Today about not to talk
• Design and Process of
software development
for globalization
• Process of localization
• See the book!
https://book.impress.co.jp/books/1117101057
Slide 12
Slide 12 text
About i18n
Slide 13
Slide 13 text
i18n
l10n
Slide 14
Slide 14 text
What is i18n?
• Internationalization
• Generalize products in the format that doesn't
depend on specific language, region and
culture
i nternationalizatio n
18 letters
i18n
Slide 15
Slide 15 text
What is l10n?
• Localization
• Specialize products in the format that depend
on specific language, region and culture
l ocalizatio n
10 letters
l10n
Slide 16
Slide 16 text
i18n <-> l10n
i18n + l10n = g11n (globalization)
11 letters
Base
ja
en fr
l10n
i18n
g11n
Slide 17
Slide 17 text
FYI: Numeronym
• A numeronym is a number-based word.
l10n
i18n g11n
https://en.wikipedia.org/wiki/Numeronym
World Wide Web Consortium
w3c
a11y k8s d11n
Slide 18
Slide 18 text
Joke: How to call i18n?
i18n (͍͍Μ: iiyann)
https://twitter.com/Konboi/status/1031431327692681216
Slide 19
Slide 19 text
By the way,
Do you support i18n
at your app ?
Slide 20
Slide 20 text
Are you Serious?
Slide 21
Slide 21 text
Importance of i18n
• hard to support software maintainance after
release …
• Leak easily, when software has became
large scale.
• Need to re-design the UI layouts.
• Database, and other systems are also
affected …
Slide 22
Slide 22 text
• And also hard to localization …
• The translator fix the source code.
-> Really? It’s impossible …
• Even if i18n had been fixed with translator,
there is the possibility of breaking app ...
Importance of i18n
Breaking …
Slide 23
Slide 23 text
If cannot support i18n…
• Can not keep usability of product …
What’s mean?
Slide 24
Slide 24 text
NG Case
https://twitter.com/frsyuki/status/1030189494371115008
Slide 25
Slide 25 text
Current of i18n
Slide 26
Slide 26 text
Platform
• Built-in i18n and
provide for
application
• Windows / Mac
OS / Linux (*nix)
• iOS / Android
• Web
Slide 27
Slide 27 text
Development Environment
• Built-in i18n to Programming languages
• ex: datetime, string encoding
• Built-in i18n to Application Framework
• ex: Ruby on Rails
Slide 28
Slide 28 text
Contents of i18n
Slide 29
Slide 29 text
Display of resource
Slide 30
Slide 30 text
App
Resource interpolation
i18n lib
Resource
Catalog
configure
en_US.json
ja_JP.json
read
select
interpolate
Slide 31
Slide 31 text
Format of Resource Catalog
• .pot, .po (gettext)
• JSON
• YAML
• .strings (macOS / iOS)
• XML base (Android, Java, .NET)
• And more …
Slide 32
Slide 32 text
Resource Catalog Structure
• Flat key pairs
• Hierarchical
Slide 33
Slide 33 text
Message Formatting
• ICU (International Components for Unicode)
Message Format
http://userguide.icu-project.org/formatparse/messages
Slide 34
Slide 34 text
ICU
• Plural
Slide 35
Slide 35 text
ICU
• Date & Time
Slide 36
Slide 36 text
ICU
• Number
Slide 37
Slide 37 text
DateTime
Slide 38
Slide 38 text
Built-in programming languages
• Support in most programming languages
• e.g. Ruby
• DateTime class
• Time class
• Date class
Slide 39
Slide 39 text
Character Code
Slide 40
Slide 40 text
Character encoding
• Unicode have almost becoming defaults.
https://unicode.org/
• Web:
ex, html
• programming languages:
ex, python (v3)
Slide 41
Slide 41 text
Collation
• Database
e.g. MySQL
Slide 42
Slide 42 text
Collation
• ECMAScript i18n API
Slide 43
Slide 43 text
Emoji
Slide 44
Slide 44 text
Skin tone
http://std.dkuug.dk/JTC1/SC2/WG2/docs/n4599.pdf
Reference from: “WG2 N4599 Skin tone modifier symbols” Unicode Consortium,
Slide 45
Slide 45 text
Font
Slide 46
Slide 46 text
Web Fonts
Slide 47
Slide 47 text
Google Fonts
https://fonts.google.com/
Slide 48
Slide 48 text
Standardization
Slide 49
Slide 49 text
It’s almost standardized
• character encoding: ISO/IEC 10646
• datetime: ISO8601
• language tags: BCP 47 (IETF)
• countries/states/subdivisions : ISO3166
• phone number: E.164
• currency: ISO4217
• and more …
Slide 50
Slide 50 text
Libraries
Slide 51
Slide 51 text
Enhanced with external libs
• Phone Number
libphonenumber (C/C++):
https://github.com/googlei18n/libphonenumber
• Countries
i18n-iso-countries(Node.js):
https://github.com/michaelwittig/node-i18n-iso-countries
• DateTime
spacetime (JavaScript):
https://github.com/spencermountain/spacetime
• and more …
Slide 52
Slide 52 text
UI libraries
Slide 53
Slide 53 text
Date Picker
Slide 54
Slide 54 text
Calendar
Slide 55
Slide 55 text
Emoji Picker
Slide 56
Slide 56 text
g11n is easy to start!
due to i18n fruitful
environments.
Slide 57
Slide 57 text
However …
Slide 58
Slide 58 text
https://qiita.com/yugui/items/55f2529c5a731badeff7
Slide 59
Slide 59 text
Hard to perfect …
Slide 60
Slide 60 text
Problems of i18n
Slide 61
Slide 61 text
Languages
• One country have multi languages (e.g. india)
• Dialect (e.g. bt-pt)
• Middle name (e.g. es)
• Honorifix
• and more …
Slide 62
Slide 62 text
Feature of product
• Law
• Tax (payment, order)
• Accounts (tax, audit)
• GDPR
• Culture
• Foods (tastes, … etc)
• and more …
Slide 63
Slide 63 text
UI
• LTR (most)
• RTL (arabia, …etc)
• Vertical line (japanese, … etc)
Slide 64
Slide 64 text
Database
• Collation
• Attribute of records dependent on country or
culture
• e.g. Middle name
• Contents management for languages
• and more …
Slide 65
Slide 65 text
• I/F Miss-match of i18n per library
• Resource catalog management
Server
Client / Server
Client
i18n lib
i18n lib
e.g. API or BFF model
Resource catalog
(errors, etc)
Resource catalog
(client specific)
Client
i18n lib
Resource catalog
(client specific)
Slide 66
Slide 66 text
Client / Server
• UI placeholder and contents management
https://developer.kaizenplatform.com/entry/axross/2018-07-02
Reference: kaizen platform engineer blog
Slide 67
Slide 67 text
Edge case of project
• Grouping of Prefectures and Countries
(no standardization)
Slide 68
Slide 68 text
Edge case of project
• Convert csv to shift_jis from utf-8
• e.g. Excel
Slide 69
Slide 69 text
Edge case of project
• Convert to ISO8601 from un-standardized
datetime string
Slide 70
Slide 70 text
i18n fatigue …
Slide 71
Slide 71 text
By the way,
What state of l10n?
Slide 72
Slide 72 text
Translation
• Translation have improved dramatically with
machine learning!
https://arxiv.org/pdf/1609.08144.pdf
Reference:
Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Slide 73
Slide 73 text
Translation process
• Out-sourcing model
• Crowd-sourcing model
Translator
Software
Company
Translation
Company
Software
Company
Translator
Software
Company
Translator
Software
Company
Translator
Translator …
out-sourcing crowd-sourcing
…
Translator
Slide 74
Slide 74 text
i18n-able content service
https://www.contentful.com/r/knowledgebase/localization/
Slide 75
Slide 75 text
Support i18n integration
• Import / Export resource catalog
Localization Services
.po
.json
.resx
…
Import
Export
Slide 76
Slide 76 text
Support i18n integration
• API
https://medium.com/@jamuhl/vue-js-from-internationalization-i18n-to-localization-l10n-and-back-again-c3e5f7cc5e71
Reference: vue.js: from internationalization (i18n) to localization (l10n) and back again
Slide 77
Slide 77 text
We need the
NEW i18n
environments !
Slide 78
Slide 78 text
Q & A
Slide 79
Slide 79 text
Next
generation
i18n
Slide 80
Slide 80 text
What kind of i18n do
we want?
Slide 81
Slide 81 text
I thought in App dev position
• I don't want to worry about key naming of
resource catalog …
• I don't want to worry about resource catalog
management …
• I don't want to effort pseudo localization test …
• I don't want to worry about choice of i18n
library …
Slide 82
Slide 82 text
I want to focus on
App development !!
Slide 83
Slide 83 text
Next generation i18n
Slide 84
Slide 84 text
✨DX✨
(Developer Experience)
Slide 85
Slide 85 text
Need to provide DX
to developer!
Slide 86
Slide 86 text
However,
How can do you
provide DX?
Slide 87
Slide 87 text
So,
I think that some
factors are necessary
Slide 88
Slide 88 text
Required factors for DX
• Usability
• Maintainability
• Testability
• Extendability
• Integrity
Slide 89
Slide 89 text
Usability
• i18n takes time and effort …
• Need to be the simply and easily to use it
Slide 90
Slide 90 text
Maintainability
• Just like UI, frequently occur the specification
changing.
• e.g. add/change/delete the resource
Slide 91
Slide 91 text
Testability
• Hard to i18n test …
• e.g. Test per language
• e.g. Setup language environment
Slide 92
Slide 92 text
Extendability
• need to support edge-case of i18n
• e.g. String encoding:
• utf-8 -> shift-jis
• e.g. Decision Shortcut key:
• en: enter
• ja: shift + enter
Slide 93
Slide 93 text
Integrity
• Need to integrate l10n tools or services
• e.g. continuous localization
Slide 94
Slide 94 text
So, how should such i18n be
provided with these factors?
Slide 95
Slide 95 text
I thought about it!
Slide 96
Slide 96 text
• SaaS for i18n
• i18n framework
Slide 97
Slide 97 text
SaaS for i18n?
• Provide some SaaS of development
environments!
Need the i18n as a Service!
source code
management
ci code coverage
platform environment error tracking
Slide 98
Slide 98 text
i18n framework?
Resource Catalog
Message
Formarting
Custom X
CLI Testing tools
UI supporting
tools
Slide 99
Slide 99 text
Ideally,
Saas is best!
Slide 100
Slide 100 text
However,
We’re not an entrepreneur,
but a developer.
Slide 101
Slide 101 text
I talk about i18n
framework after this!
Slide 102
Slide 102 text
i18n framework
• Requirement
• Resolve the current problems
• Provide 5 factors
Slide 103
Slide 103 text
What kind of framework
do we provide?
Slide 104
Slide 104 text
By the way,
regarding to framework …
Slide 105
Slide 105 text
We have the image of full-stack!
Slide 106
Slide 106 text
Is i18n framework
the best
full-stack framework?
Slide 107
Slide 107 text
No,
I think so not best.
Slide 108
Slide 108 text
Because …
• We don’t need all the things!
• High learning cost…
Resource Catalog
Message
Formarting
Custom X
CLI Testing tool
UI supporting
tool
Slide 109
Slide 109 text
I think i18n framework
should be like
progressive framework.
Slide 110
Slide 110 text
What’s
Progressive Framework?
https://docs.google.com/presentation/d/1WnYsxRMiNEArT3xz7xXHdKeH1C-jT92VxmptghJb5Es/edit#slide=id.p
From the below slide:
Slide 111
Slide 111 text
Advocate
Evan You
Vue.js creator
Slide 112
Slide 112 text
Framework
Slide 113
Slide 113 text
“In development, framework must resolve the
inherent complexity of application.”
Slide 114
Slide 114 text
“However, framework have instrumental complexity.”
Slide 115
Slide 115 text
“Framework complexity is the price that we
pay in handling application complexity.”
Slide 116
Slide 116 text
Insufficient metaphor
Slide 117
Slide 117 text
Overkill metaphor
Slide 118
Slide 118 text
“Need to select the right framework.”
Slide 119
Slide 119 text
However,
In Business, requests are
constantly changing …
Slide 120
Slide 120 text
Progressive
Framework
Slide 121
Slide 121 text
“Framework that can solve problems in respond
to progressive request changes of applications.”
install libs and tools when need to resolve
Slide 122
Slide 122 text
We are usually doing it …
What is different?
Slide 123
Slide 123 text
•Define problem regions and
provide resolve ways.
•Support the least official libs
and tools possible.
Slide 124
Slide 124 text
In Vue.js …
Slide 125
Slide 125 text
No content
Slide 126
Slide 126 text
Vue.js Core Vue Router Vuex Vue CLI
If necessary, use progressively libs and tools
Slide 127
Slide 127 text
Also in i18n …
Slide 128
Slide 128 text
Just like it,
I believe we can compose.
Slide 129
Slide 129 text
Problem regions
• i18n Core
• String encoding
• Date & Time
• and more …
• Resource interpolating
• Message syntax format (ICU or other?)
• Interpolating system
Slide 130
Slide 130 text
Problem regions
• Resource catalog system
• Resource structure
• Resource bundling
• Resource format
• Resource management
• CLI
Slide 131
Slide 131 text
Problem regions
• Large scale supporting system
• UI layouting tool
• Test utilities
• Integration supprting
• … and more!
Slide 132
Slide 132 text
i18n
Core
Resource
Catalog
System
Resource
Management
Resource
Interpolation
Large
Scale
Supporting
System
Slide 133
Slide 133 text
Resource
Catalog
System
Resource
Management
Resource
Interpolation
Large
Scale
Supporting
System
? ? ? ?
?
i18n
Core
Slide 134
Slide 134 text
Cases
• small-case: e.g. client only
• i18n Core
• Resource interpolation
• Resource catalog system
• large-case: e.g. client / server
• Above + …
• Resource management
• Large scale supporting system
Slide 135
Slide 135 text
Architecture
Slide 136
Slide 136 text
Plugin System
• Enhance the features available
• Add the methods or functions
i18n Core
i18n plugin-x
(ISO3166)
App
(EC Service)
i18n plugin-1
(E.164)
Slide 137
Slide 137 text
Technology
Slide 138
Slide 138 text
No content
Slide 139
Slide 139 text
What’s WebAssembly?
• WebAssembly (abbreviated WASM) is a binary
instruction format for a stack-based virtual
machine.
• WASM is designed as a portable target for
compilation of high-level languages like C/C++/
Rust, enabling deployment on the web for client and
server applications.
https://webassembly.org/
Reference From top of website
Slide 140
Slide 140 text
This is it!
Slide 141
Slide 141 text
WASM implementation
i18n Core
&
Plugins
WASM VM WASM VM
WASM VM
Load Load
Run
Run
module
Slide 142
Slide 142 text
Tools
Slide 143
Slide 143 text
CLI
• Resource Catalog Management tool
• Resource Interpolate optimization
• and more …
i18n add zh
i18n extract ja.json, fr.json
i18n convert en.csv en.json
i18n compile
…
Slide 144
Slide 144 text
Editor
• Provide the extension of editor
• Like the IntelliSense of VSCode
Easy resource key inputing !
Slide 145
Slide 145 text
UI layout
• Like the Storybook? (NOTE: My image)
https://github.com/joscha/storybook-addon-i18n-tools
Reference From `I18n Toolbox Storybook Add-on`
Slide 146
Slide 146 text
Test utilities
• Pesodu Localization?
https://medium.com/netflix-techblog/pseudo-localization-netflix-12fff76fbcbe
Reference From `Pseudo Localization @ Netflix`
Slide 147
Slide 147 text
Integration
Slide 148
Slide 148 text
• Support continuous localization
App
Hook API for l10n service
i18n Core Localization Service
hook
Call
Notify
Translate
Translator
Slide 149
Slide 149 text
TBD
of
i18n framework
Slide 150
Slide 150 text
TBD
• WASM is still MVP…
• NONE-JavaScript environments don't
support …
• The Implementation of plugin system is
difficult …
• How to distribution i18n framework
• Library implemented with WebAssembly
(Should we use existing package manager?)
Slide 151
Slide 151 text
TBD
• Should we need to re-implement the bellow?
• date & time
• String encoding
• and more …
Slide 152
Slide 152 text
Conclusions
Slide 153
Slide 153 text
Conclusions
• About i18n
• Current of i18n
• Problems of i18n
• Next generation i18n
Slide 154
Slide 154 text
Q & A
Slide 155
Slide 155 text
Welcome
feedback & patron!
https://www.patreon.com/kazupon
Slide 156
Slide 156 text
Thank You!