Nuts and Bolts of Internationalization

An in-depth dive into building an internationalized application, focusing on Express, but with concepts applicable to many situations.

Presented at Nodevember 2015 in Nashville, TN

Transcript

1. Nuts and bolts of internationalization
2. What is i18n? Internationlization is the process of making your application able to handle multiple languages. I, 18 letters, n. i18n.
3. How many of you can read five languages?
4. How many of you can read two languages?
5. Just one?
6. Supporting multiple languages is hard
7. Why do we do this?
8. Internationalization is accessibility
9. Empathy Imagine you’re planning a vacation
10. Empathy Grab your iPad, sit down on the patio in your house in El Pedregal, Mexico City.
11. Empathy Type vacaciones gran cañon into google.com.mx.
12. Human languages have some irregular bits. console.log("There are " + items.length + " " + ( items.length == 1 ? "item" : "items" ) + " in your cart")
13. in Polish istnieją 0 produkty w koszyku. istnieje 1 produkt w koszyku. istnieją 2 produkty w koszyku. istnieją 3 produkty w koszyku. istnieją 4 produkty w koszyku. istnieje 5 produktów w koszyku
14. Polish console.log(( items.length == 0 ? "istnieją " + items.length + " produkty" : items.length == 1 ? "znajduje się " + items.length + " produkt" : items.length % 10 == 2 || items.length % 10 == 3 || items.length % 10 == 4 ? "istnieją " + items.length + " produkty" : "istnieje " + items.length + " produktów" ) + " w koszyku");
15. We’ve created a monster console.log( lang == "pl" ? ( items.length == 0 ? "istnieją " + items.length + " produkty" : items.length == 1 ? "znajduje się " + items.length + " produkt" : items.length % 10 == 2 || items.length % 10 == 3 || items.length % 10 == 4 ? "istnieją " + items.length + " produkty" : "istnieje " + items.length + " produktów" ) + " w koszyku" ) : lang == "en" ? ( "There are " + items.length + " " + (items.length == 1 ? "item" : "items") + " in your cart" ) : "unsupported language" );
16. "dependencies": { "the-english-language": "^2015.0.0", "academie-francaise": "^2005.33.9" } Never make your code depend on English.
17. Because we’ve integrated this into our code, we have to scatter i18n into all sorts of places, deep and high in the stack.
18. Adding a new language means editing the entire codebase. Translations take time. This means several edits. Merge conflicts with every piece of the codebase that has user- visible text.
19. Let’s find a better way.
20. MessageFormat (or gettext, or ...) Push the list of cases out into each translation. Polish specifics go in the Polish language files. Programmers see only one string in the source code.
21. Message formatters usually use a key in the source code, plus placeholder values to fill in numbers and dates. Essentially, a function call.
22. English { "cart": { "items": "There are {items, number, =1 {item}, other {#items}} in your cart" } }
23. Polish (as line-wrapped JSON) { "cart": { "items": "{items, number, one {znajduje się # produkt w koszyku.} few {istnieją # produkty w koszyku.} many {istnieje # produktów w koszyku}}" } }
24. And in our code: formatMessage(messages.items.cart, { items: 3 });
25. That was the easy part.
26. Workflows The hard part.
27. The ongoing pain Applications change over time.
28. git commit -m 'updated translations for user interface' Not that simple.
29. Translation takes time.
30. Updates should flow one way
31. Decide on a definitive source translation. Update that, then retranslate the changed pieces in each language. Remember that you have to maintain any specialization.
32. ¡Vamos a crearlo! So let’s do this!
33. server.js var express = require('express'); var path = require('path'); var app = express(); var hbs = require('hbs'); var hbsIntl = require('handlebars-intl'); var engine = hbs.create(); hbsIntl.registerWith(engine); app.engine("hbs", engine.__express); app.listen(process.env.PORT || 8080);
34. app.use(function selectLanguageForRequest(req, res, next) { var lang = req.query.lang || 'en'; // Or use req.headers['Accept-Language'] // Or use the user's account settings. // Or use multiple strategies. req.messages = require(path.resolve(__dirname, 'locales', lang + '.json')) next(); });
35. views/hello.hbs <!doctype html> <p>{{formatMessage messages.hello}}</p> Handler app.get('/', function (req, res) { res.render('hello.hbs', { messages: req.messages }); });
36. locales/es.json { "hello": "¡Hola al mundo!" } locales/en.json { "hello": "Hello, World!" }
37. Let’s try it out. $ PORT=8080 npm start $ curl http://localhost:$PORT <!doctype html> <p>Hello, World!</p> $ curl http://localhost:$PORT?lang=es <!doctype html> <p>¡Hola al mundo!</p>
38. Let’s add more
39. English { "bag": "There {items, plural, one{is one item} other {are # items}} in your bag" }
40. Spanish { "bag": "Hay {items, plural, one {# itema} other {# itemas}} en su bolso" }
41. views/bag.hbs <!doctype html> <p>{{formatMessage messages.bag items=items}}}</p> Handler app.get('/bag', function (req, res) { res.render('bag.hbs', { messages: req.messages, items: req.query.items }); });
42. http://localhost:8080/bag?items=2 <!doctype html> <p>There are 2 items in your bag</p> http://localhost:8080/bag?items=1 <!doctype html> <p>There is 1 item in your bag</p> http://localhost:8080/bag?items=2&lang=es <!doctype html> <p>Hay 2 itemas en su bolso</p>
43. Now let’s do it in the browser
44. app.use('/locales', serveStatic(path.resolve(__dirname, 'locales')));
45. var MessageFormat = require('message-format'); function Formatter(dict) { this.dict = dict; } Formatter.prototype.format = function format(message, args) { if (!this.dict[message]) { console.warn('no translation found for', message); } message = this.dict[message] || message; return new MessageFormat(message).format(args); } module.exports = Formatter;
46. // A trivial ‘render’ function for my component^Wapplication module.exports = function render(formatter) { document.querySelector('p').innerText = formatter.format("bag", { items: 2}); }
47. // Polyfills are scratchy require('intl'); require('intl/locale-data/jsonp/en.js'); require('intl/locale-data/jsonp/es.js'); var fetch = require('isomorphic-fetch'); var Promise = require('bluebird'); var Formatter = require('./formatter'); var render = require('./render');
48. var dramaticPause = 3000; var lang = document.documentElement.getAttribute('lang'); var messages = fetch('/locale/' + lang + '.json').then(function (res) { return res.json(); }); messages.then(function (dict) { alert('A dramatic pause...'); return Promise.delay(dramaticPause)).then(function () { var formatter = new Formatter(dict); render(formatter); }); });
49. <!doctype html> <html lang="{{ lang }}"> <p>Loading...</p> <script src='built-app.js'></script> </html>
50. Angular? Content service exposes key-value pairs. Supply some!
51. Dust templating? formatjs.io has dust-intl. It’s great, and supplies message formatting. Just supply text strings.
52. User Interface Concerns • Finding word boundaries isn’t always easy • Japanese sentences involving imported words can be very long • German words get very long and finding good wrapping gets tricky • Arabic and 8 other currently used scripts start on the right and go left.
53. Culture Matters • Names don’t work everywhere the same way they do in your country. • Names don’t even work the way you think they do. • Not everyone writes numbers the same way.
54. Warnings Language != locale English is spoken in the US. English is spoken in the UK. But we spell colour differently and we write our dates inside out in the US. Same language, different specifics. You can call the language with the local details a "locale".
55. BCP47 document from the IETF has a whole standard for identifiers for languages. en-US en-UK en i-navajo zh-CN-hanz
56. Tips for language tags If you’re parsing a language expectation from an external source, you may have more or less to the language tag than you expect. Use the bcp47 module to parse them. Use bcp47-serialize to get them as a string again. Canonicalize into a locale you support early on.
57. Tips for language tags Plan to do matching and fallback when you get a request for a language that’s close to one you support but not quite right.
58. Tips for language tags Pass locale tags as opaque strings whenever possible -- it’s far easier to get right. "en-US" // Better {lang: 'en', region: 'US'} // You will make mistakes Especially once you add i-navajo and zh-CN-hanz.
59. Tips for long form Handle long form content separately. Use one language per file. Keep it simple.
60. Command-line apps Look at the LANG environment variable, or the pieces and parts, LC_*. How you sort, display numbers, each specifiable separately. Consider gettext format messages to be similar to apps in C.
61. And in closing Your native language isn’t the right way, it’s just a way.