The Localization of Stack Overflow- QCon China 2014

Slides supporting the "Localization of Stack OVerflow" talk presented at QCon China in April 2014

Transcript

1. The Localization Of Marco Cecconi @sklivvz sklivvz@stackoverflow.com
2. Who are we?
5. 561,027,840 pageviews in the last 30 days* (~100% growth year over year) *source: Quantcast
6. Why did we want to do this?
7. Expert programmers Fluent English EN Stack Overflow
8. Expert programmers Fluent English EN Stack Overflow Fluent Portuguese PT Stack Overflow
9. Different languages make Different communities
10. Expert programmers Stack Overflow
11. We want to make the internet a better place for all
12. Architectural Requirements
13. Easy.
14. Easy. Fast.
15. Performance is a feature
18. 1. Allocates two objects per translated string 2. Does a lot of lookups
20. 1. No allocation (strings are interned) 2. No lookups 3. Not “easy”, not “usable”
22. Simplest/easiest possible code
23. Simplest/easiest possible code Implementation
24. Source Code Compile Time Run Time aspnet_compiler Look Up
25. Simplest/easiest possible code Compiled to the equivalent of this
26. Source Code Compile Time Run Time extended aspnet_compiler using Roslyn Look Up
27. Javascript - No GC pressure so we don’t care about interned strings - Can’t really precompile either - We simply create one set of JS files per language, e.g. “stub.en.js” and “stub.pt.js” - For all that follows, the same APIs are available to Javascript
28. Our solution
30. API _s(string value) Meaning “Substitute String” _m(string value) Meaning “Substitute Markdown”
31. Languages are WEIRD (part 1)
32. _s("Hello $name$", new { name = "Marco" }) Hello Marco
33. 一隻雞 兩隻雞
34. 1 chicken 2 chickens
35. Language Name Code Category Examples Rules Chinese zh other 0-999; 1.2... other → everything English en one 1 one → n is 1; other → everything else other 0, 2-999; 1.2, 2.07...
36. one 1, 21, 31, 41, 51, 61, 71, 81, 101, 1001, … i % 10 = 1 and i % 100 != 11 few 2~4, 22~24, 32~34, 42~44, 52~54, 62, 102, 1002, … i % 10 = 2..4 and i % 100 != 12..14 many 0, 5~19, 100, 1000, 10000, 100000, 1000000, … i % 10 = 0 or i % 10 = 5..9 or i % 100 = 11..14 other 0.0~1.5, 10.0, 100.0, 1000.0, 10000.0, 100000.0, 1000000.0, … other 0~15, 100, 1000, 10000, 100000, 1000000, … Ukranian!
37. Welsh has SIX modes…
38. _s(“#num# chickens", new { num = 3 }) 3 chickens
39. Behind the scenes All combinations are generated for each language and sent to translators:  For Chinese: “$num:other$ chickens” will be sent  For a 2 mode language: “$num:one$ chickens” and “$num:other$ chickens” will be sent Rules have to be evaluated at runtime to choose the correct translation.
40. Languages are WEIRD (part 2)
42. 10 classes called Class I to Class X and containing all sorts of arbitrary groupings but often characterised as • people, • long objects, • animals, • miscellaneous objects, • large objects and liquids, • small objects, • languages, • pejoratives, • infinitives, • mass nouns Uganda
43. _s("Active $~posts$") Attivi _s("Active $~questions$") Attive
44. Post Mortem
45. Some numbers 700 views localized 100,000 lines of code A lot of javascript A LOT of refactoring/fixing/tech debt repayed Very little performance impact ~6 months of work (team of ~3)
46. More numbers Portuguese released Dec. 12 4k Questions 7k Answers 8k Users One of the best performing new communities ever
48. Lessons learned
49. Never put non-content text data in the DB It’s A Good Thing™ if all the text to be localized is in the views or javascript.
50. Never compose sentences in code n==1? "1 unicorn": n.ToString() + " unicorns"
52. Never assume anything about the language 10 genders and 6 plurals? REALLY?
53. Designing a global application is hard
54. Conclusions
55. 1. It’s possible to internationalize quickly and cheaply, without performance hits. 2. Localization is a surprisingly rich problem. There are many gotchas that can be painful later, like pluralization “bugs”. Fun! 3. Localization is a very healthy choice for Stack Overflow and we hope to provide more and more users with a native interface some day :-)
56. Questions? Marco Cecconi @sklivvz sklivvz@stackoverflow.com