Emojis! 🎉🙌😎 (GDG Riga 2017)

Over the past decade the popularity of emojis has grown from relative obscurity to a big-budget Hollywood picture 🎞️.

Emojis used to be solely in the domain of chat apps but these days any app having user generated content is expected to have emoji support.
In this session we'll take a technical look at how Emojis are represented in Unicode 🤓, identify common pitfalls 🐛 and find ways to circumvent them (pitfalls that is, not emojis 😎).

Transcript

1. Emojis! !"# @madisp
2. @madisp
3. !
4. !"
5. !"#
6. !"#$
7. !"#$%
8. 1999 NTT DoCoMo, Shigetaka Kurita
9. → 2010 Unicode 6.0, 722 characters → 2011 iOS 5 released → 2013 Android 4.3 released
10. ! word of the year 2015 Oxford Dictionaries
12. !
14. Unicode is everywhere
15. Unicode is everywhere == emojis are everywhere
16. Unicode is everywhere https://en.wikipedia.org/wiki/!
17. Unicode is everywhere https://en.wikipedia.org/wiki/!
18. Unicode is everywhere $ git checkout -b ! Switched to a new branch '!'
19. Unicode is everywhere $ git checkout -b ! Switched to a new branch '!' $ git checkout -b " Switched to a new branch '"'
20. Unicode is everywhere $ git checkout -b ! Switched to a new branch '!' $ git checkout -b " Switched to a new branch '"' $ git branch -l madis/fix-filescan-race master release-1.x * " !
21. Unicode is (almost) everywhere → Payment message: Money !"# → Payment message: Money
22. Emojis on the JVM >>> "!" !
23. Emojis on the JVM >>> "!" ! >>> "!".length 2
24. Emojis on the JVM >>> "!" ! >>> "!".length 2 >>> "!".substring(0,1) ?
25. !
26. Text encodings → US-ASCII, 7 bits (128 chars) → ISO-8859-1 .. 16, 8 bits (256 chars) → Unicode!
27. Unicode / UCS-2 → 16 bits (65536 chars) → Windows (wchar_t), Java, JavaScript, Python 2 → U+0000..U+FFFF → Basic Multilingual Plane
28. Unicode / UTF-16 → 17 planes in total → Reserve U+D800..U+DFFF for surrogate pairs 0xD800 = 1101100000000000 0xDFFF = 1101111111111111
29. Unicode / UTF-16 0xD800 = 110110 0000000000 0xDFFF = 110111 1111111111
30. Unicode / UTF-16 0xD800 = 110110 0000000000 0xDFFF = 110111 1111111111 110110 XXXXX XXXXX <-- high surrogate 110111 YYYYY YYYYY <-- low surrogate
31. UTF-16 Surrogate Pairs → ! - U+1F60E → Subtract 0x10000: 1F60E - 10000 = F60E → Split into 10 bits high and low: F60E = 0000111101.1000001110 = 003D.020E → Add D800 to high and DC00 to low: 003D + D800 = D83D 020E + DC00 = DE0E
32. UTF-16 Surrogate Pairs Surrogate pair is D83D DE0E
33. UTF-16 Surrogate Pairs Surrogate pair is D83D DE0E >>> "!"[0].toInt().toString(16) d83d
34. UTF-16 Surrogate Pairs Surrogate pair is D83D DE0E >>> "!"[0].toInt().toString(16) d83d >>> "!"[1].toInt().toString(16) de0e
35. UTF-16 Surrogate Pairs → Take care with string operations, especially with length() and substring(...) → StringBuilder.reverse actually works
36. UTF-16 Surrogate Pairs static int length(String s) { int len = 0; for (char c : s.toCharArray()) { // count surrogate pairs once len += Character.isLowSurrogate(c) ? 0 : 1; } return len; }
37. Multi-character Emojis → Regional symbols A-Z → ! + " = # → $ + $ = % → & + $ = ' → $ + & = (
38. Multi-character Emojis >>> "!".length 4
39. Multi-character Emojis >>> "!".length 4 >>> StringBuilder("!").reverse().toString()
40. Multi-character Emojis >>> "!".length 4 >>> StringBuilder("!").reverse().toString() "
41. Multi-character Emojis → Zero-Width Joiner (U+200D) → !+ZWJ+"+ZWJ+#+ZWJ+$ = %
42. Multi-character Emojis >>> "!".length 11
43. Multi-character Emojis >>> "!".length 11 >>> "!".substring(0, 8) "
44. Multi-character Emojis
45. Unicode versions → 6 major Unicode versions → ! (Person facepalming, added in 9.0) → (Zombie, added in 10.0)
46. ! added in 6.1
47. ! added in 6.0
48. Emoji One (Web) 1 EmojiCompat (Android) 2 2 https://developer.android.com/guide/topics/ui/look-and-feel/emoji-compat.html 1 https://github.com/emojione/emojione
49. TL;DR → Know your platform! → Take care with string operations → Use a compatibility library → https://emojipedia.org/
50. Thanks! ? twitter.com/madisp speakerdeck.com/madisp