$30 off During Our Annual Pro Sale. View Details »

Evolution of Emoji 🖼🔤✨

Evolution of Emoji 🖼🔤✨

A presentation on emoji history.
Presented at DurasnoConf in Durazno, Uruguay.

Part of the deck includes GIFs, you can see them in this tweet thread.
https://twitter.com/kosamari/status/1047978154143506434

Mariko Kosaka

October 04, 2018
Tweet

More Decks by Mariko Kosaka

Other Decks in Technology

Transcript

  1. (This is a slides from my talk I gave at DuraznoConf in Durazno, Uruguay. October 4th 2018)

    (Hand drawn illustrations are by me @kosamari, and each pictures used have credit at the bottom)

    Hello Durazno() Conf !

    View Slide

  2. I’m Mariko, you can find me as @kosamari on twitter.

    I’m a web developer form NYC working with team that builds Google Chrome.

    Say hi or send me questions!

    View Slide

  3. I usually write things like how does browser work & how to make compiler. With …

    View Slide

  4. … hand drawn illustrations like these.

    View Slide

  5. Because I really like learning about complex topics and explaining in a simple way.

    View Slide

  6. … so of-cause my compiler post has hand drawn pictures too :)

    View Slide

  7. Today, I want to tell you about emoji!

    Where it started, and how it evolved to be what it is now.

    View Slide

  8. But why? It’s a mix of many reasons why I’m interested in Emoji.

    I studied communication in university. I’m fascinated by how people comunicate especially using
    computer technologies and the internet.

    I also love history. Especially computing history. I can get lost in reading old mailing list archive or
    archival video.

    View Slide

  9. I love discovering cause and effects in history.


    Perhaps that’s the reason why I ended up in web development. I build tools for communication, by
    dealing with cause and effects everyday in code.

    View Slide

  10. So, emoji - Even though it might look like it comes from “emoticons”… it is actually a Japanese word.

    View Slide

  11. …which literally translates to picture letters.

    View Slide

  12. So how did emoji start?

    View Slide

  13. If you google “first emoji” you will find this one. MoMA actually acquired original emoji to their
    collection.

    View Slide

  14. This “original” emoji set is from i-mode - a Japanese mobile web network started in 1999

    View Slide

  15. So in 1999, you could buy a phone that look like this, which had a browser and email client.

    You could select emoji in your text message or if you are web developer, you could use emoji on your
    page.

    View Slide

  16. If you go to Computer History Museum in Mountain View, they describe i-mode as “first popular
    system for mobile Web access”

    View Slide

  17. Even thought i-mode was the first one that included a set of emoji. Emoji was used in communication
    device already. This Japanese version of iPhone (DP-211SW) had touch pad, supported Kanji
    characters, and emoji (spot the !).

    Also, pager was really popular device, from business person to high school girls. If you dial 88, certain
    pagers displayed — which was very popular.

    View Slide

  18. So in 1997, NTT DoCoMo starts i-mode project. 2 years before it’s release.

    View Slide

  19. To research for this new mobile web service, a project member was sent to California to try AT&T’s
    PocketNet service, which had mobile web access.

    Shigetaka Kurita - who was testing Pocket Net would send email from the PocketNet phone on a train
    in SanFrancisco, make international phone call to the office in Tokyo and ask if they received it. 


    View Slide

  20. While using PocketNet, like looking up weather, Kurita noticed that reading weather in text was rather
    dry… and wondered if he can use weather icons like they do in TV weather forecast.

    View Slide

  21. Kurita comes back home and pitch the emoji idea to phone device manufacturer.

    Manufacturer says “Fine, we’ll do it, but you have to tell us exactly what you want (in bitmap art)”

    So Kurita was tasked to design emoji set.

    View Slide

  22. But why emoji needed to be text and displayed as font? Why not image?

    Because back then i-mode had to have separate sockets - one for text and one for image, so that was
    not only slow to download but also used lot more data.

    Also, i-mode only supported GIF for image format. Back in late 1990s, patent of GIF was still not clear
    so they wanted to stay out of potential trouble.

    View Slide

  23. If icons are delivered as text (or code point) then they can save connection and data because it is
    delivered as text and pictogram is already installed in the phone.

    View Slide

  24. So at the end of the design process, this was the first emoji set.

    View Slide

  25. It has weather icons just like Kurita imagined while using PocketNet.

    View Slide

  26. Since they were designing information portal, Rock Paper Scissors were added for potential game use.

    View Slide

  27. Kurita says visualizing “location service” was difficult task but ended up with “a pin” which you can
    see later in services like Google Maps.

    View Slide

  28. Emoticons were added for expressive communication.

    View Slide

  29. And 4 different kinds of hearts !

    Kurita worked as sales person for pager (pocket bell) and experienced 1st hand some customers turn
    away from device because it didn’t support —. So this time, he made sure there were hearts included.

    View Slide

  30. Interesting one is Cars, original set had 2 different types of car. 

    Back then Kurita was really into snowboarding so regular town car and SUV for outdoor was different
    thing for him :)

    View Slide

  31. You might notice there is no religion or race/gender related icons in the original set.

    Kurita says “I purposefully did not include emoji related to religion, country, race, or gender to avoid
    future conflicts”. Perhaps because it was done in conservative place (NTT - DoCoMo’s parent
    company was THE biggest phone company which used to be owned by government). Perhaps it was
    great vision in designing features that has never existed before.

    View Slide

  32. While I was researching, 2 resources came up often as inspirations for emoji.

    View Slide

  33. 1st is Tokyo Olympic 1964. This was the first Olympic game hosted in Asia. Signs written in Japanese
    does not even use alphabets, so Olympic organizers needed to come up a way to communicate with
    international audience.

    They used pictogram for that. Each sports had its own pictogram (this became Olympic tradition, every
    Olympic game since then has their own pictograms.)

    View Slide

  34. Not only for type of sports, they also defined informational icons like bathroom sign and restaurant/
    cafe signs. This is a news paper reporting the design effort back in 1064.


    At the end of design process, designers abandoned their copyright & released pictograms they
    designed as public domain so that it can be used everywhere.

    View Slide

  35. So in original emoji, you see trace of that universal signs.

    View Slide

  36. Another inspiration is Japanese comic book (or Manga).

    View Slide

  37. Manga has it’s own visual idioms to communicate emotions. This might not look anything to you, but
    to me who grew up reading manga and immersed in this visual communication, I can see 3 different
    scene/emotions just by adding small visual elements.

    (surprised, anxious (cold sweat), sad/devastated (shadow))

    View Slide

  38. You can see those visual idioms in the original emoji set

    View Slide

  39. So when I read text like this, as Japanese who is trained in those visual idiom, I read the text in much
    more comical/expressive way than plain text.

    View Slide

  40. In a way, I think emoji is still under used for its original purpose.

    View Slide

  41. This is a “text only” NPR site created in response to hurricane.

    There are no image/video/ads so that people can read news even in the unreliable connection.

    View Slide

  42. I think this is a exactly the situation emoji can shine!

    Emoji can add more visual flare without adding any more data (because emoji fonts are already
    installed in the device). Plus, don’t you think it’d be nice to have more visual differentiator when you are
    in the stressful mental state, in stead of trying to parse wall of text?

    View Slide

  43. Let’s look at what happened after original emoji was introduced.

    View Slide

  44. Other phone companies followed DoCoMo’s success and provided their own emoji for their service.

    View Slide

  45. In fact, emoji became a competitive aspects of phone service.

    Companies advertised “we have more emoji!” Or “our emoji animates!”. In result, a lot more emoji
    were introduced to the market.

    View Slide

  46. So let’s look at how emoji was handled on encoding level.

    View Slide

  47. In many encoding system our phone or computers turn data they received into a reference (like code
    points) and use that to look up letters.

    ASCII or Central European encoding has been used by languages that use alphabets, for Japanese we
    have one called Shift-JIS that supports a lot more characters that we use in Japanese.

    Shift-JIS also have empty space left for application specific use, so emoji used this space.

    View Slide

  48. Things to note here is that each phone company defined their own emoji system. There were no
    interoperability.

    View Slide

  49. So back in 2000 if you send emoji from Phone company A to B, emoji would turn into ˤ(geta). In
    modern days it might be □(tofu).

    View Slide

  50. Finally in 2006, three major phone companies in Japan made it possible to use emoji interchangeably.

    View Slide

  51. It was done as conversion table system.

    View Slide

  52. And unfortunate bug (like sending music note emoji turn into poop) occurred.

    View Slide

  53. Conversation table system is not really optimal, each company has to maintain table and if new
    company comes into market, they have to create new table again.

    Better approach may be having single table to look up to (this will come in later!)

    View Slide

  54. To recap, emoji was release to public in 1999, gained popularity in Japan in 2000s.

    View Slide

  55. Next comes globalization phase.

    View Slide

  56. While emoji mature in Japanese market, it started gaining adoption outside of Japan.

    View Slide

  57. Gmail supported emoji input in 2008.

    This is also the time Google was trying to break into Japanese market with gmail. To get adoption in
    Japan as communication service, emoji support was a kay.

    View Slide

  58. iPhone first supported emoji only in Japanese market with iOS 2.2.

    It is rumored Masayoshi Son, CEO of Softbank which had exclusive rights to sell iPhone in Japan,
    asked Steve Jobs to support emoji in iPhone in order to sell them in Japan.

    View Slide

  59. With those activities happening, Google and Apple delegations decides to propose emoji to Unicode.

    View Slide

  60. Unicode is a project to document every letters for every language in one place. 

    Systems that support Unicode can reference one master source. (no more encoding differences).

    View Slide

  61. Here is a screen shot from the proposal.

    Of-cause, some people questioned if emoji really need to be included in Unicode.

    View Slide

  62. Rationale for including emoji in Unicode was

    1. Interoperability

    2. Already done same for ARIB


    ARIB symbols are Japanese broad cast specific symbols which are brought into Unicode before emoji.

    View Slide

  63. When you read mailing list discussion from back then, you see discussion like “do you really need 4
    different kind of books?” “Do you really want to include these racial stereotype emoji?” “why don’t my
    country have emoji flag?”.

    View Slide

  64. Flag is an interesting one.

    It was proposed as FLAG_SYMBOL_[country] but flag designs could change. So next proposal was to
    change to EMOJI_SYMBOL_[country] and let implementers decide how to display them. But country
    could also change, so it was settled as combination of 2 REGIONAL_INDICATOR_SYMBOLS. (visual
    representation is still up-to implementers. In reality mostly displayed as flag).

    View Slide

  65. Because Israel’s country code is IL and Liechtenstein’s country code is LI. If you put array of Israel flag
    emoji in code editor and hit back space, you see Liechtenstein flag show up.

    View Slide

  66. Flag and country is a highly political subject. Recently there was a bug which crashed iPhone if iPhone
    user with region set to China received Taiwanese flag.

    Since Taiwan is not recognized as country in China, Apple added system to mask Taiwanese flag if
    Chinese user received Taiwanese flag, unintentionally this caused the bug that crashed iPhone entirely.

    View Slide

  67. Another interesting one from this time is this document made by Japanese publisher explaining
    Japanese comic expressions.

    View Slide

  68. They noticed proposed emoji example (in orange line) was loosing original meaning. They explained
    the manga visual idioms and proposed change (in green line).

    View Slide

  69. In the process of defining emoji for Unicode, skin tone modifier was added to support diverse skin
    colors.

    View Slide

  70. In order to display emoji with different skin color, a default emoji and skin tone modifier are sent as set.
    It is then displayed as one emoji if the system supports them or 2 (default emoji + skin tone) if not
    supported.

    View Slide

  71. Ways to create compound emoji was developed in order to create more emoji from existing emoji.

    View Slide

  72. Compound emoji use thing called zero width joiner which glues multiple emoji into one.

    View Slide

  73. A family is constructed from 4 emoji…

    View Slide

  74. … so if you type family emoji and hit backspace in code editor, you can see emoji changes, deleting
    one family member at a time.

    View Slide

  75. Two ladies kissing is constructed with two women, a heart, and a kiss emoji.

    View Slide

  76. Rainbow flag is made of plain flag and rainbow emoji.

    View Slide

  77. This is how emoji with different profession are made.

    View Slide

  78. Lady computer programmer is Woman + Computer

    View Slide

  79. Lady mechanic is Woman + Wrench

    View Slide

  80. Lady justice is Woman + Scales

    View Slide

  81. To recap, emoji adoption in different services caused emoji to be included in Unicode which brought
    emoji to more universal icon set.

    View Slide

  82. So what’s the future of emoji?

    View Slide

  83. Probably the most interesting thing about emoji is that you get to help define new emoji. I think it’s the
    only language in Unicode that is actively developed in stead of just documenting existing letters.

    View Slide

  84. Unicode has a process to propose new emoji. There is a guideline so you can’t just add your brand’s
    icon as emoji, but if you think certain emoji would be universally useful, you can make your own
    proposal.

    View Slide

  85. For example, Apple proposed accessibility emoji set that includes guide dog and person with cane. It
    is now a draft candidate so this might make it to official unicode someday!

    View Slide

  86. This is how person with headscarf (Hijab) emoji was added. A lady who wear hijab thought her
    community should be represented in emoji so she contacted Unicode.

    View Slide

  87. Dumpling or Empanada depending on where you are, was also added this way.

    View Slide

  88. Salt shaker emoji is in newest version of Unicode and coming to device near you soon!

    View Slide

  89. And that’s muy salado right? (I was told in Uruguay Spanish muy salado is a slang for “very good”).

    Thank you for listening! If you have any questions, comments, concerns, please send me tweet at
    @kosamari

    View Slide