Evolution of Emoji 🖼🔤✨

Slide 1

Slide 1 text

(This is a slides from my talk I gave at DuraznoConf in Durazno, Uruguay. October 4th 2018) (Hand drawn illustrations are by me @kosamari, and each pictures used have credit at the bottom) Hello Durazno() Conf !

Slide 2

Slide 2 text

I’m Mariko, you can ﬁnd me as @kosamari on twitter. I’m a web developer form NYC working with team that builds Google Chrome.  Say hi or send me questions!

Slide 3

Slide 3 text

I usually write things like how does browser work & how to make compiler. With …

Slide 4

Slide 4 text

… hand drawn illustrations like these.

Slide 5

Slide 5 text

Because I really like learning about complex topics and explaining in a simple way.

Slide 6

Slide 6 text

… so of-cause my compiler post has hand drawn pictures too :)

Slide 7

Slide 7 text

Today, I want to tell you about emoji! Where it started, and how it evolved to be what it is now.

Slide 8

Slide 8 text

But why? It’s a mix of many reasons why I’m interested in Emoji.  I studied communication in university. I’m fascinated by how people comunicate especially using computer technologies and the internet.  I also love history. Especially computing history. I can get lost in reading old mailing list archive or archival video.

Slide 9

Slide 9 text

I love discovering cause and eﬀects in history.   Perhaps that’s the reason why I ended up in web development. I build tools for communication, by dealing with cause and eﬀects everyday in code.

Slide 10

Slide 10 text

So, emoji - Even though it might look like it comes from “emoticons”… it is actually a Japanese word.

Slide 11

Slide 11 text

…which literally translates to picture letters.

Slide 12

Slide 12 text

So how did emoji start?

Slide 13

Slide 13 text

If you google “ﬁrst emoji” you will ﬁnd this one. MoMA actually acquired original emoji to their collection.

Slide 14

Slide 14 text

This “original” emoji set is from i-mode - a Japanese mobile web network started in 1999

Slide 15

Slide 15 text

So in 1999, you could buy a phone that look like this, which had a browser and email client. You could select emoji in your text message or if you are web developer, you could use emoji on your page.

Slide 16

Slide 16 text

If you go to Computer History Museum in Mountain View, they describe i-mode as “ﬁrst popular system for mobile Web access”

Slide 17

Slide 17 text

Even thought i-mode was the ﬁrst one that included a set of emoji. Emoji was used in communication device already. This Japanese version of iPhone (DP-211SW) had touch pad, supported Kanji characters, and emoji (spot the !).  Also, pager was really popular device, from business person to high school girls. If you dial 88, certain pagers displayed — which was very popular.

Slide 18

Slide 18 text

So in 1997, NTT DoCoMo starts i-mode project. 2 years before it’s release.

Slide 19

Slide 19 text

To research for this new mobile web service, a project member was sent to California to try AT&T’s PocketNet service, which had mobile web access. Shigetaka Kurita - who was testing Pocket Net would send email from the PocketNet phone on a train in SanFrancisco, make international phone call to the oﬃce in Tokyo and ask if they received it.  

Slide 20

Slide 20 text

While using PocketNet, like looking up weather, Kurita noticed that reading weather in text was rather dry… and wondered if he can use weather icons like they do in TV weather forecast.

Slide 21

Slide 21 text

Kurita comes back home and pitch the emoji idea to phone device manufacturer. Manufacturer says “Fine, we’ll do it, but you have to tell us exactly what you want (in bitmap art)” So Kurita was tasked to design emoji set.

Slide 22

Slide 22 text

But why emoji needed to be text and displayed as font? Why not image? Because back then i-mode had to have separate sockets - one for text and one for image, so that was not only slow to download but also used lot more data. Also, i-mode only supported GIF for image format. Back in late 1990s, patent of GIF was still not clear so they wanted to stay out of potential trouble.

Slide 23

Slide 23 text

If icons are delivered as text (or code point) then they can save connection and data because it is delivered as text and pictogram is already installed in the phone.

Slide 24

Slide 24 text

So at the end of the design process, this was the ﬁrst emoji set.

Slide 25

Slide 25 text

It has weather icons just like Kurita imagined while using PocketNet.

Slide 26

Slide 26 text

Since they were designing information portal, Rock Paper Scissors were added for potential game use.

Slide 27

Slide 27 text

Kurita says visualizing “location service” was diﬃcult task but ended up with “a pin” which you can see later in services like Google Maps.

Slide 28

Slide 28 text

Emoticons were added for expressive communication.

Slide 29

Slide 29 text

And 4 diﬀerent kinds of hearts ! Kurita worked as sales person for pager (pocket bell) and experienced 1st hand some customers turn away from device because it didn’t support —. So this time, he made sure there were hearts included.

Slide 30

Slide 30 text

Interesting one is Cars, original set had 2 diﬀerent types of car.   Back then Kurita was really into snowboarding so regular town car and SUV for outdoor was diﬀerent thing for him :)

Slide 31

Slide 31 text

You might notice there is no religion or race/gender related icons in the original set. Kurita says “I purposefully did not include emoji related to religion, country, race, or gender to avoid future conﬂicts”. Perhaps because it was done in conservative place (NTT - DoCoMo’s parent company was THE biggest phone company which used to be owned by government). Perhaps it was great vision in designing features that has never existed before.

Slide 32

Slide 32 text

While I was researching, 2 resources came up often as inspirations for emoji.

Slide 33

Slide 33 text

1st is Tokyo Olympic 1964. This was the ﬁrst Olympic game hosted in Asia. Signs written in Japanese does not even use alphabets, so Olympic organizers needed to come up a way to communicate with international audience. They used pictogram for that. Each sports had its own pictogram (this became Olympic tradition, every Olympic game since then has their own pictograms.)

Slide 34

Slide 34 text

Not only for type of sports, they also deﬁned informational icons like bathroom sign and restaurant/ cafe signs. This is a news paper reporting the design eﬀort back in 1064.    At the end of design process, designers abandoned their copyright & released pictograms they designed as public domain so that it can be used everywhere.

Slide 35

Slide 35 text

So in original emoji, you see trace of that universal signs.

Slide 36

Slide 36 text

Another inspiration is Japanese comic book (or Manga).

Slide 37

Slide 37 text

Manga has it’s own visual idioms to communicate emotions. This might not look anything to you, but to me who grew up reading manga and immersed in this visual communication, I can see 3 diﬀerent scene/emotions just by adding small visual elements. (surprised, anxious (cold sweat), sad/devastated (shadow))

Slide 38

Slide 38 text

You can see those visual idioms in the original emoji set

Slide 39

Slide 39 text

So when I read text like this, as Japanese who is trained in those visual idiom, I read the text in much more comical/expressive way than plain text.

Slide 40

Slide 40 text

In a way, I think emoji is still under used for its original purpose.

Slide 41

Slide 41 text

This is a “text only” NPR site created in response to hurricane. There are no image/video/ads so that people can read news even in the unreliable connection.

Slide 42

Slide 42 text

I think this is a exactly the situation emoji can shine! Emoji can add more visual ﬂare without adding any more data (because emoji fonts are already installed in the device). Plus, don’t you think it’d be nice to have more visual diﬀerentiator when you are in the stressful mental state, in stead of trying to parse wall of text?

Slide 43

Slide 43 text

Let’s look at what happened after original emoji was introduced.

Slide 44

Slide 44 text

Other phone companies followed DoCoMo’s success and provided their own emoji for their service.

Slide 45

Slide 45 text

In fact, emoji became a competitive aspects of phone service. Companies advertised “we have more emoji!” Or “our emoji animates!”. In result, a lot more emoji were introduced to the market.

Slide 46

Slide 46 text

So let’s look at how emoji was handled on encoding level.

Slide 47

Slide 47 text

In many encoding system our phone or computers turn data they received into a reference (like code points) and use that to look up letters. ASCII or Central European encoding has been used by languages that use alphabets, for Japanese we have one called Shift-JIS that supports a lot more characters that we use in Japanese. Shift-JIS also have empty space left for application speciﬁc use, so emoji used this space.

Slide 48

Slide 48 text

Things to note here is that each phone company deﬁned their own emoji system. There were no interoperability.

Slide 49

Slide 49 text

So back in 2000 if you send emoji from Phone company A to B, emoji would turn into ˤ(geta). In modern days it might be □(tofu).

Slide 50

Slide 50 text

Finally in 2006, three major phone companies in Japan made it possible to use emoji interchangeably.

Slide 51

Slide 51 text

It was done as conversion table system.

Slide 52

Slide 52 text

And unfortunate bug (like sending music note emoji turn into poop) occurred.

Slide 53

Slide 53 text

Conversation table system is not really optimal, each company has to maintain table and if new company comes into market, they have to create new table again. Better approach may be having single table to look up to (this will come in later!)

Slide 54

Slide 54 text

To recap, emoji was release to public in 1999, gained popularity in Japan in 2000s.

Slide 55

Slide 55 text

Next comes globalization phase.

Slide 56

Slide 56 text

While emoji mature in Japanese market, it started gaining adoption outside of Japan.

Slide 57

Slide 57 text

Gmail supported emoji input in 2008. This is also the time Google was trying to break into Japanese market with gmail. To get adoption in Japan as communication service, emoji support was a kay.

Slide 58

Slide 58 text

iPhone ﬁrst supported emoji only in Japanese market with iOS 2.2.  It is rumored Masayoshi Son, CEO of Softbank which had exclusive rights to sell iPhone in Japan, asked Steve Jobs to support emoji in iPhone in order to sell them in Japan.

Slide 59

Slide 59 text

With those activities happening, Google and Apple delegations decides to propose emoji to Unicode.

Slide 60

Slide 60 text

Unicode is a project to document every letters for every language in one place.   Systems that support Unicode can reference one master source. (no more encoding diﬀerences).

Slide 61

Slide 61 text

Here is a screen shot from the proposal.  Of-cause, some people questioned if emoji really need to be included in Unicode.

Slide 62

Slide 62 text

Rationale for including emoji in Unicode was 1. Interoperability 2. Already done same for ARIB   ARIB symbols are Japanese broad cast speciﬁc symbols which are brought into Unicode before emoji.

Slide 63

Slide 63 text

When you read mailing list discussion from back then, you see discussion like “do you really need 4 diﬀerent kind of books?” “Do you really want to include these racial stereotype emoji?” “why don’t my country have emoji ﬂag?”.

Slide 64

Slide 64 text

Flag is an interesting one. It was proposed as FLAG_SYMBOL_[country] but ﬂag designs could change. So next proposal was to change to EMOJI_SYMBOL_[country] and let implementers decide how to display them. But country could also change, so it was settled as combination of 2 REGIONAL_INDICATOR_SYMBOLS. (visual representation is still up-to implementers. In reality mostly displayed as ﬂag).

Slide 65

Slide 65 text

Because Israel’s country code is IL and Liechtenstein’s country code is LI. If you put array of Israel ﬂag emoji in code editor and hit back space, you see Liechtenstein ﬂag show up.

Slide 66

Slide 66 text

Flag and country is a highly political subject. Recently there was a bug which crashed iPhone if iPhone user with region set to China received Taiwanese flag. Since Taiwan is not recognized as country in China, Apple added system to mask Taiwanese flag if Chinese user received Taiwanese flag, unintentionally this caused the bug that crashed iPhone entirely.

Slide 67

Slide 67 text

Another interesting one from this time is this document made by Japanese publisher explaining Japanese comic expressions.

Slide 68

Slide 68 text

They noticed proposed emoji example (in orange line) was loosing original meaning. They explained the manga visual idioms and proposed change (in green line).

Slide 69

Slide 69 text

In the process of deﬁning emoji for Unicode, skin tone modiﬁer was added to support diverse skin colors.

Slide 70

Slide 70 text

In order to display emoji with diﬀerent skin color, a default emoji and skin tone modiﬁer are sent as set. It is then displayed as one emoji if the system supports them or 2 (default emoji + skin tone) if not supported.

Slide 71

Slide 71 text

Ways to create compound emoji was developed in order to create more emoji from existing emoji.

Slide 72

Slide 72 text

Compound emoji use thing called zero width joiner which glues multiple emoji into one.

Slide 73

Slide 73 text

A family is constructed from 4 emoji…

Slide 74

Slide 74 text

… so if you type family emoji and hit backspace in code editor, you can see emoji changes, deleting one family member at a time.

Slide 75

Slide 75 text

Two ladies kissing is constructed with two women, a heart, and a kiss emoji.

Slide 76

Slide 76 text

Rainbow ﬂag is made of plain ﬂag and rainbow emoji.

Slide 77

Slide 77 text

This is how emoji with diﬀerent profession are made.

Slide 78

Slide 78 text

Lady computer programmer is Woman + Computer

Slide 79

Slide 79 text

Lady mechanic is Woman + Wrench

Slide 80

Slide 80 text

Lady justice is Woman + Scales

Slide 81

Slide 81 text

To recap, emoji adoption in diﬀerent services caused emoji to be included in Unicode which brought emoji to more universal icon set.

Slide 82

Slide 82 text

So what’s the future of emoji?

Slide 83

Slide 83 text

Probably the most interesting thing about emoji is that you get to help deﬁne new emoji. I think it’s the only language in Unicode that is actively developed in stead of just documenting existing letters.

Slide 84

Slide 84 text

Unicode has a process to propose new emoji. There is a guideline so you can’t just add your brand’s icon as emoji, but if you think certain emoji would be universally useful, you can make your own proposal.

Slide 85

Slide 85 text

For example, Apple proposed accessibility emoji set that includes guide dog and person with cane. It is now a draft candidate so this might make it to oﬃcial unicode someday!

Slide 86

Slide 86 text

This is how person with headscarf (Hijab) emoji was added. A lady who wear hijab thought her community should be represented in emoji so she contacted Unicode.

Slide 87

Slide 87 text

Dumpling or Empanada depending on where you are, was also added this way.

Slide 88

Slide 88 text

Salt shaker emoji is in newest version of Unicode and coming to device near you soon!

Slide 89

Slide 89 text

And that’s muy salado right? (I was told in Uruguay Spanish muy salado is a slang for “very good”). Thank you for listening! If you have any questions, comments, concerns, please send me tweet at @kosamari