AI transcript
0:00:05 Hi everyone, welcome to the A6NZ podcast. I’m Sonal. This week we’re resurfacing one of our
0:00:10 favorite episodes of all time, all about emojis. It’s timely not just because a whole new set
0:00:15 of emojis are about to hit many people’s phones soon, but because the episode, which was actually
0:00:21 first recorded in 2015, is really an evergreen and broader conversation about how innovation comes
0:00:27 about. Because we cover the tension between open standards versus close and proprietary systems,
0:00:31 the economics of creativity, the governance and politics of design, and much more.
0:00:35 We begin with the brief tour of different emoji and how they came about,
0:00:39 where emoji fit in the taxonomy of visual and social communication,
0:00:43 and why that matters, especially when it comes to machines reading emotions.
0:00:48 And finally, we talk about the ambiguities and difficulties to translating emoji
0:00:53 when it’s not really meant to be a language. Our guests are Fred Benenson, who was a second
0:00:58 employee at Kickstarter and VP of their data. He also famously kickstarted a project to translate
0:01:04 Moby Dick entirely into emoji, and Jenny Aitley, former New York Times reporter who was a member
0:01:10 and is now vice chair of the Unicode Emoji subcommittee. She first led the effort to get
0:01:13 the dumpling emoji, which is where we briefly start this conversation.
0:01:20 I wasn’t a really big emoji user. In fact, the first time I ever heard of emoji was when
0:01:25 Fred started his Kickstarter called Emoji Dick, and I was like, “What the fuck are emoji?”
0:01:29 This is before they showed up on our iPhones, like perky little yellow faces.
0:01:31 I was like, “What? It sounds something very bizarre.”
0:01:35 I just started. I didn’t even actually just be blunt. I had a very hard time using emoji,
0:01:39 because I didn’t quite understand how to even, frankly, use that moment. I don’t understand
0:01:44 it when people send it to me, if it’s not the obvious heart, etc. But as I’ve been using it
0:01:48 more, I found myself sort of expressing myself now in kind of quirky ways, and I don’t know
0:01:50 if people really get it or not, but I’m getting a kick out of it.
0:01:52 That’s the fun of the ambiguity.
0:01:58 I have a friend who showed me an exchange between a friend of his who was dating a guy,
0:02:01 and he would only send her emoji, and she was like, “I just can’t handle this.”
0:02:05 And he showed me these screenshots of their exchange, and it was hilarious.
0:02:06 You’re helping translate.
0:02:07 But yeah, it’s so like, I was like, “Oh, this is-“
0:02:09 You looked a certain abridgerack in like emoji.
0:02:11 Yeah, I was like, “This is what this means.”
0:02:16 I can definitely see it being like sort of an irreconcilable difference between people and
0:02:21 relationships. I fast forward many, many years. Emoji have showed up on our iPhone,
0:02:25 and I’m texting with my friend Yiying Lu, who’s best known as the designer of the Twitter fail
0:02:29 whale. So we’re texting back and forth about like dumplings, and so I send her a picture of the
0:02:33 dumplings I’m making, and then she texts me back, knife and fork, knife and fork, yum, yum, yum, yum,
0:02:38 and she goes, “Wait, Apple doesn’t have a dumpling emoji.” I was like, “How could that be?”
0:02:43 I was like, because there’s so many obscure Japanese food emojis and emojis are from Japan,
0:02:50 like you have everything ranging from ramen to curry rice to tempura to like the rice thingies
0:02:54 on a stick. There’s even the like triangle rice ball that looks like it had a bikini wax.
0:03:00 Right, there’s also the fish cake, which is the white one with the purple sorrel in it.
0:03:06 And I was like, “How could there not be dumplings?” Because it’s such a universal food,
0:03:12 because there’s like pierogies in Poland and momos and gyoza and empanadas. It’s just like
0:03:15 a food from around the world. I mean, technically a samosa is a dumpling.
0:03:25 Samosa, ravioli. And I was like, “Okay, emoji are universal.” And then dumplings are universal.
0:03:31 How could there not be a dumpling emoji? And just in my mind, I was just like clearly whatever
0:03:34 system in place has failed. How do you solve a problem like the dumpling emoji?
0:03:38 Yeah, and I found out that emoji are regulated by the Unicode Consortium, which is a non-profit
0:03:45 organization based in Mountain View, California. It now has 12 full voting members that pay $18,000
0:03:50 a year just to vote on issues, including like emoji and other kind of like technical numbers.
0:03:51 Are all those numbers in Mountain View?
0:03:58 No, so of those 12, nine are U.S. multinational tech companies, Oracle, IBM, Google, Yahoo,
0:04:04 Adobe, Facebook, Microsoft, and Symantec. Then of the other three full voting members,
0:04:09 one is the German software company SAP, another is the Chinese telecom company Huawei,
0:04:13 and the last is the government of Oman. That’s a really interesting crew.
0:04:16 It’s an interesting crew. And they have these quarterly meetings.
0:04:20 And then I just shop. And they’re very welcoming. They’re like, “Thank you for coming. What brings
0:04:26 you here? Tell us about yourself.” It felt like showing up at church. Like a new church,
0:04:29 you’re a new member. They all knew each other very well. They’re very excited that there’s
0:04:35 like someone young and diverse that’s just like randomly shown up. And so I, in that process,
0:04:43 learn how you get emoji passed and how they’re regulated. And so in January of 2016, we submitted
0:04:48 a full proposal for dumplings, takeout box, chopsticks, and fortune cookies and got those
0:04:54 all passed. So those will be in Unicode 10, which means that that’s announced in June of 2017.
0:04:58 And so they’ll actually hit your phones several months after that. I was like,
0:05:02 “Wow, billions of keyboards will be impacted by this.” That’s amazing.
0:05:05 Were there other proposals submitted at the time? Oh, they’re constant proposals.
0:05:10 There’s this whole process that people like Jenny, some of them make it through.
0:05:14 It’s a lot of work. It does reduce some good, useful bars actually for making sure quality
0:05:16 gets through at some point. Yeah. And to their credit, the Unicode Consortium
0:05:22 has an amazing list of emoji criteria where they say, “Okay, here’s what we’re looking for
0:05:27 for emoji. It’s got to have like, you know, kind of a unique meaning in that it’s not covered by
0:05:31 other stuff, but it also should have like, you know, some ambiguity. So it’s not just like
0:05:35 literally one thing. It could be used in other contexts.” Also, there’s one of the more interesting
0:05:41 rules, which is no celebrities, deities, or logos. Whoa. The Easter Island head is kind
0:05:46 of a violation of that one, but that’s got its own story. A couple of years ago, with a big update,
0:05:51 the Easter Island head showed up in like the back of the travel section of emoji. And I was like,
0:05:55 “What is that doing there? Who’s traveling to Easter Island so often that they need to use
0:05:58 the Easter Island head emoji?” And it kind of just stuck in my mind. And then I started using it in
0:06:04 this kind of like, slightly culturally insensitive way to like reference some supernatural phenomenon
0:06:07 that I didn’t understand, right? Like if I was in a conversation with somebody and I was just like,
0:06:10 completely flummoxed, I’d just like, send that one. Yeah. It’s like your version of Bermuda Triangle.
0:06:14 Yeah. Yeah. I was just like, who knows? Stone face. Other people use it for like,
0:06:19 stoned, right? Like, there’s lots of combinations in there. The reason why it’s in there is that
0:06:25 there’s a statue in downtown Tokyo, I think it’s a Shibuya station that is called Moyai,
0:06:30 which is a name of just like, it’s a proper noun of that statue, which was made by an artist
0:06:36 that was like a reference to original Easter Island head. So it turns out Japanese teenagers use
0:06:42 this waypoint to meet each other. And so that’s how it ended up in Japanese cell phones. And that’s
0:06:46 why it ended up in emoji. The artist used this inspiration of Easter Island. The interesting
0:06:51 twist is that when you look at it on the iPhone, it doesn’t look anything like the statue in Tokyo.
0:06:56 At some point, Apple was like, we’re not going to make it like this Tokyo’s one. We’re going to do
0:07:03 the original one. Android, on the other hand, their Moyai emoji looks like the Tokyo station one.
0:07:07 So fascinating. I read a study, I actually included in our newsletter months ago, of someone
0:07:12 comparing how emojis look on different platforms and how it actually changes meaning. Because
0:07:15 you can actually think you’re sending one thing and you get something else.
0:07:19 That’s going to happen in any system that has standardization. Like, you’re going to try really
0:07:24 hard to make sure people hue to the specification. But you know, people do their own implementations
0:07:30 and things change. In fact, the whole reason why emoji are in Unicode was because you would send
0:07:35 your friend an emoji and then their cell phone would actually just render the incorrect one.
0:07:39 It could be so much worse. And the fact that there is a standard means that like,
0:07:41 you only get these like weird edge cases.
0:07:46 There’s still some interesting vestiges of like the different telcos between Apple and Google.
0:07:52 One was DoCoMo and the other one was SoftBank. So they’re basically, depending on who their partner
0:07:58 was locally, they kind of inherited those generations of emojis. For example, on Apple,
0:08:02 women with bunny ears is like two women dancing in kind of like a let’s party kind of way with
0:08:07 their bunny ears. Whereas on Android, it’s just the headshot of a woman with bunny ears.
0:08:10 And it’s referencing this slightly misogynist part of Japanese culture of
0:08:15 bunny woman, which is itself a reference to the Playboy bunny. And so like they were cocktail
0:08:20 waitresses working in nightclubs. That made its way into the Japanese set. And then so when it
0:08:23 came over to America, like I think Apple must have been like, let’s make this a little more fun.
0:08:27 One of the easiest things actually to get emoji pass is showing that a vendor uses it.
0:08:32 Another argument is for completion. This is actually why chopsticks got passed fairly easily
0:08:36 because we had like knife and fork. Oh, so you need completion of a set.
0:08:40 You need completion. So that’s you can tell a whole story like stringing together a bunch of
0:08:44 no, I just think that it’s like they’re engineers, right? You can’t have ABCDE and
0:08:50 actually, one of the weird issues is that they’re red, yellow, green, purple, blue,
0:08:55 hearts, not orange. So one of the big lobbying efforts has been to fill in the orange. So the
0:08:59 case of the Apple bunny ears and the Japanese bunny women, that was a case where there was an
0:09:03 intentional translation translation to sort of obscure the cultural reference. More than that.
0:09:08 They often try the map technically the same emoji, but it’s like rendered and sort of interpreted
0:09:12 differently. They like emoji that can have multiple meanings. You can also just have like
0:09:16 emoji to have one meaning, but it really has to be a really good one was going to be one meaning.
0:09:21 So for us, the Chinese takeout box, for example, one of the arguments that we made,
0:09:26 we made is that it’s, it’s one, it’s an iconic shape. It also symbolizes both an entire cuisine,
0:09:30 which is Chinese food, and also a means of eating, which is delivery and takeout, right?
0:09:36 And so, so in that one symbol, you get a lot of sort of secondary meaning. And with fortune
0:09:40 cookies, like it’s technically a cookie, but it also means like mysterious in the future and the
0:09:44 unknown and like so like so our primary secondary meaning one of the criteria for an emoji to get
0:09:49 past is that it has to have a certain element of ambiguity to it. So I love this. I’ve been thinking
0:09:53 about this so much. When I did emoji deck, it was more of an experiment around crowdsourcing
0:10:00 an emoji itself. Like I wasn’t like so much interested in making a formal case that emoji
0:10:04 could be a language because it was still so early. Could it get there maybe one day? Yeah.
0:10:08 But Unicode makes a really good point. They’re like, emoji’s not a language. It shouldn’t be a
0:10:13 language. The value is that it’s ambiguous. And I’ve really come around to that thinking and this
0:10:17 idea that the charm of sending an emoji is that it can be interpreted in a couple of different
0:10:22 ways. And, and that’s actually why we value it. And, and I’ll go further and say that a lot of
0:10:26 people ask me why emoji have become so popular. And I think it’s tied to the fact that we now
0:10:33 are just inundated with text. We live in a text culture, right? We, we communicate via text.
0:10:41 Our careers are run over email. We read constantly. Everything we do is mediated through almost
0:10:47 literal words. And so emoji represents this kind of reaction to that. And the popularity of emoji,
0:10:51 I think, is largely due to the fact that we need some other way of expressing ourselves over text.
0:10:57 If the pipes are so mechanical, like phones and machine, you no longer have the nonverbal
0:11:01 aspects. So this is actually replacing sort of this, this human element of the glimmer in your
0:11:05 eye or like the cheeky, the blush on your cheek. There’s an emoji that does that.
0:11:09 You think about the amount of signal you get from somebody’s voice on an analog telephone.
0:11:12 And when you strip that out and all you’re communicating is like LOL, you don’t actually
0:11:18 know how sincere that laugh is or that chuckle or whatever that person’s trying to convey.
0:11:23 And so emoji gives us a much bigger palette to convey this kind of like extra like limbic
0:11:27 meaning that we want to have in our, in our communications, but we, we can’t because we’re
0:11:31 just, we’re texting all the time. So to break down the taxonomy of figural representation,
0:11:35 not using literal text, let’s talk about where emoji fits. We have emoticons,
0:11:41 which are like a colon and a parentheses and that gives you a smiley face or like a semicolon and
0:11:45 a parentheses and that gives you a wink. Right. Using punctuation. Using punctuation is emoticon.
0:11:48 Often ascii-ish. Right. Because it’s not ascii art as well.
0:11:52 Some of the earliest references to emoticons go back to the 19th century as well where people,
0:11:58 yeah, yeah. People were using colons and dashes and parentheses to express like a wink. It goes
0:12:03 way back. It’s important to add in hieroglyphs and iconography. Other humans have had this idea
0:12:07 before, right? Like the, the medium and the technology is kind of like incidental.
0:12:11 I’m so glad you brought that up because it’s so important to not get caught up in technology time.
0:12:16 Well, technically technology includes like sticks and on stones. So that does go back in time,
0:12:22 but in the context of this machine web that we live in, then we have emoticons as part of the
0:12:25 taxonomy and then we have emoji. But how would you guys define emoji?
0:12:29 It’s Japanese. Drawing language. Emoji. I don’t know how to pronounce in Japanese,
0:12:34 but the Chinese. The emo is not for emoticon or emotion or anything. It’s just totally
0:12:41 coincidental. Wow. It’s hard not to just hue to the Unicode standard and say it’s the,
0:12:49 it’s the set of icons defined in Unicode that represent objects and nouns and actions and.
0:12:53 The way that I explain it to people is an emoji is a character, an emoji is something you can
0:12:58 put in the subject line of an email because it literally is text. So, so in the same way that
0:13:04 Unicode is kind of defined a standard to unify all the graphical representation of different
0:13:08 languages throughout the world and even non languages. So like, you know, the windings and
0:13:12 all that kind of stuff. Emoji actually slip into that entire system. So there is literally
0:13:19 what they would call a code point assigned to each emoji. Or sorry, not every single one because
0:13:24 now they’re like compound emoji, but there are code points assigned to emoji, which basically
0:13:30 says, you know, when you, when a computer sees this code point, they render it in a certain way.
0:13:34 But that, that it’s important to kind of wrap your head around what’s actually happening inside
0:13:39 the computer because the emoji is being sent as text. If your computer supports UTF-8, UTF-16,
0:13:45 that’s just like a standard way for your computer to handle text, whether it’s your phone or your
0:13:50 laptop, then it’s being told render this emoji. But it’s actually up to your computer’s operating
0:13:55 system, whether it’s OSX or iOS or Android or whatever, to go fish out a little image and put
0:14:00 it on your screen. And so that image is actually controlled by the hardware manufacturer or the
0:14:05 software manufacturer, you know, when it’s actually rendered on your screen, the operating systems
0:14:09 choosing which image to show you. And those images are actually stored, you know, in the same way
0:14:13 that other images are stored on your computer as little PNG files. And so Apple, you know,
0:14:17 puts those on your computer and your computer chooses to render those, which is why you may get
0:14:21 slightly different, you know, interpretation. Right. I’m glad this is actually really interesting
0:14:27 because recently Facebook just introduced their own emoji and that like basically hijack Apple
0:14:31 emoji. So you can turn that honor off, but essentially they’ve, they will replace, they’ll
0:14:35 swap out all the ones on the app. And Twitter’s had their own set for a while. And so why is that?
0:14:39 So they’re interesting copyright considerations here. My guess is a lot of those companies are
0:14:44 doing it because, A, they can afford to make their own set. B, they want to avoid the legal
0:14:50 liability of using Apple set. And C, like they think they might kind of have some like moment of
0:14:55 like, Hey, did you see Twitter’s new emoji? Right. And so they’re, you know, these large companies
0:15:00 are kind of innovating on emoji. Yeah. Yeah. Like re-innovating and re-illustrating their emoji.
0:15:04 And I think, you know, I think Microsoft actually just evolved to a new set or wasn’t
0:15:08 Android. I think it might have been Google Android. They, they just upgraded to make it
0:15:11 seem a little bit more normal. Like they had gone from like- That’s just not a
0:15:15 the terrible blue and white. Yeah. Yeah. So it’s like the blobby ones or-
0:15:19 Yeah, the Android, I think Google had blobby ones for a while. Now they’re doing somewhat normal ones.
0:15:26 Scariest emoji ever, the Microsoft emoji are like blue and gray. And they look like monsters
0:15:29 that hide underneath your bed. Why? Why are they blue and gray? I think it’s just like-
0:15:33 It doesn’t attempt to be like, like different from like a yellow skin tone.
0:15:37 Well, also, you have to, part of the original emoji is you wanted things that were
0:15:43 skin tone neutral. So Apple and Google chose yellow, but Microsoft for some reason chose gray.
0:15:47 Oh, gray. Cause I was going to say for Hindu, like blue is actually not a bad thing to have your skin
0:15:51 blue. It’s like a god. The other thing is if you have your own set of emoji, you can actually
0:15:58 start adding to that set without going through the unicode. So like a very good example is the
0:16:02 gay family emoji originally where they’re not, it’s not actually one emoji. You know,
0:16:09 the one where it’s like man, man, kid, kid. That is actually a compound emoji of four characters
0:16:14 glued together using something called a quote zero with joiner, which is basically like an invisible
0:16:20 glue. So if you are sending that emoji to someone else who doesn’t have the ability to render that,
0:16:25 it actually unravels itself into like a multiple character. Now what you’re seeing is a lot of
0:16:31 vendors making compound emojis. So like actually one of the places where this is being debated
0:16:34 for use is the need for a professional female emoji, right? Cause one of the big problems right
0:16:40 now on the existing set of women as represented by emoji is like, there are only like really
0:16:45 four roles for women to play compared to men. You know, men, you can be a sleuth or you can be,
0:16:51 you know, a policeman. You can be sort of a medical worker. You can even be Santa Claus,
0:16:58 but as a woman, the four things you can be as a role are basically bride, princess, dancer,
0:17:02 ploy by bunny. That’s it. Oh my God. It just goes to show you how the, I mean, of course,
0:17:06 this is the politics of human life play out in these systems. I mean, the perfect example I was
0:17:11 thinking of is a rifle emoji. And the case of, I believe Apple, Google and Facebook,
0:17:16 Charlie Warzel at Buzzfeed wrote a really detailed article investigating this and about how they
0:17:20 sort of help suppress as part of the Unicode consortium, the rifle emoji. Right. Emoji already
0:17:25 has a gun in it, right? And it’s like, okay, so how many more versions of that do we need? And
0:17:30 you’re right, it’s absolutely a political topic. I mean, that issue manifests itself in so many
0:17:35 other places in emoji. The country flag stuff is super interesting because that uses kind of what
0:17:39 Jenny’s talking about with these compound emojis. Unicode didn’t actually want to decide which
0:17:44 flags were and weren’t in emoji. So they did. Right, because you’re legitimizing then political
0:17:49 issues. What they did was they built this kind of like meta country system so that you would
0:17:54 actually be pairing these country letter emojis together. So CNN would go together and then
0:17:59 it would be up to your phone to decide if you showed the Chinese flag. They pushed that decision
0:18:03 making, that like political decision making of which flags to support off to the handset
0:18:08 manufacturers. Microsoft actually does something weird there. What do they do? They just show the,
0:18:12 they don’t show a flag. They show a flag plus the two letters. Right, right. Microsoft doesn’t render
0:18:17 it normally. Yeah, to the point about politics being kind of embedded in emoji. It’s not just
0:18:23 because these are icons that represent the parts of our lives that we feel passionate about. It’s
0:18:27 because there’s a finite palette. It’s not like language where you can only, you know, you can
0:18:31 kind of combine say whatever you want. It’s combinatorial. You can take multiple combinations
0:18:35 and turn into whatever you want. You get way more degrees of freedom to kind of express yourself.
0:18:39 There’s a finite number of food items that are ready to go in there. And when you think about the
0:18:43 vast like multitudes of humanity, whether it’s, you know, people’s relationship status or structural
0:18:48 orientation or skin color, it’s like, like emojis never going to be able to express that. And so
0:18:54 like how do you contain this thing that’s like growing and kind of has to grow as more and more
0:18:59 people use it, but also by definition has to be a finite list of icons. Well, how do they handle
0:19:03 the skin tone issue? Because one of the things that I noticed is that an apple, because I use
0:19:08 an Android, so I didn’t notice this, you can press down on a thumbs up, for example, and then you
0:19:14 can pick among 15 different shades to like pick a skin coat shade that’s closest to you. It’s based
0:19:20 on a Fitzpatrick skin tone. Yeah, it’s actually used, it’s the same skin tone system that dermatologists
0:19:24 use to categorize. This reminds me a little bit of being a kid when like you had Crayola box,
0:19:28 I remember that the only shade you had, there was like a nude shade or like a skin tone. Yeah,
0:19:33 nude was always Caucasian. I remember using sepia to represent my skin color. I mean,
0:19:37 there’s a great history about this in, this is going to sound weird for me to say, but like women’s
0:19:42 pantyhose like had this issue where nude was always considered Caucasian. And people were like,
0:19:47 this is ridiculous. It was one of the earliest blind spots of emoji, I remember. Right. Well,
0:19:51 I mean, if you have like only white men designing them, do you remember when Slack, there was this
0:19:56 guy who wrote a post about Justin Brown, and I remember it was so meaningful because it’s such a
0:20:02 minor seemingly arbitrary thing. But then it is true, like the first time I saw that I could find
0:20:09 my skin color in a system and to be able to use it was kind of amazing and empowering. And I’d
0:20:13 think there’s something significant about that. I would totally agree. I don’t share your experience
0:20:19 as that as the person on the other side. And so it’s funny for me because I don’t,
0:20:29 he’s a white male. Yeah. I don’t share that like sense of identification with the
0:20:38 bright white skin. Right. I’m like, it feels odd to opt into that, which speaks to my privilege
0:20:42 as a white male. I mean, if you’re not exposed to it, you’re not exposed to it. The bottom line is
0:20:45 if you’re any person of color, you’re always aware of your color. Especially if you’re in a
0:20:49 context where everyone else is not the same color as you. And so when I texted my friends who are
0:20:53 not white and I’m like, should I be choosing that one? And I just choose to choose the yellow
0:20:59 skin tone. That’s just like the way more comfortable. To my solution is I often send four. It’s like,
0:21:05 it’ll be like yellow, light, dark, and then like the beige one. It’s like a Benetton add emoji.
0:21:11 Emoji, that’s fabulous. So now the kind of evolution is that we have yellow for like all the
0:21:15 human face characters, and then you can choose skin tones for some of them. But it doesn’t get at
0:21:20 like more nuanced issues about like cultural and racial identity, having to do a facial structure
0:21:24 or hairstyle. Oh, right. That’s a great point, actually, because one of the pet peeves I have
0:21:28 is when I used to go to foreign countries and look at billboards, it always glorified that
0:21:32 aquiline knows the face structure, whereas there’s a totally different type of face structure in
0:21:38 different areas. Emoji probably won’t ever have that amount of like customization and Unicode gets
0:21:45 this. And they actually say like, we’re adding like 60 emoji a year. This is fun, sustainable.
0:21:50 We feel like the future is inline images. And that kind of breaks my heart as like kind of a,
0:21:55 you know, nerd standardization guy who like, who really appreciates all the hard work that went
0:22:00 into Unicode and the idea that it is a standard, because if you’re just sending inline images
0:22:05 forever, then like, you know, you have no idea what’s going to be on the other side if they can
0:22:10 render the image. So stickers, I mean, so Kim Emoji, for example, Kim Kardashian’s quote Emoji,
0:22:16 they’re not actually emoji, those are just stickers or images that you can text back and forth.
0:22:20 But, you know, again, you know, standards, can you put it in the subject line email and those
0:22:24 you can’t you can’t so that therefore they don’t qualify. So they’re not technically emoji. Right.
0:22:28 So then going back to our hierarchy, we went from emoticon to emoji and now stickers. Stickers are
0:22:33 basically inline images. I mean, stickers are just images that you can pick from a palette or
0:22:37 like, and I think you can, you know, in certain apps, you can like apply a sticker to an image
0:22:41 that it like sits on top of it. But you’re then in this kind of like proprietary ecosystem of
0:22:46 that’s okay. But like, you think about the stuff that really works and the stuff that
0:22:51 really changes the future of the web and communication, it’s all standardized.
0:22:55 You’re saying this as a standardization person, because my friend Connie, who wrote a wonderful
0:23:01 post on the topic of stickers, argues that emoji are very limited for what you need to do because
0:23:05 she feels that you have so much more expression and the ability to convey so much more with
0:23:09 stickers than you do with emoji. Emoji doesn’t preclude the use of stickers. There is some
0:23:17 subset of images that are universal enough that should be hardwired into the operating systems
0:23:24 and are basically can be cross-platform that an iOS device can talk to, you know, Microsoft windows
0:23:29 and can talk to like an Android device can talk to your Mac laptop. Like the fact that at least
0:23:33 there is you’re not going to get little square boxes as long as your operating systems are
0:23:37 fairly up to date. Well, that goes to your point about why standardization is important,
0:23:42 because you’re now giving up that you’re in this proprietary ecosystem like WeChat or Line,
0:23:45 and you only have their stickers set and you can’t always transfer all these stickers across.
0:23:49 I mean, also, if you think about the accessibility issues around stickers, right, like people using
0:23:53 screen readers, they’re not going to be able to interpret an image and like emoji actually have
0:23:59 names. And so in theory, there’s much better accessibility for emoji for somebody who’s visually
0:24:07 impaired. Like for example, last year, Oxford English Dictionary chose Facebook Tears of Joy,
0:24:14 which I always thought looked very sad. The thing with the eyes and it’s like balling, but
0:24:18 that’s actually Face of Tears of Joy. And that is how you know that because, you know, all these
0:24:23 emoji have. They say the label. Oxford put that in there. So the word of the year was an emoji.
0:24:27 Part of the reason they chose that was that it ended up as number one on my friend’s site called
0:24:31 Emojitracker.com. Oh, right. That’s right. The Emojitracker, which tracks all the use of emoji
0:24:35 on Twitter. And for a while, it was just like, it was like the heart emoji or something or just
0:24:39 the smiling face emoji. So I think it’s really interesting when the top emoji shuffle because,
0:24:44 you know, whenever you start texting with somebody who hasn’t used emoji before,
0:24:49 they’re like choosing like the safest ones. Going back to this idea of some of the companies
0:24:55 owning their own emoji and some of the proprietary open tension between standardization,
0:24:59 freedom of expression. What do you make of this notion that part of what we’re doing here is
0:25:04 essentially also creating a more machine-readable web in terms of emotional reading? Because
0:25:09 essentially you’re now adding a whole new layer where you can codify people’s emotions, sentiment,
0:25:12 in ways beyond just a black and white like don’t like. I’ve been thinking about this so much,
0:25:17 actually, and not in the context of emoji, but actually Facebook reactions. Yeah, me too. I used
0:25:20 to assign and edit top ads on this topic because it was very accessible. Yeah, I think it’s a really
0:25:25 interesting topic because if you look at traditional sentiment analysis in the data world,
0:25:29 it’s kind of a joke. You have to have training data, you have to know good cases.
0:25:33 Right. And just to interject for a moment, as someone who’s been tested a million of those
0:25:38 systems and can never find one that actually works for my needs, they’re so binary. You
0:25:42 don’t get anything useful and you’re not getting insight. One of the reasons there is that words
0:25:47 have these degrees of freedom. They can be used sarcastically and you would never know it based
0:25:53 on the semantics. And so traditional sentiment analysis is really broken because you’re using
0:25:58 these kind of like stale, rigid, semantic definitions. What’s really interesting about
0:26:03 Facebook reactions is you think you’re saying, “I love this thing,” or “I’m sad about this,”
0:26:09 or “I’m angry about this,” but what you’re actually doing in conjunction with that is giving Facebook
0:26:16 really great labeled data for sentiment analysis. That’s right. Machine readable data. That is a
0:26:22 coley grail of emotional sentiment understanding. When I was at Wired, I assigned a piece to a
0:26:28 sociologist, Evan Solinger, because I wanted to coin this phrase, the mood graph, because we have
0:26:33 an interest graph, social graph, all kinds of other graphs that link all these nodes and ideas,
0:26:37 and now to have a mood graph to essentially be able to put your pulse on someone’s mood,
0:26:41 something very finite yet constantly changing. It’s just a fascinating thing to be able to
0:26:46 codify this. The sentiment stuff generally coley started strongly with human face and body.
0:26:52 So, I think this is also why people agitate so much for emoji that look like themselves,
0:26:56 like the redheads, and people with beards, and people who are bald.
0:27:00 Or anyone with curly hair. People with curly hair relate to other people with curly hair.
0:27:04 And so, I think people really love seeing themselves represented in emoji, which is why
0:27:09 Bitmoji, which is highly, highly, highly customized stickers in sort of emoji spirit.
0:27:13 Oh, and my cousins and I use Bitmoji on WhatsApp all the time. I think there’s something really
0:27:19 symbolically important about Bitmoji, because you are putting yourself in it and conveying in
0:27:23 this sticker form. The fact that Snapchat bought it, I think, is really telling.
0:27:24 Oh yeah, for a hundred million dollars.
0:27:28 Right, especially given that they are changing this culture of how you express yourself through
0:27:32 your facial expressions with face swapping and filters. Connie and I made the argument that it’s
0:27:36 sort of like selfies, as selfies as a form of stickers. So, what we’re talking about the machine
0:27:40 readable is a little distinct than this, but it’s sort of an interesting idea.
0:27:44 I also think it ties into this slightly dubious notion of the Uncanny Valley,
0:27:47 where if you want to try to represent yourself and you want to have like configurability around
0:27:52 that, it needs to be kind of cartoonish for it to be believable. I think what we’re seeing with
0:27:56 Snapchat filters, and I don’t know if you guys have played with Snow yet. That’s like a,
0:27:59 it’s like, take Snapchat filters and just multiply them by a thousand. It’s like,
0:28:04 it’s like, just like amazing amounts of diversity around the amount of stuff you can put on your
0:28:11 face. It is this weird convergence on identity and emoji that’s kind of happening.
0:28:14 I agree. And in fact, this is going to be a little, sound like a little out of left field for a
0:28:19 moment, but the whole notion around the Chewbacca mask lady, when, you know, that was the most
0:28:23 popular Facebook live video ever, it got like unprecedented views. And there was simply a woman
0:28:27 who was trying on her Chewbacca mask in the car and she’s laughing and giggling about it.
0:28:32 And then she puts her mask on and then she takes it off and she laughs so uninhibitedly.
0:28:37 It’s insane. And I make the argument that what was so empowering because it was totally took off
0:28:42 for obvious reasons is not the fact that she was laughing so uninhibitedly. It’s a fact that
0:28:47 it took putting on and then taking off the mask for her to do that, which is a lot,
0:28:50 not unlike what happens with communication through these filters and being able to now
0:28:56 express yourself through these cartoon like ways. I mean, honestly, it takes me back to like theater
0:28:59 and like Shakespeare in like seventh and eighth grade. I remember having these like really intense
0:29:03 discussions about like what it is to put on a mask and what a mask represents about yourself.
0:29:07 It’s a very Cambilian idea, right? The Joseph Campbell like mask and the myth and the man.
0:29:10 You’re right. There’s a theater, I mean, that’s why people say improv is so interesting for any
0:29:14 career field. But I think that there is an interesting moment now coming together with
0:29:20 selfies, stickers, emoji, bitmojis all together where we do have this new emotional web coming
0:29:25 together. And using emoji as the first time I thought about this could be kind of like putting
0:29:34 on a mask over your words to convey to yourself this extra, this kind of additional layer,
0:29:37 this emphasis of your emotion that you otherwise might not get.
0:29:41 Okay. So going back to you writing an entire book in emoji and yet you’re saying that you’ve kind
0:29:46 of evolved into thinking that emoji is not necessarily language, but clearly it is a visual
0:29:51 language and it is a tool for communication. It’s not complete. So how did you translate that?
0:29:54 I mean, what were some of the trade-offs and decisions you made? And by the way,
0:29:58 for the audience, that book was like 2009 or that was like many years ago.
0:30:01 So what emoji space were you working off? Did you make them up? Like what’d you do?
0:30:06 So I’d gotten in a text from my college roommate whose wife is Japanese. He sent me an emoji and I
0:30:11 was like, what is that? They told me you could download like basically a Japanese app and it
0:30:17 would like awaken your iPhone to the emoji keyboard. It just spoke to me in the like,
0:30:21 like you have to hack the iPhone to get the special keyboard of like Japanese icons. And I
0:30:26 was like, oh my God, I want this so bad. I was like, this is amazing. I should write a book
0:30:29 in emoji. And I was like, oh, that’s a lot of work. And I don’t know if I can write a whole book
0:30:32 in emoji. And then I was like, well, maybe I can translate a book in emoji. I was like, okay,
0:30:37 what books would work? And I was like, well, it has to be in the public domain because I worked
0:30:41 a lot in like the copyright reform space. Nobody’s going to just like, let me translate their
0:30:44 book into emoji without a lot of effort. For a moment, I thought about the Bible and I was like,
0:30:48 that’s too obvious. What’s like, what’s like totally even more inappropriate.
0:30:55 So Moby Dick came to mind as like this, this like impossible book to trans to put into these
0:30:58 symbolic characters. As soon as I thought, I was like, no, I can’t do that. That’s crazy. And I
0:31:00 was like, that’s like too hard. Honestly, it’s a little bit like, I just came back from saying
0:31:04 Hamilton. And so it’s a little bit like the idea of putting a wrap to like the founding fathers.
0:31:08 That’s what I find so fascinating. I would say Hamilton was probably a mashup of mediums and
0:31:13 time and culture. And it’s like one of those things where you tell to somebody and they’re like,
0:31:16 you can’t do that. That’s crazy. And then you’re like, well, the fact that you just said that made
0:31:21 me want to do it. And so not only that, there are not one, but two whale emoji. Were there at that
0:31:29 time? No, there’s only the original, the cute one that kind of was battling the cute whale.
0:31:35 The second, I think it’s called sperm whale didn’t come up until later. So I was like, okay,
0:31:39 wow, that would be really interesting to do all of Moby Dick because it’s also like really long.
0:31:44 I mean, it’s 10,000 sentences. And okay, well, if I don’t want to do this, maybe hire somebody to
0:31:48 do this. And I was like, experimenting with mechanical Turk at the same time. I think it was
0:31:53 like one of the original Amazon web services. It was like, it would later become part of that
0:31:58 AWS umbrella. I remember people using it for research and stuff. Right. It’s still used for
0:32:02 research. It’s still being valuable for that. But a couple other people have done an experiment
0:32:09 here or there of using it off label. I had made a task of mechanical Turk just to ask Turk workers,
0:32:14 if you could ask anyone like to do anything on mechanical Turk, what would you have them do?
0:32:17 And they came up with this long list of stuff. And I don’t think translate a book into emoji
0:32:20 was one of them, but there’s some creativity out there. I was like, okay, I’m going to,
0:32:24 I’m going to try this thing, or I’m going to hire people to translate Moby Dick into emoji,
0:32:28 some portion of it and see if this works. So I did the first chapter and the results came back
0:32:33 and they were hilarious. They’re so good. What do you mean you did the first chapter? Like,
0:32:38 did they break it down word by word? So how do you capture that in emoji? So I decided I was
0:32:43 going to do it as on a per sentence basis. And that actually turned out to be one of the challenging
0:32:48 parts of the project was like, splicing sentences is actually like kind of like a classically hard
0:32:52 and all natural language processing problem. And so I kind of like figured out a hack to like
0:32:55 chop it up. And I wrote a lot of regular expressions to basically book into sentences.
0:33:00 But you decided basically the sentence was a unit of analysis, not a phrase in the task.
0:33:04 And you say, pick any of these emoji and then actually wrote my own little emoji picker because
0:33:09 these things didn’t exist at the time. I had gotten the emoji from a friend. He had reverse
0:33:17 engineered the iPhone SDK and basically hacked out the PNG files from the software kit to basically
0:33:21 have the raw emoji in image form. And so I took that and just made like a little JavaScript,
0:33:25 like HTML thing and you know, dump that into mechanical Turk and like came back and I was like,
0:33:30 hey, this works. And so I think the sentence that’s kind of like on the cover of the book,
0:33:35 if you go to the website, it’s like the website being Emoji Dick. Emoji Dick.com.
0:33:41 Call Me Ishmael is the first sentence of Moby Dick. And the emoji that the Turk worker chose was
0:33:49 telephone man with face, sailboat, whale emoji. That’s perfect. The rest of it was just like
0:33:53 indecipherable emoji nonsense. And some of the people were just like, all right, give me my
0:33:57 five cents. I’m going to click some random emoji. And other people just like clicked every single
0:34:03 emoji. So the plan became have people translate the same sentence multiple times. So you get
0:34:08 three different emoji translations for one sentence and then have another set of tasks
0:34:14 where people vote on the best, most appropriate translation. So like of the three, which one
0:34:18 got the meaning across the best. And I was like, oh, I’m just like getting really excited about
0:34:21 this. And I started doing the math on how much it was going to cost. And I was like, oh, it’s going
0:34:25 to be thousands and thousands of dollars. That summer I met the Kickstarter guys that started
0:34:28 talking with Andy Bayo. He was like, you should put on Kickstarter. So that night I went home and
0:34:32 put it on Kickstarter launch the next day and ended up working for them. And by the way, how
0:34:36 much is the campaign? How much money did the campaign make? My goal was like 3,500 and I ended
0:34:41 raising 3,700. So I worked on it for nights and weekends for another eight or nine months. And
0:34:46 then self-published it on Lulu.com. You can still buy it. It gets printed on demand. Do people still
0:34:53 buy it? I’ve sold like thousands of dollars of emoji deck. And I’d say hundreds of copies,
0:34:59 probably like five or 600 copies of it have sold since then, which is not a lot. I bet this podcast
0:35:02 is going to sell a bunch. Yeah, well. You better share some of the proceeds with me. Okay, so there
0:35:07 are two copies. One, there’s a black and white copy, which is like the easy to print one. And
0:35:12 that’s like $20 or $30. And then there’s the full color one, which like is obviously preferable
0:35:18 because emojis are so colorful. But when you’re printing on demand, 800 pages of color, laser,
0:35:24 hardbound copy, it’s actually really expensive. So that thing costs like $180.
0:35:26 Right, because you’re not printing in bulk. Because you actually save money when you’re
0:35:31 printing in bulk. So I have to sell that one for that much. And people still buy it. In 2013,
0:35:36 the Library of Congress contacted me and they said, “We would like to acquire emoji deck as our
0:35:40 first emoji book.” I was like, “Are you sure?” They’re like, “Yeah, yeah, we’re sure.” I was
0:35:46 telling a friend and David Gallagher, I think you must know from the times. And he’s like, “You
0:35:49 know, everyone submits their stuff to the Library of Congress. It’s not that big of a deal.” And I
0:35:53 was like, “No, man, they asked for it. Like they’re acquiring it.” I think it’s a big deal because
0:35:57 it’s a curatorial point of view. Totally. They’re saying, “This is a cultural moment. It’s not just
0:36:01 a book that was published. And we need to figure out how to acquire it.” I was like, “All right,
0:36:07 I’ll spare a copy. I signed it. I sent it to them.” And then they sent me this little certificate in
0:36:11 digital form. And what’s hilarious, and this is my favorite part, is that it’s somehow listed as a
0:36:15 translation of “Moby Dick.” So when you look up “Emoji Dick,” it says all these libraries have it
0:36:19 because it’s really just saying that they have a translation, they have the original “Moby Dick.”
0:36:22 Now it’s got a life of its own and people still discover it.
0:36:25 That’s amazing. I mean, you actually even created an art show. Did you base on this?
0:36:31 Friends of mine put together a kind of emoji survey art show and there were some really great
0:36:36 stuff. And their emoji tracker was there. There was a programming language built out of emoji.
0:36:40 I mean, emojis can have their URL. I mean, that’s another thing. They’re literally text. So you
0:36:44 can have like emoji, well, I don’t know, but you can have emoji in your email address.
0:36:47 Oh, you can. Oh, you can also buy emoji domains.
0:36:52 So you have an emoji book, you have emoji art shows, emoji hackathons, emoji hackathons.
0:36:57 So our big news this week is that in November in San Francisco, we are going to throw the
0:37:01 first ever emoji con. What? Is it like Comic Con?
0:37:05 It’s like Comic Con, but emojis. I really hope people show up dressed in emoji costumes.
0:37:08 Yeah, I’m going to show up as, oh, you guys are, Ying’s going to show up as a dumpling emoji for
0:37:13 sure. Or like, you know, poop emoji or like the ghost emoji. So it has many different
0:37:17 elements to it. So one is definitely sort of this whole emoji learn aspect where it’s like
0:37:22 panels and talks. And there’s a sort of emoji film festival. And then there’s an emoji hackathon.
0:37:27 And then there’s an emoji art show. And then of course, the opening party emoji where, you know,
0:37:31 our goals only have food that is also, also emoji. So why a conference? I mean, of course,
0:37:35 I see the cultural significance, but to bring people together around this first,
0:37:38 this idea of a first every emoji con, like, what’s the significance of that?
0:37:42 I thought it already existed. And to me, to be honest, when you just said that, I was like,
0:37:46 what? Yeah. And then I was like, the fact it didn’t exist. And I kind of have this issue where
0:37:51 of like, I think something needs to be, I will make it exist. Right. So we did that with dumpling
0:37:55 emoji. We did it with emoji con. And so we actually have some really cool sponsors. We’re
0:38:00 going to have a lot of kind of emoji activists kind of out there. And also, you know, from our
0:38:05 perspective, you know, there are a lot of policy decisions around emoji. And obviously, the world
0:38:09 really cares about emoji, whether or not it’s a rifle emoji or the condom emoji or like professional
0:38:13 women emoji. Part of the goal of emoji con is to open up that discussion. So it is not just how
0:38:17 that the Unicode level. So to hear… So our Unicode members going to be attending this conference?
0:38:21 Oh, members of the Unicode emoji subcommittee, including like, you know, the co-chairs. And
0:38:26 we timed it in November between the Unicode conference itself and the Unicode technical
0:38:32 committee meeting. And also, like, it’s right around election day. Well, you guys, thank you
0:38:35 for joining the A6NZ podcast. Thanks for having us. This is so much fun. This is so much fun.
0:38:39 So much fun. We could keep going. Hours and hours on emoji con. Yeah, I wish we could.
0:00:10 favorite episodes of all time, all about emojis. It’s timely not just because a whole new set
0:00:15 of emojis are about to hit many people’s phones soon, but because the episode, which was actually
0:00:21 first recorded in 2015, is really an evergreen and broader conversation about how innovation comes
0:00:27 about. Because we cover the tension between open standards versus close and proprietary systems,
0:00:31 the economics of creativity, the governance and politics of design, and much more.
0:00:35 We begin with the brief tour of different emoji and how they came about,
0:00:39 where emoji fit in the taxonomy of visual and social communication,
0:00:43 and why that matters, especially when it comes to machines reading emotions.
0:00:48 And finally, we talk about the ambiguities and difficulties to translating emoji
0:00:53 when it’s not really meant to be a language. Our guests are Fred Benenson, who was a second
0:00:58 employee at Kickstarter and VP of their data. He also famously kickstarted a project to translate
0:01:04 Moby Dick entirely into emoji, and Jenny Aitley, former New York Times reporter who was a member
0:01:10 and is now vice chair of the Unicode Emoji subcommittee. She first led the effort to get
0:01:13 the dumpling emoji, which is where we briefly start this conversation.
0:01:20 I wasn’t a really big emoji user. In fact, the first time I ever heard of emoji was when
0:01:25 Fred started his Kickstarter called Emoji Dick, and I was like, “What the fuck are emoji?”
0:01:29 This is before they showed up on our iPhones, like perky little yellow faces.
0:01:31 I was like, “What? It sounds something very bizarre.”
0:01:35 I just started. I didn’t even actually just be blunt. I had a very hard time using emoji,
0:01:39 because I didn’t quite understand how to even, frankly, use that moment. I don’t understand
0:01:44 it when people send it to me, if it’s not the obvious heart, etc. But as I’ve been using it
0:01:48 more, I found myself sort of expressing myself now in kind of quirky ways, and I don’t know
0:01:50 if people really get it or not, but I’m getting a kick out of it.
0:01:52 That’s the fun of the ambiguity.
0:01:58 I have a friend who showed me an exchange between a friend of his who was dating a guy,
0:02:01 and he would only send her emoji, and she was like, “I just can’t handle this.”
0:02:05 And he showed me these screenshots of their exchange, and it was hilarious.
0:02:06 You’re helping translate.
0:02:07 But yeah, it’s so like, I was like, “Oh, this is-“
0:02:09 You looked a certain abridgerack in like emoji.
0:02:11 Yeah, I was like, “This is what this means.”
0:02:16 I can definitely see it being like sort of an irreconcilable difference between people and
0:02:21 relationships. I fast forward many, many years. Emoji have showed up on our iPhone,
0:02:25 and I’m texting with my friend Yiying Lu, who’s best known as the designer of the Twitter fail
0:02:29 whale. So we’re texting back and forth about like dumplings, and so I send her a picture of the
0:02:33 dumplings I’m making, and then she texts me back, knife and fork, knife and fork, yum, yum, yum, yum,
0:02:38 and she goes, “Wait, Apple doesn’t have a dumpling emoji.” I was like, “How could that be?”
0:02:43 I was like, because there’s so many obscure Japanese food emojis and emojis are from Japan,
0:02:50 like you have everything ranging from ramen to curry rice to tempura to like the rice thingies
0:02:54 on a stick. There’s even the like triangle rice ball that looks like it had a bikini wax.
0:03:00 Right, there’s also the fish cake, which is the white one with the purple sorrel in it.
0:03:06 And I was like, “How could there not be dumplings?” Because it’s such a universal food,
0:03:12 because there’s like pierogies in Poland and momos and gyoza and empanadas. It’s just like
0:03:15 a food from around the world. I mean, technically a samosa is a dumpling.
0:03:25 Samosa, ravioli. And I was like, “Okay, emoji are universal.” And then dumplings are universal.
0:03:31 How could there not be a dumpling emoji? And just in my mind, I was just like clearly whatever
0:03:34 system in place has failed. How do you solve a problem like the dumpling emoji?
0:03:38 Yeah, and I found out that emoji are regulated by the Unicode Consortium, which is a non-profit
0:03:45 organization based in Mountain View, California. It now has 12 full voting members that pay $18,000
0:03:50 a year just to vote on issues, including like emoji and other kind of like technical numbers.
0:03:51 Are all those numbers in Mountain View?
0:03:58 No, so of those 12, nine are U.S. multinational tech companies, Oracle, IBM, Google, Yahoo,
0:04:04 Adobe, Facebook, Microsoft, and Symantec. Then of the other three full voting members,
0:04:09 one is the German software company SAP, another is the Chinese telecom company Huawei,
0:04:13 and the last is the government of Oman. That’s a really interesting crew.
0:04:16 It’s an interesting crew. And they have these quarterly meetings.
0:04:20 And then I just shop. And they’re very welcoming. They’re like, “Thank you for coming. What brings
0:04:26 you here? Tell us about yourself.” It felt like showing up at church. Like a new church,
0:04:29 you’re a new member. They all knew each other very well. They’re very excited that there’s
0:04:35 like someone young and diverse that’s just like randomly shown up. And so I, in that process,
0:04:43 learn how you get emoji passed and how they’re regulated. And so in January of 2016, we submitted
0:04:48 a full proposal for dumplings, takeout box, chopsticks, and fortune cookies and got those
0:04:54 all passed. So those will be in Unicode 10, which means that that’s announced in June of 2017.
0:04:58 And so they’ll actually hit your phones several months after that. I was like,
0:05:02 “Wow, billions of keyboards will be impacted by this.” That’s amazing.
0:05:05 Were there other proposals submitted at the time? Oh, they’re constant proposals.
0:05:10 There’s this whole process that people like Jenny, some of them make it through.
0:05:14 It’s a lot of work. It does reduce some good, useful bars actually for making sure quality
0:05:16 gets through at some point. Yeah. And to their credit, the Unicode Consortium
0:05:22 has an amazing list of emoji criteria where they say, “Okay, here’s what we’re looking for
0:05:27 for emoji. It’s got to have like, you know, kind of a unique meaning in that it’s not covered by
0:05:31 other stuff, but it also should have like, you know, some ambiguity. So it’s not just like
0:05:35 literally one thing. It could be used in other contexts.” Also, there’s one of the more interesting
0:05:41 rules, which is no celebrities, deities, or logos. Whoa. The Easter Island head is kind
0:05:46 of a violation of that one, but that’s got its own story. A couple of years ago, with a big update,
0:05:51 the Easter Island head showed up in like the back of the travel section of emoji. And I was like,
0:05:55 “What is that doing there? Who’s traveling to Easter Island so often that they need to use
0:05:58 the Easter Island head emoji?” And it kind of just stuck in my mind. And then I started using it in
0:06:04 this kind of like, slightly culturally insensitive way to like reference some supernatural phenomenon
0:06:07 that I didn’t understand, right? Like if I was in a conversation with somebody and I was just like,
0:06:10 completely flummoxed, I’d just like, send that one. Yeah. It’s like your version of Bermuda Triangle.
0:06:14 Yeah. Yeah. I was just like, who knows? Stone face. Other people use it for like,
0:06:19 stoned, right? Like, there’s lots of combinations in there. The reason why it’s in there is that
0:06:25 there’s a statue in downtown Tokyo, I think it’s a Shibuya station that is called Moyai,
0:06:30 which is a name of just like, it’s a proper noun of that statue, which was made by an artist
0:06:36 that was like a reference to original Easter Island head. So it turns out Japanese teenagers use
0:06:42 this waypoint to meet each other. And so that’s how it ended up in Japanese cell phones. And that’s
0:06:46 why it ended up in emoji. The artist used this inspiration of Easter Island. The interesting
0:06:51 twist is that when you look at it on the iPhone, it doesn’t look anything like the statue in Tokyo.
0:06:56 At some point, Apple was like, we’re not going to make it like this Tokyo’s one. We’re going to do
0:07:03 the original one. Android, on the other hand, their Moyai emoji looks like the Tokyo station one.
0:07:07 So fascinating. I read a study, I actually included in our newsletter months ago, of someone
0:07:12 comparing how emojis look on different platforms and how it actually changes meaning. Because
0:07:15 you can actually think you’re sending one thing and you get something else.
0:07:19 That’s going to happen in any system that has standardization. Like, you’re going to try really
0:07:24 hard to make sure people hue to the specification. But you know, people do their own implementations
0:07:30 and things change. In fact, the whole reason why emoji are in Unicode was because you would send
0:07:35 your friend an emoji and then their cell phone would actually just render the incorrect one.
0:07:39 It could be so much worse. And the fact that there is a standard means that like,
0:07:41 you only get these like weird edge cases.
0:07:46 There’s still some interesting vestiges of like the different telcos between Apple and Google.
0:07:52 One was DoCoMo and the other one was SoftBank. So they’re basically, depending on who their partner
0:07:58 was locally, they kind of inherited those generations of emojis. For example, on Apple,
0:08:02 women with bunny ears is like two women dancing in kind of like a let’s party kind of way with
0:08:07 their bunny ears. Whereas on Android, it’s just the headshot of a woman with bunny ears.
0:08:10 And it’s referencing this slightly misogynist part of Japanese culture of
0:08:15 bunny woman, which is itself a reference to the Playboy bunny. And so like they were cocktail
0:08:20 waitresses working in nightclubs. That made its way into the Japanese set. And then so when it
0:08:23 came over to America, like I think Apple must have been like, let’s make this a little more fun.
0:08:27 One of the easiest things actually to get emoji pass is showing that a vendor uses it.
0:08:32 Another argument is for completion. This is actually why chopsticks got passed fairly easily
0:08:36 because we had like knife and fork. Oh, so you need completion of a set.
0:08:40 You need completion. So that’s you can tell a whole story like stringing together a bunch of
0:08:44 no, I just think that it’s like they’re engineers, right? You can’t have ABCDE and
0:08:50 actually, one of the weird issues is that they’re red, yellow, green, purple, blue,
0:08:55 hearts, not orange. So one of the big lobbying efforts has been to fill in the orange. So the
0:08:59 case of the Apple bunny ears and the Japanese bunny women, that was a case where there was an
0:09:03 intentional translation translation to sort of obscure the cultural reference. More than that.
0:09:08 They often try the map technically the same emoji, but it’s like rendered and sort of interpreted
0:09:12 differently. They like emoji that can have multiple meanings. You can also just have like
0:09:16 emoji to have one meaning, but it really has to be a really good one was going to be one meaning.
0:09:21 So for us, the Chinese takeout box, for example, one of the arguments that we made,
0:09:26 we made is that it’s, it’s one, it’s an iconic shape. It also symbolizes both an entire cuisine,
0:09:30 which is Chinese food, and also a means of eating, which is delivery and takeout, right?
0:09:36 And so, so in that one symbol, you get a lot of sort of secondary meaning. And with fortune
0:09:40 cookies, like it’s technically a cookie, but it also means like mysterious in the future and the
0:09:44 unknown and like so like so our primary secondary meaning one of the criteria for an emoji to get
0:09:49 past is that it has to have a certain element of ambiguity to it. So I love this. I’ve been thinking
0:09:53 about this so much. When I did emoji deck, it was more of an experiment around crowdsourcing
0:10:00 an emoji itself. Like I wasn’t like so much interested in making a formal case that emoji
0:10:04 could be a language because it was still so early. Could it get there maybe one day? Yeah.
0:10:08 But Unicode makes a really good point. They’re like, emoji’s not a language. It shouldn’t be a
0:10:13 language. The value is that it’s ambiguous. And I’ve really come around to that thinking and this
0:10:17 idea that the charm of sending an emoji is that it can be interpreted in a couple of different
0:10:22 ways. And, and that’s actually why we value it. And, and I’ll go further and say that a lot of
0:10:26 people ask me why emoji have become so popular. And I think it’s tied to the fact that we now
0:10:33 are just inundated with text. We live in a text culture, right? We, we communicate via text.
0:10:41 Our careers are run over email. We read constantly. Everything we do is mediated through almost
0:10:47 literal words. And so emoji represents this kind of reaction to that. And the popularity of emoji,
0:10:51 I think, is largely due to the fact that we need some other way of expressing ourselves over text.
0:10:57 If the pipes are so mechanical, like phones and machine, you no longer have the nonverbal
0:11:01 aspects. So this is actually replacing sort of this, this human element of the glimmer in your
0:11:05 eye or like the cheeky, the blush on your cheek. There’s an emoji that does that.
0:11:09 You think about the amount of signal you get from somebody’s voice on an analog telephone.
0:11:12 And when you strip that out and all you’re communicating is like LOL, you don’t actually
0:11:18 know how sincere that laugh is or that chuckle or whatever that person’s trying to convey.
0:11:23 And so emoji gives us a much bigger palette to convey this kind of like extra like limbic
0:11:27 meaning that we want to have in our, in our communications, but we, we can’t because we’re
0:11:31 just, we’re texting all the time. So to break down the taxonomy of figural representation,
0:11:35 not using literal text, let’s talk about where emoji fits. We have emoticons,
0:11:41 which are like a colon and a parentheses and that gives you a smiley face or like a semicolon and
0:11:45 a parentheses and that gives you a wink. Right. Using punctuation. Using punctuation is emoticon.
0:11:48 Often ascii-ish. Right. Because it’s not ascii art as well.
0:11:52 Some of the earliest references to emoticons go back to the 19th century as well where people,
0:11:58 yeah, yeah. People were using colons and dashes and parentheses to express like a wink. It goes
0:12:03 way back. It’s important to add in hieroglyphs and iconography. Other humans have had this idea
0:12:07 before, right? Like the, the medium and the technology is kind of like incidental.
0:12:11 I’m so glad you brought that up because it’s so important to not get caught up in technology time.
0:12:16 Well, technically technology includes like sticks and on stones. So that does go back in time,
0:12:22 but in the context of this machine web that we live in, then we have emoticons as part of the
0:12:25 taxonomy and then we have emoji. But how would you guys define emoji?
0:12:29 It’s Japanese. Drawing language. Emoji. I don’t know how to pronounce in Japanese,
0:12:34 but the Chinese. The emo is not for emoticon or emotion or anything. It’s just totally
0:12:41 coincidental. Wow. It’s hard not to just hue to the Unicode standard and say it’s the,
0:12:49 it’s the set of icons defined in Unicode that represent objects and nouns and actions and.
0:12:53 The way that I explain it to people is an emoji is a character, an emoji is something you can
0:12:58 put in the subject line of an email because it literally is text. So, so in the same way that
0:13:04 Unicode is kind of defined a standard to unify all the graphical representation of different
0:13:08 languages throughout the world and even non languages. So like, you know, the windings and
0:13:12 all that kind of stuff. Emoji actually slip into that entire system. So there is literally
0:13:19 what they would call a code point assigned to each emoji. Or sorry, not every single one because
0:13:24 now they’re like compound emoji, but there are code points assigned to emoji, which basically
0:13:30 says, you know, when you, when a computer sees this code point, they render it in a certain way.
0:13:34 But that, that it’s important to kind of wrap your head around what’s actually happening inside
0:13:39 the computer because the emoji is being sent as text. If your computer supports UTF-8, UTF-16,
0:13:45 that’s just like a standard way for your computer to handle text, whether it’s your phone or your
0:13:50 laptop, then it’s being told render this emoji. But it’s actually up to your computer’s operating
0:13:55 system, whether it’s OSX or iOS or Android or whatever, to go fish out a little image and put
0:14:00 it on your screen. And so that image is actually controlled by the hardware manufacturer or the
0:14:05 software manufacturer, you know, when it’s actually rendered on your screen, the operating systems
0:14:09 choosing which image to show you. And those images are actually stored, you know, in the same way
0:14:13 that other images are stored on your computer as little PNG files. And so Apple, you know,
0:14:17 puts those on your computer and your computer chooses to render those, which is why you may get
0:14:21 slightly different, you know, interpretation. Right. I’m glad this is actually really interesting
0:14:27 because recently Facebook just introduced their own emoji and that like basically hijack Apple
0:14:31 emoji. So you can turn that honor off, but essentially they’ve, they will replace, they’ll
0:14:35 swap out all the ones on the app. And Twitter’s had their own set for a while. And so why is that?
0:14:39 So they’re interesting copyright considerations here. My guess is a lot of those companies are
0:14:44 doing it because, A, they can afford to make their own set. B, they want to avoid the legal
0:14:50 liability of using Apple set. And C, like they think they might kind of have some like moment of
0:14:55 like, Hey, did you see Twitter’s new emoji? Right. And so they’re, you know, these large companies
0:15:00 are kind of innovating on emoji. Yeah. Yeah. Like re-innovating and re-illustrating their emoji.
0:15:04 And I think, you know, I think Microsoft actually just evolved to a new set or wasn’t
0:15:08 Android. I think it might have been Google Android. They, they just upgraded to make it
0:15:11 seem a little bit more normal. Like they had gone from like- That’s just not a
0:15:15 the terrible blue and white. Yeah. Yeah. So it’s like the blobby ones or-
0:15:19 Yeah, the Android, I think Google had blobby ones for a while. Now they’re doing somewhat normal ones.
0:15:26 Scariest emoji ever, the Microsoft emoji are like blue and gray. And they look like monsters
0:15:29 that hide underneath your bed. Why? Why are they blue and gray? I think it’s just like-
0:15:33 It doesn’t attempt to be like, like different from like a yellow skin tone.
0:15:37 Well, also, you have to, part of the original emoji is you wanted things that were
0:15:43 skin tone neutral. So Apple and Google chose yellow, but Microsoft for some reason chose gray.
0:15:47 Oh, gray. Cause I was going to say for Hindu, like blue is actually not a bad thing to have your skin
0:15:51 blue. It’s like a god. The other thing is if you have your own set of emoji, you can actually
0:15:58 start adding to that set without going through the unicode. So like a very good example is the
0:16:02 gay family emoji originally where they’re not, it’s not actually one emoji. You know,
0:16:09 the one where it’s like man, man, kid, kid. That is actually a compound emoji of four characters
0:16:14 glued together using something called a quote zero with joiner, which is basically like an invisible
0:16:20 glue. So if you are sending that emoji to someone else who doesn’t have the ability to render that,
0:16:25 it actually unravels itself into like a multiple character. Now what you’re seeing is a lot of
0:16:31 vendors making compound emojis. So like actually one of the places where this is being debated
0:16:34 for use is the need for a professional female emoji, right? Cause one of the big problems right
0:16:40 now on the existing set of women as represented by emoji is like, there are only like really
0:16:45 four roles for women to play compared to men. You know, men, you can be a sleuth or you can be,
0:16:51 you know, a policeman. You can be sort of a medical worker. You can even be Santa Claus,
0:16:58 but as a woman, the four things you can be as a role are basically bride, princess, dancer,
0:17:02 ploy by bunny. That’s it. Oh my God. It just goes to show you how the, I mean, of course,
0:17:06 this is the politics of human life play out in these systems. I mean, the perfect example I was
0:17:11 thinking of is a rifle emoji. And the case of, I believe Apple, Google and Facebook,
0:17:16 Charlie Warzel at Buzzfeed wrote a really detailed article investigating this and about how they
0:17:20 sort of help suppress as part of the Unicode consortium, the rifle emoji. Right. Emoji already
0:17:25 has a gun in it, right? And it’s like, okay, so how many more versions of that do we need? And
0:17:30 you’re right, it’s absolutely a political topic. I mean, that issue manifests itself in so many
0:17:35 other places in emoji. The country flag stuff is super interesting because that uses kind of what
0:17:39 Jenny’s talking about with these compound emojis. Unicode didn’t actually want to decide which
0:17:44 flags were and weren’t in emoji. So they did. Right, because you’re legitimizing then political
0:17:49 issues. What they did was they built this kind of like meta country system so that you would
0:17:54 actually be pairing these country letter emojis together. So CNN would go together and then
0:17:59 it would be up to your phone to decide if you showed the Chinese flag. They pushed that decision
0:18:03 making, that like political decision making of which flags to support off to the handset
0:18:08 manufacturers. Microsoft actually does something weird there. What do they do? They just show the,
0:18:12 they don’t show a flag. They show a flag plus the two letters. Right, right. Microsoft doesn’t render
0:18:17 it normally. Yeah, to the point about politics being kind of embedded in emoji. It’s not just
0:18:23 because these are icons that represent the parts of our lives that we feel passionate about. It’s
0:18:27 because there’s a finite palette. It’s not like language where you can only, you know, you can
0:18:31 kind of combine say whatever you want. It’s combinatorial. You can take multiple combinations
0:18:35 and turn into whatever you want. You get way more degrees of freedom to kind of express yourself.
0:18:39 There’s a finite number of food items that are ready to go in there. And when you think about the
0:18:43 vast like multitudes of humanity, whether it’s, you know, people’s relationship status or structural
0:18:48 orientation or skin color, it’s like, like emojis never going to be able to express that. And so
0:18:54 like how do you contain this thing that’s like growing and kind of has to grow as more and more
0:18:59 people use it, but also by definition has to be a finite list of icons. Well, how do they handle
0:19:03 the skin tone issue? Because one of the things that I noticed is that an apple, because I use
0:19:08 an Android, so I didn’t notice this, you can press down on a thumbs up, for example, and then you
0:19:14 can pick among 15 different shades to like pick a skin coat shade that’s closest to you. It’s based
0:19:20 on a Fitzpatrick skin tone. Yeah, it’s actually used, it’s the same skin tone system that dermatologists
0:19:24 use to categorize. This reminds me a little bit of being a kid when like you had Crayola box,
0:19:28 I remember that the only shade you had, there was like a nude shade or like a skin tone. Yeah,
0:19:33 nude was always Caucasian. I remember using sepia to represent my skin color. I mean,
0:19:37 there’s a great history about this in, this is going to sound weird for me to say, but like women’s
0:19:42 pantyhose like had this issue where nude was always considered Caucasian. And people were like,
0:19:47 this is ridiculous. It was one of the earliest blind spots of emoji, I remember. Right. Well,
0:19:51 I mean, if you have like only white men designing them, do you remember when Slack, there was this
0:19:56 guy who wrote a post about Justin Brown, and I remember it was so meaningful because it’s such a
0:20:02 minor seemingly arbitrary thing. But then it is true, like the first time I saw that I could find
0:20:09 my skin color in a system and to be able to use it was kind of amazing and empowering. And I’d
0:20:13 think there’s something significant about that. I would totally agree. I don’t share your experience
0:20:19 as that as the person on the other side. And so it’s funny for me because I don’t,
0:20:29 he’s a white male. Yeah. I don’t share that like sense of identification with the
0:20:38 bright white skin. Right. I’m like, it feels odd to opt into that, which speaks to my privilege
0:20:42 as a white male. I mean, if you’re not exposed to it, you’re not exposed to it. The bottom line is
0:20:45 if you’re any person of color, you’re always aware of your color. Especially if you’re in a
0:20:49 context where everyone else is not the same color as you. And so when I texted my friends who are
0:20:53 not white and I’m like, should I be choosing that one? And I just choose to choose the yellow
0:20:59 skin tone. That’s just like the way more comfortable. To my solution is I often send four. It’s like,
0:21:05 it’ll be like yellow, light, dark, and then like the beige one. It’s like a Benetton add emoji.
0:21:11 Emoji, that’s fabulous. So now the kind of evolution is that we have yellow for like all the
0:21:15 human face characters, and then you can choose skin tones for some of them. But it doesn’t get at
0:21:20 like more nuanced issues about like cultural and racial identity, having to do a facial structure
0:21:24 or hairstyle. Oh, right. That’s a great point, actually, because one of the pet peeves I have
0:21:28 is when I used to go to foreign countries and look at billboards, it always glorified that
0:21:32 aquiline knows the face structure, whereas there’s a totally different type of face structure in
0:21:38 different areas. Emoji probably won’t ever have that amount of like customization and Unicode gets
0:21:45 this. And they actually say like, we’re adding like 60 emoji a year. This is fun, sustainable.
0:21:50 We feel like the future is inline images. And that kind of breaks my heart as like kind of a,
0:21:55 you know, nerd standardization guy who like, who really appreciates all the hard work that went
0:22:00 into Unicode and the idea that it is a standard, because if you’re just sending inline images
0:22:05 forever, then like, you know, you have no idea what’s going to be on the other side if they can
0:22:10 render the image. So stickers, I mean, so Kim Emoji, for example, Kim Kardashian’s quote Emoji,
0:22:16 they’re not actually emoji, those are just stickers or images that you can text back and forth.
0:22:20 But, you know, again, you know, standards, can you put it in the subject line email and those
0:22:24 you can’t you can’t so that therefore they don’t qualify. So they’re not technically emoji. Right.
0:22:28 So then going back to our hierarchy, we went from emoticon to emoji and now stickers. Stickers are
0:22:33 basically inline images. I mean, stickers are just images that you can pick from a palette or
0:22:37 like, and I think you can, you know, in certain apps, you can like apply a sticker to an image
0:22:41 that it like sits on top of it. But you’re then in this kind of like proprietary ecosystem of
0:22:46 that’s okay. But like, you think about the stuff that really works and the stuff that
0:22:51 really changes the future of the web and communication, it’s all standardized.
0:22:55 You’re saying this as a standardization person, because my friend Connie, who wrote a wonderful
0:23:01 post on the topic of stickers, argues that emoji are very limited for what you need to do because
0:23:05 she feels that you have so much more expression and the ability to convey so much more with
0:23:09 stickers than you do with emoji. Emoji doesn’t preclude the use of stickers. There is some
0:23:17 subset of images that are universal enough that should be hardwired into the operating systems
0:23:24 and are basically can be cross-platform that an iOS device can talk to, you know, Microsoft windows
0:23:29 and can talk to like an Android device can talk to your Mac laptop. Like the fact that at least
0:23:33 there is you’re not going to get little square boxes as long as your operating systems are
0:23:37 fairly up to date. Well, that goes to your point about why standardization is important,
0:23:42 because you’re now giving up that you’re in this proprietary ecosystem like WeChat or Line,
0:23:45 and you only have their stickers set and you can’t always transfer all these stickers across.
0:23:49 I mean, also, if you think about the accessibility issues around stickers, right, like people using
0:23:53 screen readers, they’re not going to be able to interpret an image and like emoji actually have
0:23:59 names. And so in theory, there’s much better accessibility for emoji for somebody who’s visually
0:24:07 impaired. Like for example, last year, Oxford English Dictionary chose Facebook Tears of Joy,
0:24:14 which I always thought looked very sad. The thing with the eyes and it’s like balling, but
0:24:18 that’s actually Face of Tears of Joy. And that is how you know that because, you know, all these
0:24:23 emoji have. They say the label. Oxford put that in there. So the word of the year was an emoji.
0:24:27 Part of the reason they chose that was that it ended up as number one on my friend’s site called
0:24:31 Emojitracker.com. Oh, right. That’s right. The Emojitracker, which tracks all the use of emoji
0:24:35 on Twitter. And for a while, it was just like, it was like the heart emoji or something or just
0:24:39 the smiling face emoji. So I think it’s really interesting when the top emoji shuffle because,
0:24:44 you know, whenever you start texting with somebody who hasn’t used emoji before,
0:24:49 they’re like choosing like the safest ones. Going back to this idea of some of the companies
0:24:55 owning their own emoji and some of the proprietary open tension between standardization,
0:24:59 freedom of expression. What do you make of this notion that part of what we’re doing here is
0:25:04 essentially also creating a more machine-readable web in terms of emotional reading? Because
0:25:09 essentially you’re now adding a whole new layer where you can codify people’s emotions, sentiment,
0:25:12 in ways beyond just a black and white like don’t like. I’ve been thinking about this so much,
0:25:17 actually, and not in the context of emoji, but actually Facebook reactions. Yeah, me too. I used
0:25:20 to assign and edit top ads on this topic because it was very accessible. Yeah, I think it’s a really
0:25:25 interesting topic because if you look at traditional sentiment analysis in the data world,
0:25:29 it’s kind of a joke. You have to have training data, you have to know good cases.
0:25:33 Right. And just to interject for a moment, as someone who’s been tested a million of those
0:25:38 systems and can never find one that actually works for my needs, they’re so binary. You
0:25:42 don’t get anything useful and you’re not getting insight. One of the reasons there is that words
0:25:47 have these degrees of freedom. They can be used sarcastically and you would never know it based
0:25:53 on the semantics. And so traditional sentiment analysis is really broken because you’re using
0:25:58 these kind of like stale, rigid, semantic definitions. What’s really interesting about
0:26:03 Facebook reactions is you think you’re saying, “I love this thing,” or “I’m sad about this,”
0:26:09 or “I’m angry about this,” but what you’re actually doing in conjunction with that is giving Facebook
0:26:16 really great labeled data for sentiment analysis. That’s right. Machine readable data. That is a
0:26:22 coley grail of emotional sentiment understanding. When I was at Wired, I assigned a piece to a
0:26:28 sociologist, Evan Solinger, because I wanted to coin this phrase, the mood graph, because we have
0:26:33 an interest graph, social graph, all kinds of other graphs that link all these nodes and ideas,
0:26:37 and now to have a mood graph to essentially be able to put your pulse on someone’s mood,
0:26:41 something very finite yet constantly changing. It’s just a fascinating thing to be able to
0:26:46 codify this. The sentiment stuff generally coley started strongly with human face and body.
0:26:52 So, I think this is also why people agitate so much for emoji that look like themselves,
0:26:56 like the redheads, and people with beards, and people who are bald.
0:27:00 Or anyone with curly hair. People with curly hair relate to other people with curly hair.
0:27:04 And so, I think people really love seeing themselves represented in emoji, which is why
0:27:09 Bitmoji, which is highly, highly, highly customized stickers in sort of emoji spirit.
0:27:13 Oh, and my cousins and I use Bitmoji on WhatsApp all the time. I think there’s something really
0:27:19 symbolically important about Bitmoji, because you are putting yourself in it and conveying in
0:27:23 this sticker form. The fact that Snapchat bought it, I think, is really telling.
0:27:24 Oh yeah, for a hundred million dollars.
0:27:28 Right, especially given that they are changing this culture of how you express yourself through
0:27:32 your facial expressions with face swapping and filters. Connie and I made the argument that it’s
0:27:36 sort of like selfies, as selfies as a form of stickers. So, what we’re talking about the machine
0:27:40 readable is a little distinct than this, but it’s sort of an interesting idea.
0:27:44 I also think it ties into this slightly dubious notion of the Uncanny Valley,
0:27:47 where if you want to try to represent yourself and you want to have like configurability around
0:27:52 that, it needs to be kind of cartoonish for it to be believable. I think what we’re seeing with
0:27:56 Snapchat filters, and I don’t know if you guys have played with Snow yet. That’s like a,
0:27:59 it’s like, take Snapchat filters and just multiply them by a thousand. It’s like,
0:28:04 it’s like, just like amazing amounts of diversity around the amount of stuff you can put on your
0:28:11 face. It is this weird convergence on identity and emoji that’s kind of happening.
0:28:14 I agree. And in fact, this is going to be a little, sound like a little out of left field for a
0:28:19 moment, but the whole notion around the Chewbacca mask lady, when, you know, that was the most
0:28:23 popular Facebook live video ever, it got like unprecedented views. And there was simply a woman
0:28:27 who was trying on her Chewbacca mask in the car and she’s laughing and giggling about it.
0:28:32 And then she puts her mask on and then she takes it off and she laughs so uninhibitedly.
0:28:37 It’s insane. And I make the argument that what was so empowering because it was totally took off
0:28:42 for obvious reasons is not the fact that she was laughing so uninhibitedly. It’s a fact that
0:28:47 it took putting on and then taking off the mask for her to do that, which is a lot,
0:28:50 not unlike what happens with communication through these filters and being able to now
0:28:56 express yourself through these cartoon like ways. I mean, honestly, it takes me back to like theater
0:28:59 and like Shakespeare in like seventh and eighth grade. I remember having these like really intense
0:29:03 discussions about like what it is to put on a mask and what a mask represents about yourself.
0:29:07 It’s a very Cambilian idea, right? The Joseph Campbell like mask and the myth and the man.
0:29:10 You’re right. There’s a theater, I mean, that’s why people say improv is so interesting for any
0:29:14 career field. But I think that there is an interesting moment now coming together with
0:29:20 selfies, stickers, emoji, bitmojis all together where we do have this new emotional web coming
0:29:25 together. And using emoji as the first time I thought about this could be kind of like putting
0:29:34 on a mask over your words to convey to yourself this extra, this kind of additional layer,
0:29:37 this emphasis of your emotion that you otherwise might not get.
0:29:41 Okay. So going back to you writing an entire book in emoji and yet you’re saying that you’ve kind
0:29:46 of evolved into thinking that emoji is not necessarily language, but clearly it is a visual
0:29:51 language and it is a tool for communication. It’s not complete. So how did you translate that?
0:29:54 I mean, what were some of the trade-offs and decisions you made? And by the way,
0:29:58 for the audience, that book was like 2009 or that was like many years ago.
0:30:01 So what emoji space were you working off? Did you make them up? Like what’d you do?
0:30:06 So I’d gotten in a text from my college roommate whose wife is Japanese. He sent me an emoji and I
0:30:11 was like, what is that? They told me you could download like basically a Japanese app and it
0:30:17 would like awaken your iPhone to the emoji keyboard. It just spoke to me in the like,
0:30:21 like you have to hack the iPhone to get the special keyboard of like Japanese icons. And I
0:30:26 was like, oh my God, I want this so bad. I was like, this is amazing. I should write a book
0:30:29 in emoji. And I was like, oh, that’s a lot of work. And I don’t know if I can write a whole book
0:30:32 in emoji. And then I was like, well, maybe I can translate a book in emoji. I was like, okay,
0:30:37 what books would work? And I was like, well, it has to be in the public domain because I worked
0:30:41 a lot in like the copyright reform space. Nobody’s going to just like, let me translate their
0:30:44 book into emoji without a lot of effort. For a moment, I thought about the Bible and I was like,
0:30:48 that’s too obvious. What’s like, what’s like totally even more inappropriate.
0:30:55 So Moby Dick came to mind as like this, this like impossible book to trans to put into these
0:30:58 symbolic characters. As soon as I thought, I was like, no, I can’t do that. That’s crazy. And I
0:31:00 was like, that’s like too hard. Honestly, it’s a little bit like, I just came back from saying
0:31:04 Hamilton. And so it’s a little bit like the idea of putting a wrap to like the founding fathers.
0:31:08 That’s what I find so fascinating. I would say Hamilton was probably a mashup of mediums and
0:31:13 time and culture. And it’s like one of those things where you tell to somebody and they’re like,
0:31:16 you can’t do that. That’s crazy. And then you’re like, well, the fact that you just said that made
0:31:21 me want to do it. And so not only that, there are not one, but two whale emoji. Were there at that
0:31:29 time? No, there’s only the original, the cute one that kind of was battling the cute whale.
0:31:35 The second, I think it’s called sperm whale didn’t come up until later. So I was like, okay,
0:31:39 wow, that would be really interesting to do all of Moby Dick because it’s also like really long.
0:31:44 I mean, it’s 10,000 sentences. And okay, well, if I don’t want to do this, maybe hire somebody to
0:31:48 do this. And I was like, experimenting with mechanical Turk at the same time. I think it was
0:31:53 like one of the original Amazon web services. It was like, it would later become part of that
0:31:58 AWS umbrella. I remember people using it for research and stuff. Right. It’s still used for
0:32:02 research. It’s still being valuable for that. But a couple other people have done an experiment
0:32:09 here or there of using it off label. I had made a task of mechanical Turk just to ask Turk workers,
0:32:14 if you could ask anyone like to do anything on mechanical Turk, what would you have them do?
0:32:17 And they came up with this long list of stuff. And I don’t think translate a book into emoji
0:32:20 was one of them, but there’s some creativity out there. I was like, okay, I’m going to,
0:32:24 I’m going to try this thing, or I’m going to hire people to translate Moby Dick into emoji,
0:32:28 some portion of it and see if this works. So I did the first chapter and the results came back
0:32:33 and they were hilarious. They’re so good. What do you mean you did the first chapter? Like,
0:32:38 did they break it down word by word? So how do you capture that in emoji? So I decided I was
0:32:43 going to do it as on a per sentence basis. And that actually turned out to be one of the challenging
0:32:48 parts of the project was like, splicing sentences is actually like kind of like a classically hard
0:32:52 and all natural language processing problem. And so I kind of like figured out a hack to like
0:32:55 chop it up. And I wrote a lot of regular expressions to basically book into sentences.
0:33:00 But you decided basically the sentence was a unit of analysis, not a phrase in the task.
0:33:04 And you say, pick any of these emoji and then actually wrote my own little emoji picker because
0:33:09 these things didn’t exist at the time. I had gotten the emoji from a friend. He had reverse
0:33:17 engineered the iPhone SDK and basically hacked out the PNG files from the software kit to basically
0:33:21 have the raw emoji in image form. And so I took that and just made like a little JavaScript,
0:33:25 like HTML thing and you know, dump that into mechanical Turk and like came back and I was like,
0:33:30 hey, this works. And so I think the sentence that’s kind of like on the cover of the book,
0:33:35 if you go to the website, it’s like the website being Emoji Dick. Emoji Dick.com.
0:33:41 Call Me Ishmael is the first sentence of Moby Dick. And the emoji that the Turk worker chose was
0:33:49 telephone man with face, sailboat, whale emoji. That’s perfect. The rest of it was just like
0:33:53 indecipherable emoji nonsense. And some of the people were just like, all right, give me my
0:33:57 five cents. I’m going to click some random emoji. And other people just like clicked every single
0:34:03 emoji. So the plan became have people translate the same sentence multiple times. So you get
0:34:08 three different emoji translations for one sentence and then have another set of tasks
0:34:14 where people vote on the best, most appropriate translation. So like of the three, which one
0:34:18 got the meaning across the best. And I was like, oh, I’m just like getting really excited about
0:34:21 this. And I started doing the math on how much it was going to cost. And I was like, oh, it’s going
0:34:25 to be thousands and thousands of dollars. That summer I met the Kickstarter guys that started
0:34:28 talking with Andy Bayo. He was like, you should put on Kickstarter. So that night I went home and
0:34:32 put it on Kickstarter launch the next day and ended up working for them. And by the way, how
0:34:36 much is the campaign? How much money did the campaign make? My goal was like 3,500 and I ended
0:34:41 raising 3,700. So I worked on it for nights and weekends for another eight or nine months. And
0:34:46 then self-published it on Lulu.com. You can still buy it. It gets printed on demand. Do people still
0:34:53 buy it? I’ve sold like thousands of dollars of emoji deck. And I’d say hundreds of copies,
0:34:59 probably like five or 600 copies of it have sold since then, which is not a lot. I bet this podcast
0:35:02 is going to sell a bunch. Yeah, well. You better share some of the proceeds with me. Okay, so there
0:35:07 are two copies. One, there’s a black and white copy, which is like the easy to print one. And
0:35:12 that’s like $20 or $30. And then there’s the full color one, which like is obviously preferable
0:35:18 because emojis are so colorful. But when you’re printing on demand, 800 pages of color, laser,
0:35:24 hardbound copy, it’s actually really expensive. So that thing costs like $180.
0:35:26 Right, because you’re not printing in bulk. Because you actually save money when you’re
0:35:31 printing in bulk. So I have to sell that one for that much. And people still buy it. In 2013,
0:35:36 the Library of Congress contacted me and they said, “We would like to acquire emoji deck as our
0:35:40 first emoji book.” I was like, “Are you sure?” They’re like, “Yeah, yeah, we’re sure.” I was
0:35:46 telling a friend and David Gallagher, I think you must know from the times. And he’s like, “You
0:35:49 know, everyone submits their stuff to the Library of Congress. It’s not that big of a deal.” And I
0:35:53 was like, “No, man, they asked for it. Like they’re acquiring it.” I think it’s a big deal because
0:35:57 it’s a curatorial point of view. Totally. They’re saying, “This is a cultural moment. It’s not just
0:36:01 a book that was published. And we need to figure out how to acquire it.” I was like, “All right,
0:36:07 I’ll spare a copy. I signed it. I sent it to them.” And then they sent me this little certificate in
0:36:11 digital form. And what’s hilarious, and this is my favorite part, is that it’s somehow listed as a
0:36:15 translation of “Moby Dick.” So when you look up “Emoji Dick,” it says all these libraries have it
0:36:19 because it’s really just saying that they have a translation, they have the original “Moby Dick.”
0:36:22 Now it’s got a life of its own and people still discover it.
0:36:25 That’s amazing. I mean, you actually even created an art show. Did you base on this?
0:36:31 Friends of mine put together a kind of emoji survey art show and there were some really great
0:36:36 stuff. And their emoji tracker was there. There was a programming language built out of emoji.
0:36:40 I mean, emojis can have their URL. I mean, that’s another thing. They’re literally text. So you
0:36:44 can have like emoji, well, I don’t know, but you can have emoji in your email address.
0:36:47 Oh, you can. Oh, you can also buy emoji domains.
0:36:52 So you have an emoji book, you have emoji art shows, emoji hackathons, emoji hackathons.
0:36:57 So our big news this week is that in November in San Francisco, we are going to throw the
0:37:01 first ever emoji con. What? Is it like Comic Con?
0:37:05 It’s like Comic Con, but emojis. I really hope people show up dressed in emoji costumes.
0:37:08 Yeah, I’m going to show up as, oh, you guys are, Ying’s going to show up as a dumpling emoji for
0:37:13 sure. Or like, you know, poop emoji or like the ghost emoji. So it has many different
0:37:17 elements to it. So one is definitely sort of this whole emoji learn aspect where it’s like
0:37:22 panels and talks. And there’s a sort of emoji film festival. And then there’s an emoji hackathon.
0:37:27 And then there’s an emoji art show. And then of course, the opening party emoji where, you know,
0:37:31 our goals only have food that is also, also emoji. So why a conference? I mean, of course,
0:37:35 I see the cultural significance, but to bring people together around this first,
0:37:38 this idea of a first every emoji con, like, what’s the significance of that?
0:37:42 I thought it already existed. And to me, to be honest, when you just said that, I was like,
0:37:46 what? Yeah. And then I was like, the fact it didn’t exist. And I kind of have this issue where
0:37:51 of like, I think something needs to be, I will make it exist. Right. So we did that with dumpling
0:37:55 emoji. We did it with emoji con. And so we actually have some really cool sponsors. We’re
0:38:00 going to have a lot of kind of emoji activists kind of out there. And also, you know, from our
0:38:05 perspective, you know, there are a lot of policy decisions around emoji. And obviously, the world
0:38:09 really cares about emoji, whether or not it’s a rifle emoji or the condom emoji or like professional
0:38:13 women emoji. Part of the goal of emoji con is to open up that discussion. So it is not just how
0:38:17 that the Unicode level. So to hear… So our Unicode members going to be attending this conference?
0:38:21 Oh, members of the Unicode emoji subcommittee, including like, you know, the co-chairs. And
0:38:26 we timed it in November between the Unicode conference itself and the Unicode technical
0:38:32 committee meeting. And also, like, it’s right around election day. Well, you guys, thank you
0:38:35 for joining the A6NZ podcast. Thanks for having us. This is so much fun. This is so much fun.
0:38:39 So much fun. We could keep going. Hours and hours on emoji con. Yeah, I wish we could.
This rerun podcast (first recorded in 2015, now being rerun as one of our evergreen classics/ favorites) — is ALL about emoji. But it’s really about how innovation really comes about: through the tension between open standards vs. closed/ proprietary systems; the politics of time and place; and the economics of creativity, from making to funding.
So yes, this podcast is all about emoji. But it’s also about where emoji fits in the taxonomy of social communication, and why that matters — from making emotions machine-readable to being able to add “limbic” visual expression to our world of text. And if emoji is a language, why can’t we translate it; why so ambiguous?? How do emojis work, both technically underneath the hood… and in the (committee) Room Where It Happens?
Joining this episode are former VP of Data at Kickstarter Fred Benenson (and the man behind ‘Emoji Dick’); and former New York Times reporter and current Unicode Emoji subcommittee vice-chair, Jennifer 8. Lee (and one of the women behind the dumpling emoji) — in conversation with Sonal Chokshi.
image: Yiying Lu (@yiyinglu)