ROI to advertisers? Do users become blind to too many ads? VIEWER Why do casual users turn core? How do users behave across different devices? CREATOR How do indie artists make it big? What makes a successful creator? INFRASTRUCTURE How do rebutters affect user behavior? Can we design better encoding algorithms? META Can we build a better experiment framework? Is watch time actually a good metric, and channel subscriptions a good goal?
new habit we try to instill. Recommendations are more effective when we can explain why we’re giving them, so let’s ﬁnd users in Alex’s demographic (male high-school teenagers bored after school) and sell this use case.
internal raters they had been using for a while, the Hybrid proletariat was: 7X cheaper 5X faster Higher quality: on 500 videos, their internal raters made 15 mistakes, compared to 3 mistakes by my workers on their ﬁrst exposure to the task. Faster, Cheaper, Better
up? I got some big debt!" I'm so pumped about some tasks from my work desk Transcription on the downlow, it's so damn noisy Trying to make out people sayin', "Damn! That podcast's crazy!" Categorizin' ten layers deep, trying to get it right, Brain is startin' to keep me up all night Dreamin' about those tasks, pennies raining down Probably need a break, they're all I can think about (Hitssssssssssssssss....) But shit, it paid ninety-nine cents! (Grab it!) Acceptin' it, doing it', 'bout to go and submit it Passin' up on sunshine, can't believe it’s ﬁnished. Cheap ass labor, screw it, man! I'm waitin' on something' that pays Makin' my money and I'm hella happy that's a decent wage, bitch I'mma work your favorite hit, I'mma work your favorite hit No, for real - I'm work it - and you can't say nothin' about it (Thank you)
10K channels? Raters were given a channel, asked to watch videos from the channel to understand it, and then tasked with generating 5 free-form labels. Example Results - Crude humor - Do-it-yourself - Expert advice - Grammy winning - YouTube video commentary - Boy bands - Video game walkthroughs - Clips from TV shows - First person shooters - ....
games, right? Actually, there is -- watching a Swedish guy play video games and narrate them! He's hilariously skittish, has a cute accent, and will make you feel superior when it comes to your gaming prowess. Seriously funny stuff. (for pewdiepie) Little monsters, gather ‘round as your mother has something to share with you. Through these videos, Gaga lays out who she is, through music, dance and interviews. Thought-provoking conversations, heart pumping pop and dramatic concert footage will all sweep you away to The Edge of Glory. (for LadyGagaVevo)
but recently I took up playing the guitar. Because of my new found passion (playing the guitar) my desire to watch concerts has increased. I started watching a whole lot of music festivals and concerts that are posted on Youtube and other music videos. I have spent a lot of time also watching guitar lessons on Youtube (from www.justinguitar.com).”
in the feed, etc.) show a sizable increase in raw subscriptions, but modest changes to overall watch time. The watch time effect of projects like Fan Finder is also unclear. Is it because our algorithms aren’t perfect, and we just need to keep striving?
gold standard in establishing causality, but are often impossible to run. We can’t forcibly match users with channels we know they’ll love, because we don’t know which channels these are ourselves. We can’t randomly block users from their favorite channels either.
what to buy? If you've no idea how to dress, you probably ﬁnd a shirt, some slacks, and you're done. But what if you had a personal stylist to ﬁnd you the perfect outﬁt? Would you suddenly buy a handkerchief, a cardigan, and some cufﬂinks as well?
the wide variety of content that we have • Can’t ﬁnd the right videos or channels • Don’t know how to use YouTube’s features …and if we fundamentally believe that solving these issues will lead to massive changes in watch time, then even a small curation experiment over 1000 users should exhibit a noticeable change.
me cook more at home. I'd love to receive 4-5 instructional recipe videos on Sunday, for example, so I can buy the ingredients I need for the week and then try out one recipe a night. I really love California cuisine, like the type of stuff you ﬁnd up in Wine Country. I also love variations of Italian, French, German, Spanish, Mediterranean...basically all of Eastern Europe :) My diet is reasonably healthy so I'd like to stick to ﬁsh, chicken, grains and veggies (plus any sauces and marinades) as much as possible but my guilty indulgences are always breads and cheeses and I'm always down to incorporate chorizo or a ﬁlet mignon.”
and fun. Need I say more? Gnocchi isn’t easy to make in 30 minutes (unless it’s frozen, natch), so instead I picked out a scratch fettuccini alfredo. Creamy, garlic-infused sauce envelops each strand of fettuccini like a loving hug from your Nonna. Laura even pronounces it correctly!”