Watching and Talking: Media Content as Social Nexus

A shorter version of the Social Substrates talk I gave at ICMR 2012; this talk focuses on a classification of social multimedia research areas.

Transcript

1. Watching and Talking: Media Content as Social Nexus David @ayman Shamma Lyndon Kennedy Elizabeth F. Churchill Yahoo! Research
2. ..Long Live Content! Content is Dead.
3. Dolores Park, San Francisco, 2006
4. Social Conversations Happen Around Media Dolores Park, San Francisco, 2006
5. !"#$%%&&&'()*+,'*-.%#!-/-0%*!,)01-2)3%456789 !"#$%%&&&'()*+,'*-.%#!-/-0%1-/23456.)4%7889987::;%< Media is a Social Experience; that experience changes how we perceive the media and hence the semantics can change.
6. Synchronous Asynchronous One to One Many to Many Many to Object Types of Social Multimedia Interactions Yes, socially motivated.
7. Synchronous Asynchronous One to One Many to Many Many to Object Types of Social Multimedia Interactions One to one
8. Social Conversations happen around videos Well – actually people join a session and converse afterwards. 8
10. A Short Movie a commercial 10
11. Percent of actions over time. 11
12. Chat follows the video! Surprised? Don’t be. 12 CHAT
13. CLASSIFICATION How do we know what people are watching? How can we give them better things to watch?
14. Legacy online video Tags Video duration Ratings Comments View Count Sharing Activity Category Recommendations
15. There are more “subtle” problems with genre and human classification...and even harder problems with finding what’s funny... 15
16. This was funny...wasn’t it? People & Blogs...all the way 16
18. Qualitative Research on People & Perception “Films in general tend to fit categories more narrowly...If NetFlix says that something is a screwball comedy, I know what to expect. I think on YouTube the range of possibilities for the content of the videos is much less constrained. Cause it might literally be a segment from a film or it could be something shot on a cheap digital camera.” “If I ever hear the laughter track, it doesn’t even have to be funny, it’s intended to be Comedy. The style of it...even if it’s a Music Video.” “Anything comedy is always impressed upon you with laughter in the background or some funny accompanying music...Those contextual cues.” “There’s a context that you need to have for something to be identified as comedy … when I see a video that I have no context for, I don’t know whether to identify it as funny. But if people are interacting with it in a way that makes me believe that it’s funny. Same thing for the wedding dance (JK wedding video) … my interaction with it is, ‘people are saying that this is funny.’” “They (the uploaders of videos) are uploading things and categorizing it as ‘Comedy’ because they are proposing that there’s something funny in it. Even though the content itself may not be ‘Comedy’.”
19. Classification with Using Conversation Signals 19 Type Accuracy Random Chance 23.0% Humans 60.9% YouTube Features 75.9% Social sharing patterns from Zync are highly predictive of content type. Here we can predict if a video is Sports, Comedy, Entertainment, People, or Film just from how the video is shared. This brings better recommendations for content and advertising. Zync Features 87.8%
20. Viral Content can be identified similarly and more accurately at 98.5% (F1 = 0.78) 20 Does a video have over 10M views? We did this using data from 10 people in 5 shared sessions. The drawback is you need to use a synchronous connected experience.
21. Synchronous Asynchronous One to One Many to Many Many to Object Types of Social Multimedia Interactions Many to many
22. People gather to experience events Facebook 2009 Inauguration
23. ANATOMY OF A TWEET RT: @jowyang If you are watching the debate you’re invited to participate in #tweetdebate Here is the 411 http://tinyurl.com/3jdy67 Repeated (retweet) content starts with RT Address other users with an @ Tags start with # Rich Media embeds via links
24. Inauguration 2009 http://www.flickr.com/photos/twistedart/3212723019/
25. Data Mining Feed 53,000 Tweets @ 600 per minutes
26. Drop in @conversation as onset
27. http://www.flickr.com/photos/wvs/3833148925/
28. Inauguration Importance Minute Score 0.0 0.2 0.4 0.6 0.8 1.0 0 30 60 90 Inauguration Chattiness Minute Score 0.0 0.2 0.4 0.6 0.8 1.0 0 30 60 90
29. Synchronous Asynchronous One to One Many to Many Many to Object Types of Social Multimedia Interactions Many to object (rejoice content people)
30. source: http://www.flickr.com/photos/tracer/1025272228/ Where does all of this media go? How are users consuming it? What do media acquisition patterns tell us about real-world events? Can we mine these patterns to enhance media consumption experiences? 30
31. Observations Concert videos are posted to YouTube Hundreds within a few days Browsing experience somewhat poor
32. Audio Fingerprinting provides a Socially Connected Context Audio  database
33. resultant clusters
34. relive with YouTube Multiplayer (w/ Coco)
35. relive with YouTube Multiplayer (w/ Coco) Making a Scene: Alignment of Complete Sets of Clips based on Pairwise Audio Match. Kai Su, Mor Naaman, Avadhut Gurjar, Mohsin Patel, Dan Ellis
36. Synchronous Asynchronous One to One Many to Many Many to Object
37. Title Associated description Tags: one, two, three Comments (100) Rating: ˒˒˒˒ Watching Watching Microblogs Reading Reading Posting Posting Chatting (a) Synchronous One-to-One (b) Synchronous Many-to-Many Video Event Video Object Metadata Commenting Tagging Reading Comments Reading Comments Watching Watching Remixing Scrubbing Rating (c) Asynchronous Many-to-Many (d) Asynchronous Many-to-Object
38. Fin. Thanks to J.Yew, A. Brooks, Y. Liu, S. Pentland, J. Antin, J. Dunning, Chloe S., Marc S., & M. Cameron J. Viral Actions: Predicting Video View Counts Using Synchronous Sharing Behaviors David A. Shamma; Jude Yew; Lyndon Kennedy; Elizabeth F. Churchill, ICWSM 2011 - International AAAI Conference on Weblogs and Social Media, AAAI, 2011 Knowing Funny: Genre Perception and Categorization in Social Video Sharing Jude Yew; David A. Shamma; Elizabeth F. Churchill, CHI 2011, ACM, 2011 Peaks and Persistence: Modeling the Shape of Microblog Conversations David A. Shamma; Lyndon Kennedy; Elizabeth F. Churchill, CSCW 2011, ACM, 2011 Know Your Data: Understanding Implicit Usage versus Explicit Action in Video Content Classification Jude Yew; David A. Shamma, Electronic Imaging, IS&T/SPIE, 2011 Beyond Freebird David A. Shamma, XRDS: Crossroads, ACM, 2010, 2 Conversational Shadows: Describing Live Media Events Using Short Messages David A. Shamma; Lyndon Kennedy; Elizabeth F. Churchill, International AAAI Conference on Weblogs and Social Media, AAAI, 2010 Characterizing Debate Performance via Aggregated Twitter Sentiment Nicholas A. Diakopoulos; David A. Shamma, CHI 2010, ACM, 2010 Statler: Summarizing Media through Short-Message Services David A. Shamma; Lyndon Kennedy; Elizabeth F. Churchill, CSCW, 2010 Less talk, more rock: automated organization of community-contributed collections of concert videos. Lyndon Kennedy and Mor Naaman. WWW 2009. Tweet the Debates: Understanding Community Annotation of Uncollected Sources David A. Shamma; Lyndon Kennedy; Elizabeth F. Churchill, ACM Multimedia, ACM, 2009 Understanding the Creative Conversation: Modeling to Engagement David A. Shamma; Dan Perkel; Kurt Luther, Creativity and Cognition, ACM, 2009 Spinning Online: A Case Study of Internet Broadcasting by DJs David A. Shamma; Elizabeth Churchill; Nikhil Bobb; Matt Fukuda, Communities & Technology, ACM, 2009 Zync with Me: Synchronized Sharing of Video through Instant Messaging David A. Shamma; Yiming Liu; Pablo Cesar, David Geerts, Konstantinos Chorianopoulos, Social Interactive Television: Immersive Shared Experiences and Perspectives, Information Science Reference, IGI Global, 2009 Enhancing online personal connections through the synchronized sharing of online video Shamma, D. A.; Bastéa-Forte, M.; Joubert, N.; Liu, Y., Human Factors in Computing Systems (CHI), ACM, 2008 Supporting creative acts beyond dissemination David A. Shamma; Ryan Shaw, Creativity and Cognition, ACM, 2007 Watch what I watch: using community activity to understand content David A. Shamma; Ryan Shaw; Peter Shafton; Yiming Liu, ACM Multimedia Workshop on Multimedia Information Retrival (MIR), ACM, 2007 Zync: the design of synchronized video sharing Yiming Liu; David A. Shamma; Peter Shafton; Jeannie Yang, Designing for User eXperiences, ACM, 2007