Less is More: An Empirical Investigation of the Relationship Between Amount of ‎Digital Content and User Engagement

Does more information elicit users’ compliance and engagement, or the other way around?
This paper explores the relationship between content strategy and user experience (UX). Specifically, we examine how the amount of information provided on marketing web pages, often called “landing pages,” impact users’ willingness to provide their e-mail address (a behavior called “conversion” in marketing terms). We describe the results of two large-scale online experiments (n= 535 and n= 27,900) conducted in real-world commercial settings. The observed results indicate a negative correlation between the amount of information on a web page and users’ decision-making and engagement.

Transcript

1. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA THE IMPACT OF INFORMATION AMOUNT ON ONLINE CONSUMER BEHAVIOR: EMPIRICAL EVIDENCE Nim Dvir and Ruti Gafni Virtual presentation
2. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Nim Dvir Ph.D. candidate Department of Informatics and Information Studies University at Albany - State University of New York (SUNY) Specializations: Human-computer interaction (HCI), Knowledge organization and management (KOM), Management information systems (MIS) ndvir@Albany.edu Dr. Ruti Gafni Head of the Information Systems B.Sc. program The Academic College of Tel Aviv Yaffo rutigafn@mta.ac.il https://join.skype.com/BmBGIIMzsYPR
3. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA • Does more information elicit users’ compliance and engagement, or the other way around? This paper explores the relationship between Provision of information, content strategy and user experience (UX). Less is More: An Empirical Investigation of the Relationship Between Amount of Digital Content and User Engagement
4. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Topic overview The influence of the amount of information on users’ willingness to disclose personal data • Specifically, we examine how the amount of information provided on marketing web pages, often called “landing pages,” impact users’ willingness to provide their e-mail address (a behavior called “conversion” in marketing terms). We describe the results of two large-scale online experiments (n= 535 and n= 27,900) conducted in real-world commercial settings.
5. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Topic overview • Information systems • Effective informing influences users to take desirable actions • Provision of information is a central construct in the IS discipline. • Behavioral economics • Digital nudging – Approach to how changes to the decision environment influence choices • Human-Computer Interaction (HCI) • User interface (UI) and UX design • Content is a term commonly used to describe the information made available by a website or other electronic medium. Impact experience IS BE HCI
6. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA • Literature review and rationale 1 • Experiments and findings 2 • Discussion and future work 3
7. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA What we know and what we need to know Literature review
8. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA • Method: Comprehensive, multidisciplinary review of relevant literature across various disciplines Effective informing • Tversky, A., & Kahneman, D. (1985). The Framing of Decisions and the Psychology of Choice. In Environmental Impact Assessment, Technology Assessment, and Risk Analysis (pp. 107–129). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3- 642-70634-9_6 • Cialdini, R. B., & Goldstein, N. J. (2002). The science and practice of persuasion. The Cornell Hotel and Restaurant Administration Quarterly, • Brian J. Fogg. (2013). The Web Credibility Project: Guidelines. Stanford University • Nielsen, J. (2007). Long vs. Short Articles as Content Strategy. Nielsen Norman Group. Retrieved from https://www.nngroup.com/articles /content-strategy-long-vs-short/ • Norman, D. A. (2013). The design of everyday things (Revised and expanded edition). New York, New York: Basic Books. • Menon, S., & Soman, D. (2002). Managing the power of curiosity for effective web advertising strategies. Journal of Advertising, 31(3), 1–14. Retrieved from http://www.tandfonline.com/doi/a bs/10.1080/00913367.2002.10673 672 • Norton, M. I., Frost, J. H., & Ariely, D. (2007). Less is more: The lure of ambiguity, or why familiarity breeds contempt. • Alter, A. L., & Oppenheimer, D. M. (2009). Uniting the Tribes of Fluency to Form a Metacognitive Nation. Personality and Social Psychology Review, 13(3), 219–235. Elaboration (more information) Brevity (less information
9. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA • Method: Comprehensive, multidisciplinary review of relevant literature across various disciplines Effective informing • uncertainty and risk • Trust and Credibility • Potential value • Relevance and usefulness • Quality • complexity and effort • ease of use • Cognitive Load and Limited Attention • curiosity Elaboration (more information) Brevity (less information A pertinent debate among scholars and practitioners relate to the question of information amount: Does more information elicit compliance and engagement, or the other way around?
10. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Experiment • Empirical examination • Real world (field) • In the context of landing pages – standalone web pages created explicitly for marketing or advertising campaigns.
11. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Method a between-subjects design and A/B testing. • Created two versions of landing pages: •Control - Detailed information •Treatment – no information Usefulness Trust Potential value Quality
12. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Data collection • participants (n= 535 and n= 27,900) were recruited using online advertising (Google Adwords) • Online randomized experiments (also called A/B tests, split tests) - Using Unbounce.com. • Participants were randomly directed to one of the variants. • Relevant usage metrics, such as response rate (called “conversion rate”), location, and visit time were recorded.
13. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Pilot study (n= 535) Conversion results
14. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA expanded experiments New York Atlanta Miami Los Angeles National 27,083 unique visitors
15. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA expanded experiments 42% 29% users who had less information were more inclined to provide their data.
16. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Findings Location PAGE VARIANT TOTAL UNIQUE VISITORS CONVERSION S CONVERSION RATE t National “A” (Long) 5497 1405 26% - 6.651* * “B” (Short) 5494 1718 31% Atlanta “A” (Long) 3690 1231 33% - 14.560 ** “B” (Short) 3688 1838 50% Miami “A” (Long) 1235 444 36% 2.470* * “B” (Short) 1237 639 52% Los Angeles “A” (Long) 1945 529 27% - 13.817 ** “B” (Short) 1948 937 48% New York “A” (Long) 1176 324 28% - 8.946* * “B” (Short) 1173 528 45% Total “A” (Long) 13543 3932 29% - 22.166 ** “B” (Short) 13540 5660 42% Both 27,083 9,593 Conversion results demonstrate a negative correlation between the content amount and consumer behavior Significant advantage to the pages with less content (treatment)
17. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Short Long the shorter landing pages had significantly higher conversion rates across all locations and times.
18. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Discussion 1. Information provision impact behavior • Paradoxical relationship between the increased level of information and online engagement • users that did not have any information on the service, its value, or how their data will be used still tended to convert more. • By contrast, users who were provided with this information were less inclined to convert. 2. Contrary to previous work in offline contexts, not all engagement theories are effective online 3. While not enough to establish a linear relationship, results suggesting that “less is more” can go a long way in creating commercial value. ➊ ➋ ➌
19. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Moving forward • Identify determinants - deconstruct user engagement and digital content into a set of characteristics that can be isolated and measured • Wording – Does changes in languge influence behavior? • Test other theories
20. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA Summary •Empirical evidence that content influence behavior •“Less is more” • Less content performed better •Foundation for digital content strategy • Possible to systematically facilitate behavior?
21. InSITE2018 AN INFORMING SCIENCE INSTITUTE CONFERENCE June 23-28, 2018 La Verne, California USA LET’S COLLABORATE! 1. Dvir, N., & Gafni, R. (2018). When Less Is More: Empirical Study of the Relation Between Consumer Behavior and Information Provision on Commercial Landing Pages. Informing Science: The International Journal of an Emerging Transdiscipline, 21, 019–039. doi:10.28945/4015 2. Gafni, R., & Dvir, N. (2018). How Content Volume on Landing Pages Influences Consumer Behavior: Empirical Evidence. In Proceedings of the Informing Science and Information Technology Education Conference, La Verne, California (pp. 035–053). Santa Rosa, CA: Informing Science Institute. doi:10.28945/4016 Nim Dvir ndvir@Albany.edu Ruti Gafni rutigafn@mta.ac.il https://join.skype.com/BmBGIIMzsYPR