Personalization Finder: A Search Interface for Identifying and Self-controlling Web Search Personalization

059fb717431a8cd2b509ffebc57d905a?s=47 Y. Yamamoto
August 02, 2020

Personalization Finder: A Search Interface for Identifying and Self-controlling Web Search Personalization

Yusuke Yamamoto and Takehiro Yamamoto. 2020. Personalization Finder: A Search Interface for Identifying and Self-controlling Web Search Personalization. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 (JCDL ’20). Association for Computing Machinery, New York, NY, USA, 37–46. DOI:https://doi.org/10.1145/3383583.3398519

059fb717431a8cd2b509ffebc57d905a?s=128

Y. Yamamoto

August 02, 2020
Tweet

Transcript

  1. Personalization Finder: A Search Interface for Identifying and Self-controlling Web

    Search Personalization JCDL2020 AP-L-2: User in Search Session Y. Yamamoto et. al: “Personalization Finder: A Search Interface for Identifying and Self-controlling Web Search Personalization”, Proceedings of the 20th ACM/IEEE on Joint Conference on Digital Libraries (JCDL 2020), China, Xi’an, August 2, 2020 Takehiro Yamamoto University of Hyogo, Japan t.yamamoto@sis.u-hyogo.ac.jp Yusuke Yamamoto Shizuoka University, Japan yusuke_yamamoto@acm.org
  2. Risk of personalization technology Recommended information = Biased information Personalization

    can cause a state of information isolation Filter bubble
  3. How much do web search engines personalize search results? DuckDuckGo’s

    Survey: Measuring the "Filter Bubble” - How Google is influencing what you click, https://spreadprivacy.com/google-filter-bubble-study/ Compared with 87 people’s Google results for “Gun Control” in private browsing mode They saw 19 domains ordered 31 ways DuckDuckGo Survey
  4. PERSONALIZATION Amplification of bias Topics where preferences can differ significantly

    Effective search Is personalization harmful? Topics where intellectual isolation is detrimental to society
  5. PERSONALIZATION Amplification of bias Topics where preferences can differ significantly

    Effective search Is personalization harmful? Topics where intellectual isolation is detrimental to society • Consideration of personalizaiton in web search • Controlling of personalization according to topics Necessary supports
  6. Objectives of our study User survey about people’s perception of

    web search personalization (WSP) • Do they know the existence of WSP? • When and how do they want to use WSP? Designing a web search interface to visualize and control WSP For promoting critical web search against filter bubble
  7. 1 People’s perception of Web search personalization

  8. Willingness to use web search personalization 0 20 40 60

    80 100 Shopping Entertainment Health Sports C lt re Science/Tech Nat re Economics Politics Ver illing Willing Ne tral Un illing Ver n illing Q. How willing are you to use personalization when using a web search engine?
  9. Willingness to use web search personalization 0 20 40 60

    80 100 Shopping Entertainment Health Sports C lt re Science/Tech Nat re Economics Politics Ver illing Willing Ne tral Un illing Ver n illing Q. How willing are you to use personalization when using a web search engine? 55% of participants were willing to use the personalization for entertainment/shopping
  10. Willingness to use web search personalization 0 20 40 60

    80 100 Shopping Entertainment Health Sports C lt re Science/Tech Nat re Economics Politics Ver illing Willing Ne tral Un illing Ver n illing 62/50% of participants were NOT willing to use the personalization for political/economic topics Q. How willing are you to use personalization when using a web search engine?
  11. Perception of web search personalization 0 20 40 60 80

    100 Shopping En e ainmen Heal h Spo C l e Science/Tech Na e Economic Poli ic > 50% 40-50% 30-40% 20-30% 10-20% < 10% Q. What percentage of web search results do you think that search engines personalize?
  12. Perception of web search personalization 0 20 40 60 80

    100 Shopping En e ainmen Heal h Spo C l e Science/Tech Na e Economic Poli ic > 50% 40-50% 30-40% 20-30% 10-20% < 10% Q. What percentage of web search results do you think that search engines personalize? Many participants noticed that the personalization often worked in the results of entertainment/shopping
  13. Perception of web search personalization 0 20 40 60 80

    100 Shopping En e ainmen Heal h Spo C l e Science/Tech Na e Economic Poli ic > 50% 40-50% 30-40% 20-30% 10-20% < 10% Q. What percentage of web search results do you think that search engines personalize? 30% of participants believed that few(less than 10%) results were personalized for politics/economics.
  14. Summary of user survery 0 20 40 60 80 100

    Shopping Entertainment Health Sports C lt re Science/Tech Nat re Economics Politics Ver illing Willing Ne tral Un illing Ver n illing 0 20 40 60 80 100 Shopping En e ainmen Heal h Spo C l e Science/Tech Na e Economic Poli ic > 50% 40-50% 30-40% 20-30% 10-20% < 10% In reality, web search engines personalize such results… & People want search results for politics/economics topics not to be personalized. People often believe that such results are not personalized.
  15. 2 Search Interface for Identifying and Self- controlling Web Search

    Personalization
  16. Personalization Finder Google Chrome extension to visualize and control Web

    search personalization
  17. Personalization Finder Google Chrome extension to visualize and control Web

    search personalization Makes searchers aware of personalization and promote critical web search
  18. Function 1: Exposure of personalized results

  19. Function 1: Exposure of personalized results

  20. Function 1: Exposure of personalized results Highlighting of personalized results

    on SERPs Disclosure of search results hidden by the personalization
  21. Function 1: Exposure of personalized results Highlighting of personalized results

    on SERPs Disclosure of search results hidden by the personalization
  22. Function 1: Exposure of personalized results Highlighting of personalized results

    on SERPs Disclosure of a search result hidden by the personalization
  23. Function 2: Summarization of the personalization effect on a sidebar

  24. Function 2: Summarization of the personalization effect on a sidebar

  25. Function 2: Summarization of the personalization effect on a sidebar

    Extent of personalization 1. Topic difference between personalized results and hidden results 2. Ratio of personalized results in the top 100 ones. What topics frequently appear in hidden search results? 53%
  26. How to detect personalized search results? Google Custom Search API

    1. Issues the same query as users input on Google site “Casino law” 2. Obtains API results. (We assume that such results not personalized) API search results 3. Compares user’s results on Google site with API results. (If a user’s result doesn’t appear in API results, we assume that it could be personalized)
  27. 3 User Study

  28. Hypotheses on Personalization Finder (PF) 29 H1 H2 H3 With

    PF, users will spend more time searching the web more carefully than default web searches PF will encourage users to visit more webpages to obtain more information The effect of PF will be greater for political topics than entertainment topics
  29. Search Task 30 • 220 Participants did web search to

    answer questions about political/entertainment topics. l/VDMFBS QPXFS QIBTFPVU QPMJDZl JT POF PG UIF NPTU DPOUSPWFSTJBM UPQJDT JO +BQBO 1MFBTF JNBHF UIBU ZPV IBWF CFFO BTLFE UP BOTXFS JG ZPV BHSFF PS EJTBHSFF XJUI OVDMFBS QPXFS QIBTFPVU QPMJDZ "TTVNF UIBU ZPV DPMMFDU JOGPSNBUJPO BCPVU UIF QPMJDZ WJB B XFC TFBSDI UP GPSNVMBUF ZPVS PQJOJPO 4FFL B EJTQMBZFE MJTU PG XFC TFBSDI SFTVMUT BOE WJTJU XFC QBHFT PO UIF MJTU 8IFO ZPV HFU TBUJTGBDUJPO DPODMVTJPO TUPQ UIF TFBSDI BOE SFQPSU ZPVS GJOBM PQJOJPO XJUI SFBTPOT • When they reached satisfactory conclusion, they reported their final opinion with reasons.
  30. Experimental design 31 2 x 2 mixed factorial design UI

    (between-subject) Search topics (within-subject) • Control UI = default Google Search UI • Experimental UI = Control UI + proposed function • Politics • Entertainment
  31. Two search interfaces 32 Control • Default Google Search UI.

    • Number of displayed results per page was fixed to 100 Experimental has proposed functions on Control UI ※ Knowledge graph and vertical search results were removed from both conditions’ results
  32. Result 1: Dwell time on SERPs Experimental UI made users

    spend longer browsing SERPs for both political/entertainment topics. • • • • 0 50 100 150 200 entertainment politics Task type Dwell time on SERP UI Condition • • Control Experimental • • • • 0 200 400 600 800 entertainment politics Task type Session time UI Condition • • Control Experimental UP UP
  33. Result 2: Click probability by rank position (1/3) What percentage

    of participants clicked each search result by rank position? Q.
  34. Result 2: Click probability by rank position (1/3) Experimental UI

    made more users shallower click search results positions for entertainment topics.
  35. Result 2: Click probability by rank position (2/3) Experimental UI

    made more users click deeper search results for political topics.
  36. Result 2: Click probability by rank position (3/3) Experimental UI

    did NOT encourage users to view more webpages. If users searched for political topics with the experimental UI, they were likely to seek SERPs to lower positions
  37. Result 3: Exit questionnaire Question Control UI Exp. UI Usefulness

    for objective information collection 2.95 (0.70) 3.28* (0.88) Usefulness for objective decision making 2.80 (0.75) 2.98 (0.79) Ease of use 3.10 (0.89) 3.26 (0.96) Willingness for continuous use 2.80 (0.90) 3.09* (0.90) Experimental UI could promote objective information seeking without reducing usability. ✓ ✓
  38. Hypotheses verification (1/3) H1 With PF, users will spend more

    time searching the web more carefully than default web searches A. Partially Yes - PF made participants spend longer in SERPs - It did NOT make them view webpages longer
  39. Hypotheses verification (2/3) H2 A. No, but.. - PF made

    users click search results with deeper positions for political topics. - Users with PF clicked search results with shallower positions for entertainment topics. PF will encourage users to visit more webpages to obtain more information
  40. Hypotheses verification (3/3) H3 A. Partially Yes - PF had

    a completely different influence on click probability by rank position. The effect of PF will be greater for political topics than entertainment topics
  41. Discussion Belief change through web search o PF made users

    aware of personalization effect and examine search result list longer, but it did not change their belief about topics. Personalization detection accuracy o The prototype uses Google Search API to approximate the difference between user’s search results on the Google website and non-personalized search results. o The API results are slightly older than original Google results, and so the prototype may detect personalized results incorrectly o One possible improvement would be to provide supplementary information which can make people view unfavorable information
  42. Summary Personalization Finder: A Search Interface for Identifying and Self-controlling

    Web Search Personalization 44 • It made participants seek SERPs with lower positions in searching for political topics • The prototype made participants spent longer in SERPs for political/entertainment topics • A lot of people believed that web search results about political topics are NOT personalized. Email for questions: yusuke_yamamoto@acm.org