Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Twitter bot surveys: A discrete choice experiment to increase response rates

Twitter bot surveys: A discrete choice experiment to increase response rates

Juan Pablo Alperin, Erik W. Hanson, Kenneth Shores, Stefanie Haustein.

Presented at the International Conference on Social Media & Society in Toronto, Canada.

Full paper available from the Proceedings of the 8th International Conference on Social Media & Society, Article No. 27, https://doi.org/10.1145/3097286.3097313

Juan Pablo Alperin

July 28, 2017
Tweet

More Decks by Juan Pablo Alperin

Other Decks in Science

Transcript

  1. Juan Pablo Alperin
    @juancommander
    Erik W. Hanson, Kenneth Shores, Stefanie Haustein
    https://doi.org/10.1145/3097286.3097313
    Twitter bot surveys:
    A discrete choice experiment to
    increase response rates
    #scholcommlab

    View Slide

  2. View Slide

  3. View Slide

  4. 36/49% response rate
    80,000/175,000+ responses
    @juancommander | #scholcommlab

    View Slide

  5. scielo: individual answers
    @juancommander | #scholcommlab

    View Slide

  6. but on
    social media?
    @juancommander | #scholcommlab

    View Slide

  7. View Slide

  8. 6397 Twitter accounts had linked to SciELO article
    6093 were still active (this was 2 years later)
    286 responded (5%)
    @juancommander | #scholcommlab

    View Slide

  9. 6397 Twitter accounts had linked to SciELO article
    6093 were still active (this was 2 years later)
    286 responded (5%)
    ok. not so great.
    @juancommander | #scholcommlab

    View Slide

  10. @juancommander | #scholcommlab

    View Slide

  11. @juancommander | #scholcommlab

    View Slide

  12. @juancommander | #scholcommlab

    View Slide

  13. @juancommander | #scholcommlab

    View Slide

  14. @juancommander | #scholcommlab

    View Slide

  15. @juancommander | #scholcommlab

    View Slide

  16. @juancommander | #scholcommlab

    View Slide

  17. but
    @juancommander | #scholcommlab

    View Slide

  18. but
    5% response rate
    @juancommander | #scholcommlab

    View Slide

  19. pop-up surveys microsurveys
    widely used in industry
    not well documented in academia

    View Slide

  20. benchmark response rates
    @juancommander | #scholcommlab
    pop-up surveys: ~20%
    (Couper, 2001)
    links on YouTube: 11-16%
    (Courtois, 2010; Courtois & Mechant, 2014)
    web surveys : ~11%
    (Manfreda et al., 2008)

    View Slide

  21. Egoistic appeals
    (Pedersen & Nielsen, 2016)
    Question types
    (Schwanda-Sosik et al. 2014)
    Providing context
    (Liu, M. et al. 2015)
    what affects response rates?
    @juancommander | #scholcommlab

    View Slide

  22. @juancommander | #scholcommlab

    View Slide

  23. # Question types Question text
    1
    Open
    ended
    No egoistic
    appeal
    Context @screenname Please help us understand New Yorker readers.
    What format do you prefer to read it in? Thanks!
    2 No context Please help us understand New Yorker readers.
    What format do you prefer to read it in? Thanks @screenname!
    3
    Yes/no
    Context @screenname Please help us understand New Yorker readers.
    Do you read it primarily online? Thanks!
    4 No context Please help us understand New Yorker readers.
    Do you read it primarily online? Thanks @screenname!
    5
    Multiple
    choice
    Context @screenname Please help us understand New Yorker readers.
    Do you read it online, in print, or both? Thanks!
    6 No context Please help us understand New Yorker readers.
    Do you read it online, in print, or both? Thanks @screenname!
    7
    Open
    ended
    Egoistic
    appeal
    Context
    @screenname You recently tweeted a New Yorker article, could you tell
    us:
    What format do you prefer to read it in? Thanks!
    8 No context You recently tweeted a New Yorker article, could you tell us:
    What format do you prefer to read it in? Thanks @screenname!
    9
    Yes/no
    Context
    @screenname You recently tweeted a New Yorker article, could you tell
    us:
    Do you read it primarily online? Thanks!
    10 No context You recently tweeted a New Yorker article, could you tell us:
    Do you read it primarily online? Thanks @screenname!
    11
    Multiple
    choice
    Context
    @screenname You recently tweeted a New Yorker article, could you tell
    us:
    Do you read it online, in print, or both? Thanks!
    12 No context You recently tweeted a New Yorker article, could you tell us:
    Do you read it online, in print, or both? Thanks @screenname!
    @juancommander | #scholcommlab

    View Slide

  24. #12
    #5
    #2
    #8
    @juancommander | #scholcommlab

    View Slide

  25. # Question types N Response rate
    1
    Open ended
    No egoistic
    appeal
    Context 119 16.8%
    2 No context 112 12.5%
    3
    Yes/no
    Context 116 17.2%
    4 No context 111 9.9%
    5 Multiple
    choice
    Context 106 22.6%
    6 No context 112 21.4%
    7
    Open ended
    Egoistic appeal
    Context 110 28.2%
    8 No context 109 22.0%
    9
    Yes/no
    Context 114 29.0%
    10 No context 105 24.8%
    11 Multiple
    choice
    Context 110 40.0%
    12 No context 107 35.5%
    1,331 23.2%
    @juancommander | #scholcommlab

    View Slide

  26. Question type
    Open-ended 19.9%
    Yes/no 20.2%
    Multiple choice 29.9%
    Egoistic appeal
    No 16.8%
    Yes 29.9%
    Reply/mention
    Reply 25.6%
    Mention 21.0%
    @juancommander | #scholcommlab

    View Slide

  27. next steps
    @juancommander | #scholcommlab

    View Slide

  28. time related variants
    - time and day
    - how long since event
    profile information
    - identify as bot?
    - person or institution?
    combine with ’traditional’
    social media research
    @juancommander | #scholcommlab

    View Slide

  29. Biology and Evolutionary Biology
    11 articles
    > 50 tweets
    published 2015
    @juancommander | #scholcommlab

    View Slide

  30. First tweet Last tweet
    In-degree
    (number of followers)
    Gradual and contingent evolutionary emergence of
    leaf mimicry in butterfly wing patterns
    @juancommander | #scholcommlab

    View Slide

  31. First tweet Last tweet
    In-degree
    (number of followers)
    “Cancer, metabolism, fructose, artificial
    sweeteners, and going cold turkey on sugar”
    @juancommander | #scholcommlab

    View Slide

  32. @juancommander
    #ScholCommLab
    thank you.
    Learn more at: https://www.scholcommlab.ca

    View Slide