Twitter bot surveys: A discrete choice experiment to increase response rates

Juan Pablo Alperin, Erik W. Hanson, Kenneth Shores, Stefanie Haustein.

Presented at the International Conference on Social Media & Society in Toronto, Canada.

Full paper available from the Proceedings of the 8th International Conference on Social Media & Society, Article No. 27, https://doi.org/10.1145/3097286.3097313

Transcript

1. Juan Pablo Alperin
   @juancommander
   Erik W. Hanson, Kenneth Shores, Stefanie Haustein
   https://doi.org/10.1145/3097286.3097313
   Twitter bot surveys:
   A discrete choice experiment to
   increase response rates
   #scholcommlab
   

   View Slide

2. View Slide

3. View Slide

4. 36/49% response rate
   80,000/175,000+ responses
   @juancommander | #scholcommlab
   

   View Slide

5. scielo: individual answers
   @juancommander | #scholcommlab
   

   View Slide

6. but on
   social media?
   @juancommander | #scholcommlab
   

   View Slide

7. View Slide

8. 6397 Twitter accounts had linked to SciELO article
   6093 were still active (this was 2 years later)
   286 responded (5%)
   @juancommander | #scholcommlab
   

   View Slide

9. 6397 Twitter accounts had linked to SciELO article
   6093 were still active (this was 2 years later)
   286 responded (5%)
   ok. not so great.
   @juancommander | #scholcommlab
   

   View Slide

10. @juancommander | #scholcommlab
    

    View Slide

11. @juancommander | #scholcommlab
    

    View Slide

12. @juancommander | #scholcommlab
    

    View Slide

13. @juancommander | #scholcommlab
    

    View Slide

14. @juancommander | #scholcommlab
    

    View Slide

15. @juancommander | #scholcommlab
    

    View Slide

16. @juancommander | #scholcommlab
    

    View Slide

17. but
    @juancommander | #scholcommlab
    

    View Slide

18. but
    5% response rate
    @juancommander | #scholcommlab
    

    View Slide

19. pop-up surveys microsurveys
    widely used in industry
    not well documented in academia
    

    View Slide

20. benchmark response rates
    @juancommander | #scholcommlab
    pop-up surveys: ~20%
    (Couper, 2001)
    links on YouTube: 11-16%
    (Courtois, 2010; Courtois & Mechant, 2014)
    web surveys : ~11%
    (Manfreda et al., 2008)
    

    View Slide

21. Egoistic appeals
    (Pedersen & Nielsen, 2016)
    Question types
    (Schwanda-Sosik et al. 2014)
    Providing context
    (Liu, M. et al. 2015)
    what affects response rates?
    @juancommander | #scholcommlab
    

    View Slide

22. @juancommander | #scholcommlab
    

    View Slide

23. # Question types Question text
    1
    Open
    ended
    No egoistic
    appeal
    Context @screenname Please help us understand New Yorker readers.
    What format do you prefer to read it in? Thanks!
    2 No context Please help us understand New Yorker readers.
    What format do you prefer to read it in? Thanks @screenname!
    3
    Yes/no
    Context @screenname Please help us understand New Yorker readers.
    Do you read it primarily online? Thanks!
    4 No context Please help us understand New Yorker readers.
    Do you read it primarily online? Thanks @screenname!
    5
    Multiple
    choice
    Context @screenname Please help us understand New Yorker readers.
    Do you read it online, in print, or both? Thanks!
    6 No context Please help us understand New Yorker readers.
    Do you read it online, in print, or both? Thanks @screenname!
    7
    Open
    ended
    Egoistic
    appeal
    Context
    @screenname You recently tweeted a New Yorker article, could you tell
    us:
    What format do you prefer to read it in? Thanks!
    8 No context You recently tweeted a New Yorker article, could you tell us:
    What format do you prefer to read it in? Thanks @screenname!
    9
    Yes/no
    Context
    @screenname You recently tweeted a New Yorker article, could you tell
    us:
    Do you read it primarily online? Thanks!
    10 No context You recently tweeted a New Yorker article, could you tell us:
    Do you read it primarily online? Thanks @screenname!
    11
    Multiple
    choice
    Context
    @screenname You recently tweeted a New Yorker article, could you tell
    us:
    Do you read it online, in print, or both? Thanks!
    12 No context You recently tweeted a New Yorker article, could you tell us:
    Do you read it online, in print, or both? Thanks @screenname!
    @juancommander | #scholcommlab
    

    View Slide

24. #12
    #5
    #2
    #8
    @juancommander | #scholcommlab
    

    View Slide

25. # Question types N Response rate
    1
    Open ended
    No egoistic
    appeal
    Context 119 16.8%
    2 No context 112 12.5%
    3
    Yes/no
    Context 116 17.2%
    4 No context 111 9.9%
    5 Multiple
    choice
    Context 106 22.6%
    6 No context 112 21.4%
    7
    Open ended
    Egoistic appeal
    Context 110 28.2%
    8 No context 109 22.0%
    9
    Yes/no
    Context 114 29.0%
    10 No context 105 24.8%
    11 Multiple
    choice
    Context 110 40.0%
    12 No context 107 35.5%
    1,331 23.2%
    @juancommander | #scholcommlab
    

    View Slide

26. Question type
    Open-ended 19.9%
    Yes/no 20.2%
    Multiple choice 29.9%
    Egoistic appeal
    No 16.8%
    Yes 29.9%
    Reply/mention
    Reply 25.6%
    Mention 21.0%
    @juancommander | #scholcommlab
    

    View Slide

27. next steps
    @juancommander | #scholcommlab
    

    View Slide

28. time related variants
    - time and day
    - how long since event
    profile information
    - identify as bot?
    - person or institution?
    combine with ’traditional’
    social media research
    @juancommander | #scholcommlab
    

    View Slide

29. Biology and Evolutionary Biology
    11 articles
    > 50 tweets
    published 2015
    @juancommander | #scholcommlab
    

    View Slide

30. First tweet Last tweet
    In-degree
    (number of followers)
    Gradual and contingent evolutionary emergence of
    leaf mimicry in butterfly wing patterns
    @juancommander | #scholcommlab
    

    View Slide

31. First tweet Last tweet
    In-degree
    (number of followers)
    “Cancer, metabolism, fructose, artificial
    sweeteners, and going cold turkey on sugar”
    @juancommander | #scholcommlab
    

    View Slide

32. @juancommander
    #ScholCommLab
    thank you.
    Learn more at: https://www.scholcommlab.ca
    

    View Slide