Research on data journalism: What is there to investigate? Insights from a structured literature review

Presented: April 21, 2016; University of Helsinki
at the Northern Data Journalism Conference (NODA16) Academic Pre-Conference

This presentation aims at exploring the existing research literature on data journalism. Over the past years this emerging journalistic practice has been established and has also attracted significant attention from journalism scholars. It was time to take a closer look at the existing research literature in order to find out more about how this literature has been developing. Where are the research gaps and what does the future of data journalism research hold? These questions were tackled by carefully selecting a corpus of scholarly literature with empirical foundation in data journalism. This corpus was analyzed with a mixed method approach using qualitative and quantitative techniques. In this way the development of the literature over time could be illustrated and the most influential publications could be identified. Often-used theoretical frameworks and the applied research designs hinted at certain tendencies and gaps in the research literature on data journalism, for example, the dominance of qualitative research design over quantitative ones. Also, a shortcoming of cross-national investigations and ethnographic studies became visible.

Transcript

1. MEDIA & DESIGN
   Research on data journalism:
   What is there to investigate?
   Insights from a structured literature review
   Julian Ausserhofer1,2,3, Robert Gutounig1, Michael Oppermann2,
   Sarah Matiasek1,2 & Eva Goldgruber1
   1: FH Joanneum University of Applied Sciences, Graz

   2: University of Vienna

   3: Humboldt Institute for Internet and Society, Berlin
   NODA16 Academic Pre-Conference

   #NODA16

   21.04.2016, University of Helsinki
   

   View Slide

2. MEDIA & DESIGN
   *Tool evaluation partners
   *
   *
   *
   Supported by:
   

   View Slide

3. MEDIA & DESIGN
   Research Interest
   

   View Slide

4. MEDIA & DESIGN
   @julauss
   Research literature on data journalism
   2013: "internalist tendencies at [... the] early
   stage of academic research" (Anderson, 2013, p. 1007)
   ↓

   

   2015: "an explosion in data journalism-
   oriented scholarship" (Fink & Anderson, 2015, p. 476)*

   

   "rapidly growing body" of scientific studies 

   (Lewis, 2015, p. 322)*

   

   

   

   *cited via Loosen, Reimer & Schmidt (2015, p. 2)
   4
   #NODA16
   

   View Slide

5. MEDIA & DESIGN
   How is the research literature developing?
   What are the research gaps?
   Research questions
   @julauss 5
   #NODA16
   

   View Slide

6. MEDIA & DESIGN
   Method
   

   View Slide

7. MEDIA & DESIGN
   Structured literature review
   @julauss 7
   #NODA16
   

   View Slide

8. MEDIA & DESIGN
   "to develop insights, critical reflections,
   future research paths and research questions" 

   (Massaro, Dumay & Guthrie, forthcoming)
   It adopts "a replicable, scientific and
   transparent process [...] that aims to
   minimize bias [...]"

   (Tranfield, Denyer & Smart, 2003)
   Why a structured literature review?
   @julauss 8
   #NODA16
   

   View Slide

9. MEDIA & DESIGN
   Undertaking a systematic literature review
   9
   Adapted from Massaro et al. (forthcoming)
   Writing a literature review protocol
   Developing
   insights and
   critique
   through
   analyzing
   the dataset
   Developing
   future
   research paths
   and questions
   Determining
   the type of
   studies and
   carrying out a
   comprehensiv
   e literature
   search
   Coding data
   Defining the
   questions
   that the
   literature
   review
   should
   answer
   @julauss
   #NODA16
   

   View Slide

10. MEDIA & DESIGN
    10
    Developing
    insights and
    critique
    through
    analyzing
    the dataset
    Developing
    future
    research
    paths and
    questions
    Coding data
    Defining the
    questions
    that the
    literature
    review
    should
    answer
    Determining
    the type of
    studies and
    carrying out a
    comprehensiv
    e literature
    search
    ● Empirical research on DDJ
    ● Social science focus, but open to other
    disciplines
    ● Published after 1995
    Included
    Journal articles
    Book sections
    Conference papers
    Reports (from industry and
    research projects)
    PhD theses
    Not included
    Bachelor's and Master's theses
    Press reports
    Blog posts
    @julauss
    #NODA16
    

    View Slide

11. MEDIA & DESIGN
    11
    Developing
    insights and
    critique
    through
    analyzing
    the dataset
    Developing
    future
    research
    paths and
    questions
    Coding data
    Defining the
    questions
    that the
    literature
    review
    should
    answer
    Determining
    the type of
    studies and
    carrying out a
    comprehensiv
    e literature
    search
    ● Preliminary search with “data-driven
    journalism”
    ● Extracting related terms from the keyword
    section of research papers
    @julauss
    #NODA16
    

    View Slide

12. MEDIA & DESIGN
    12
    Developing
    insights and
    critique
    through
    analyzing
    the dataset
    Developing
    future
    research
    paths and
    questions
    Coding data
    Defining the
    questions
    that the
    literature
    review
    should
    answer
    Determining
    the type of
    studies and
    carrying out a
    comprehensiv
    e literature
    search
    Search terms
    algorithmic journalism
    computational journalism
    computer-assisted reporting
    data journalism
    data-driven journalism
    data-driven reporting
    database journalism
    datajournalism
    datenjournalismus
    quantitative journalism
    No search terms
    accountability journalism
    crowdsourced journalism
    dataviz
    datavis
    ddj
    drone journalism
    investigative journalism
    online journalism
    open journalism
    @julauss
    #NODA16
    

    View Slide

13. MEDIA & DESIGN
    13
    Developing
    insights and
    critique
    through
    analyzing
    the dataset
    Developing
    future
    research
    paths and
    questions
    Coding data
    Defining the
    questions
    that the
    literature
    review
    should
    answer
    Determining
    the type of
    studies and
    carrying out a
    comprehensiv
    e literature
    search
    Scientific Databases
    ACM Digital Sowiport
    EBSCO Springer
    IEEE SpringerLink
    JSTOR Taylor & Francis Online
    ProQuest Web of Science
    Science Direct Wiley
    Scopus Google Scholar
    Sociological Abstracts
    @julauss
    #NODA16
    

    View Slide

14. MEDIA & DESIGN
    14
    Developing
    insights and
    critique
    through
    analyzing
    the dataset
    Developing
    future
    research
    paths and
    questions
    Coding data
    Defining the
    questions
    that the
    literature
    review
    should
    answer
    Determining
    the type of
    studies and
    carrying out a
    comprehensiv
    e literature
    search
    772 search results
    ↓
    Assessment of title, abstract & keywords
    - by two independently working researchers

    (Thomas et al., 2004)
    ↓
    33 research publications
    @julauss
    #NODA16
    

    View Slide

15. MEDIA & DESIGN
    15
    @julauss
    #NODA16
    Records identified
    from scientific
    databases
    (n= 772)
    Further publications
    from expert poll of
    data journalism
    researchers (n= 4)
    Excluded after
    screening
    (n= 739)
    Preliminary corpus:
    publications included
    after screening of
    records (n= 33)
    References from
    preliminary corpus
    (n = 1151)
    Final corpus: Publications
    included in the systematic
    review (n=40)
    Excluded after
    screening
    (n= 1148)
    Further publications
    included after screening
    of references
    (n = 3)
    Adapted from Fecher, Friesike & Hebing (2015)
    

    View Slide

16. MEDIA & DESIGN
    16
    @julauss
    #NODA16
    software-assisted qualitative
    content analysis
    (Kaefer, Roper, & Sinha, 2015; Mayring, 2000;
    Schreier, 2012; QSR International, 2015)
    computational analysis of
    structural aspects

    (Kreibich, 2016; Lopez, 2009)
    

    View Slide

17. MEDIA & DESIGN
    Results
    

    View Slide

18. MEDIA & DESIGN
    Development of the literature over time
    n=40
    @julauss 18
    #NODA16
    

    View Slide

19. MEDIA & DESIGN
    

    Publications by type and citations
    n=40
    bubble size = number of citations in Google Scholar
    @julauss 19
    #NODA16
    

    View Slide

20. MEDIA & DESIGN
    Affiliations & collaborations
    @julauss 20
    #NODA16
    

    View Slide

21. MEDIA & DESIGN
    1787-2015

    n=1644
    @julauss
    

    References per year
    21
    #NODA16
    

    View Slide

22. MEDIA & DESIGN
    Most-cited references
    Publication
    Nr. of
    Citations
    Meyer, P. (2002/1973). Precision journalism: A reporter’s introduction to social
    science methods (4th ed.). Oxford: Rowman & Littlefield.
    15
    Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good:
    “Computer-assisted-reporters” and “programmer-journalists” in Chicago.
    New Media & Society, 15(6), 853–871.
    15
    Gray, J., Bounegru, L., & Chambers, L. (Eds.). (2012). The data journalism
    handbook: How journalists can use data to improve the news. Sebastopol:
    O’Reilly.
    13
    @julauss 22
    #NODA16
    

    View Slide

23. MEDIA & DESIGN
    @julauss
    Theoretical frames
    ● Science and technology studies
    ● Actor network theory

    (Ausserhofer, 2015; De Maeyer, Libert, Domingo, Heinderyckx, & Le Cam, 2015;
    Parasie & Dagiral, 2013; Parasie, 2015)

    23
    #NODA16
    

    View Slide

24. MEDIA & DESIGN
    @julauss
    Research designs & data collection methods
    Method Nr. of studies
    In-depth interviews 25
    Content analysis 21
    Survey 5
    Short-term observation 3
    Newsroom ethnography 1
    Note. Content analysis includes analysis of news, databases, blogs, job ads,
    visualizations, briefings, manuals, and more. Short-term observation encompasses visits
    to the newsroom and participation in meetings. A newsroom ethnography is defined as a
    detailed study of a newsroom over the course of several days.
    24
    #NODA16
    

    View Slide

25. MEDIA & DESIGN
    Geographical scope
    Country
    Number of
    studies
    United States 16
    United Kingdom 14
    Germany 5
    International 3
    n/a 3
    Sweden 2
    Switzerland 2
    Norway 2
    Netherlands 2
    … …
    @julauss 25
    #NODA16
    

    View Slide

26. MEDIA & DESIGN
    @julauss 26
    #NODA16
    Research gaps in data journalism research
    ● Comparision of practices between countries 

    (Appelgren & Nygren, 2014; Parasie & Dagiral, 2013)
    ● Long-term studies (Davenport, 2000; Knight, 2015)
    ● Newsroom ethnographies (Parasie & Dagiral, 2013)
    ● Software studies (Garrison, 1999; Lewis, 2013; Stavelin, 2013)
    ● Reader experience studies (Segel & Heer, 2010)
    

    View Slide

27. MEDIA & DESIGN
    Conclusion
    

    View Slide

28. MEDIA & DESIGN
    @julauss 28
    #NODA16
    ● data journalism and its investigation has been
    developing rapidly
    ● quality improvements in the research
    ● issues with the literature: few publications
    refer to theory or methodology, just report
    what has been investigated
    Conclusion
    

    View Slide

29. MEDIA & DESIGN
    @julauss 29
    #NODA16
    ● practices in small news organizations, freelancers, local
    and mobile data journalism etc.
    ● gender
    ● digital methods: investigating the field through its
    platforms
    ● theory
    ● …
    Research opportunities
    

    View Slide

30. MEDIA & DESIGN
    Explore the literature online at:
    http://literature.validproject.at
    Julian Ausserhofer1,2,3,

    julian.ausserhofer@fh-joanneum.at
    @julauss
    Robert Gutounig1,

    @sextus_empirico
    Michael Oppermann2,

    @oppermann_m
    Sarah Matiasek1,2 &

    @sarahmatiasek
    Eva Goldgruber1

    @evagoldgruber
    1: FH Joanneum University of Applied Sciences, Graz

    2: University of Vienna

    3: Humboldt Institute for Internet and Society, Berlin
    NODA16 Academic Pre-Conference

    #NODA16

    21.04.2016, University of Helsinki
    

    View Slide

31. MEDIA & DESIGN
    References
    31
    

    View Slide

32. MEDIA & DESIGN
    @julauss 32
    #NODA16
    Anderson, C. W. (2013). Towards a sociology of computational and algorithmic journalism. New Media &
    Society, 15(7), 1005–1021. doi: 10.1177/1461444812465137
    Appelgren, E., & Nygren, G. (2014). Data journalism in Sweden: Introducing new methods and genres of
    journalism into “old” organizations. Digital Journalism, 2(3), 394–405. doi:
    10.1080/21670811.2014.884344
    Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im
    Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale
    Methoden in der Kommunikationswissenschaft (pp. 87–111). Berlin: digitalcommunicationresearch.de.
    doi: 10.17174/dcr.v2.5
    Davenport, L., Fico, F., & Detwiler, M. (2000). Computer–assisted reporting in Michigan daily newspapers:
    More than a decade of adoption. Presented at the Association for Education in Journalism and Mass
    Communication (AEJMC) National Convention, Phoenix, Arizona.
    De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for data journalism:
    A qualitative assessment of the anecdotal take-up of data journalism in French-speaking Belgium.
    Digital Journalism, 3(3), 432–446. doi: 10.1080/21670811.2014.976415
    Fecher, B., Friesike, S., & Hebing, M. (2015). What drives academic data sharing? PLoS ONE, 10(2),
    e0118053. doi: 10.1371/journal.pone.0118053
    Fink, K., & Anderson, C. W. (2015). Data journalism in the United States: Beyond the “usual suspects.”
    Journalism Studies, 16(4), 467–481. doi: 10.1080/1461670X.2014.939852
    Garrison, B. (1999). Newspaper size as a factor in use of computer-assisted reporting. Newspaper
    Research Journal, 20(3). Retrieved from http://com.miami.edu/car/baltimore1.htm
    Kaefer, F., Roper, J., & Sinha, P. (2015). A software-assisted qualitative content analysis of news articles:
    example and reflections. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research,
    16(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/2123
    Knight, M. (2015). Data journalism in the UK: A preliminary analysis of form and content. Journal of Media
    Practice, 16(1), 55–72. doi: 10.1080/14682753.2015.1015801
    Kreibich, C. (2016). scholar.py. Retrieved from https://github.com/ckreibich/scholar.py
    

    View Slide

33. MEDIA & DESIGN
    @julauss 33
    #NODA16
    Lewis, S. C. (2015). Journalism in an era of big data. Digital Journalism, 3(3), 321–330. doi:
    10.1080/21670811.2014.976399
    Loosen, W., Reimer, J., & Schmidt, F. (2015). When data become news: A content analysis of data
    journalism pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff.
    Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for
    scholarship publications. In M. Agosti, J. Borbinha, S. Kapidakis, C. Papatheodorou, & G. Tsakonas
    (Eds.), Research and Advanced Technology for Digital Libraries (pp. 473–474). Berlin: Springer. Retrieved
    from doi: 10.1007/978-3-642-04346-8_62
    Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the shoulders of giants: undertaking a
    structured literature review in accounting. Accounting, Auditing & Accountability Journal.
    Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative
    Social Research, 1(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/
    view/1089
    Parasie, S. (2015). Data-driven Revelation? Epistemological tensions in investigative journalism in the age
    of “big data.” Digital Journalism, 3(3), 364–380. doi: 10.1080/21670811.2014.976408
    Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-
    reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. doi:
    10.1177/1461444812463345
    QSR International. (2015). Nvivo. Retrieved from http://www.qsrinternational.com/
    Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: SAGE.
    Segel, E., & Heer, J. (2010). Narrative visualization: Telling stories with data. IEEE Trans. Visualization and
    Computer Graphics, 16(6), 1139–1148. doi: 10.1109/TVCG.2010.179
    Stavelin, E. (2013). Computational journalism: When journalism meets programming (Dissertation).
    University of Bergen, Bergen.
    Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed
    management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222.
    doi: 10.1111/1467-8551.00375
    

    View Slide