MEDIA & DESIGN Research on data journalism: What is there to investigate? Insights from a structured literature review Julian Ausserhofer1,2,3, Robert Gutounig1, Michael Oppermann2, Sarah Matiasek1,2 & Eva Goldgruber1 1: FH Joanneum University of Applied Sciences, Graz
 2: University of Vienna
 3: Humboldt Institute for Internet and Society, Berlin NODA16 Academic Pre-Conference
 #NODA16
 21.04.2016, University of Helsinki

MEDIA & DESIGN *Tool evaluation partners * * * Supported by:

MEDIA & DESIGN Research Interest

MEDIA & DESIGN @julauss Research literature on data journalism 2013: "internalist tendencies at [... the] early stage of academic research" (Anderson, 2013, p. 1007) ↓
 
 2015: "an explosion in data journalism- oriented scholarship" (Fink & Anderson, 2015, p. 476)*
 
 "rapidly growing body" of scientific studies 
 (Lewis, 2015, p. 322)*
 
 
 
 *cited via Loosen, Reimer & Schmidt (2015, p. 2) 4 #NODA16

MEDIA & DESIGN How is the research literature developing? What are the research gaps? Research questions @julauss 5 #NODA16

MEDIA & DESIGN Method

MEDIA & DESIGN Structured literature review @julauss 7 #NODA16

MEDIA & DESIGN "to develop insights, critical reflections, future research paths and research questions" 
 (Massaro, Dumay & Guthrie, forthcoming) It adopts "a replicable, scientific and transparent process [...] that aims to minimize bias [...]"
 (Tranfield, Denyer & Smart, 2003) Why a structured literature review? @julauss 8 #NODA16

MEDIA & DESIGN Undertaking a systematic literature review 9 Adapted from Massaro et al. (forthcoming) Writing a literature review protocol Developing insights and critique through analyzing the dataset Developing future research paths and questions Determining the type of studies and carrying out a comprehensiv e literature search Coding data Defining the questions that the literature review should answer @julauss #NODA16

MEDIA & DESIGN 10 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search ● Empirical research on DDJ ● Social science focus, but open to other disciplines ● Published after 1995 Included Journal articles Book sections Conference papers Reports (from industry and research projects) PhD theses Not included Bachelor's and Master's theses Press reports Blog posts @julauss #NODA16

MEDIA & DESIGN 11 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search ● Preliminary search with “data-driven journalism” ● Extracting related terms from the keyword section of research papers @julauss #NODA16

MEDIA & DESIGN 12 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search Search terms algorithmic journalism computational journalism computer-assisted reporting data journalism data-driven journalism data-driven reporting database journalism datajournalism datenjournalismus quantitative journalism No search terms accountability journalism crowdsourced journalism dataviz datavis ddj drone journalism investigative journalism online journalism open journalism @julauss #NODA16

MEDIA & DESIGN 13 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search Scientific Databases ACM Digital Sowiport EBSCO Springer IEEE SpringerLink JSTOR Taylor & Francis Online ProQuest Web of Science Science Direct Wiley Scopus Google Scholar Sociological Abstracts @julauss #NODA16

MEDIA & DESIGN 14 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensiv e literature search 772 search results ↓ Assessment of title, abstract & keywords - by two independently working researchers
 (Thomas et al., 2004) ↓ 33 research publications @julauss #NODA16

MEDIA & DESIGN 15 @julauss #NODA16 Records identified from scientific databases (n= 772) Further publications from expert poll of data journalism researchers (n= 4) Excluded after screening (n= 739) Preliminary corpus: publications included after screening of records (n= 33) References from preliminary corpus (n = 1151) Final corpus: Publications included in the systematic review (n=40) Excluded after screening (n= 1148) Further publications included after screening of references (n = 3) Adapted from Fecher, Friesike & Hebing (2015)

MEDIA & DESIGN 16 @julauss #NODA16 software-assisted qualitative content analysis (Kaefer, Roper, & Sinha, 2015; Mayring, 2000; Schreier, 2012; QSR International, 2015) computational analysis of structural aspects
 (Kreibich, 2016; Lopez, 2009)

MEDIA & DESIGN Results

MEDIA & DESIGN Development of the literature over time n=40 @julauss 18 #NODA16

MEDIA & DESIGN 
 Publications by type and citations n=40 bubble size = number of citations in Google Scholar @julauss 19 #NODA16

MEDIA & DESIGN Affiliations & collaborations @julauss 20 #NODA16

MEDIA & DESIGN 1787-2015
 n=1644 @julauss 
 References per year 21 #NODA16

MEDIA & DESIGN Most-cited references Publication Nr. of Citations Meyer, P. (2002/1973). Precision journalism: A reporter’s introduction to social science methods (4th ed.). Oxford: Rowman & Littlefield. 15 Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. 15 Gray, J., Bounegru, L., & Chambers, L. (Eds.). (2012). The data journalism handbook: How journalists can use data to improve the news. Sebastopol: O’Reilly. 13 @julauss 22 #NODA16

MEDIA & DESIGN @julauss Theoretical frames ● Science and technology studies ● Actor network theory
 (Ausserhofer, 2015; De Maeyer, Libert, Domingo, Heinderyckx, & Le Cam, 2015; Parasie & Dagiral, 2013; Parasie, 2015)
 23 #NODA16

MEDIA & DESIGN @julauss Research designs & data collection methods Method Nr. of studies In-depth interviews 25 Content analysis 21 Survey 5 Short-term observation 3 Newsroom ethnography 1 Note. Content analysis includes analysis of news, databases, blogs, job ads, visualizations, briefings, manuals, and more. Short-term observation encompasses visits to the newsroom and participation in meetings. A newsroom ethnography is defined as a detailed study of a newsroom over the course of several days. 24 #NODA16

MEDIA & DESIGN Geographical scope Country Number of studies United States 16 United Kingdom 14 Germany 5 International 3 n/a 3 Sweden 2 Switzerland 2 Norway 2 Netherlands 2 … … @julauss 25 #NODA16

MEDIA & DESIGN @julauss 26 #NODA16 Research gaps in data journalism research ● Comparision of practices between countries 
 (Appelgren & Nygren, 2014; Parasie & Dagiral, 2013) ● Long-term studies (Davenport, 2000; Knight, 2015) ● Newsroom ethnographies (Parasie & Dagiral, 2013) ● Software studies (Garrison, 1999; Lewis, 2013; Stavelin, 2013) ● Reader experience studies (Segel & Heer, 2010)

MEDIA & DESIGN Conclusion

MEDIA & DESIGN @julauss 28 #NODA16 ● data journalism and its investigation has been developing rapidly ● quality improvements in the research ● issues with the literature: few publications refer to theory or methodology, just report what has been investigated Conclusion

MEDIA & DESIGN @julauss 29 #NODA16 ● practices in small news organizations, freelancers, local and mobile data journalism etc. ● gender ● digital methods: investigating the field through its platforms ● theory ● … Research opportunities

MEDIA & DESIGN Explore the literature online at: http://literature.validproject.at Julian Ausserhofer1,2,3,
 julian.ausserhofer@fh-joanneum.at @julauss Robert Gutounig1,
 @sextus_empirico Michael Oppermann2,
 @oppermann_m Sarah Matiasek1,2 &
 @sarahmatiasek Eva Goldgruber1
 @evagoldgruber 1: FH Joanneum University of Applied Sciences, Graz
 2: University of Vienna
 3: Humboldt Institute for Internet and Society, Berlin NODA16 Academic Pre-Conference
 #NODA16
 21.04.2016, University of Helsinki

MEDIA & DESIGN References 31

MEDIA & DESIGN @julauss 32 #NODA16 Anderson, C. W. (2013). Towards a sociology of computational and algorithmic journalism. New Media & Society, 15(7), 1005–1021. doi: 10.1177/1461444812465137 Appelgren, E., & Nygren, G. (2014). Data journalism in Sweden: Introducing new methods and genres of journalism into “old” organizations. Digital Journalism, 2(3), 394–405. doi: 10.1080/21670811.2014.884344 Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (pp. 87–111). Berlin: digitalcommunicationresearch.de. doi: 10.17174/dcr.v2.5 Davenport, L., Fico, F., & Detwiler, M. (2000). Computer–assisted reporting in Michigan daily newspapers: More than a decade of adoption. Presented at the Association for Education in Journalism and Mass Communication (AEJMC) National Convention, Phoenix, Arizona. De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for data journalism: A qualitative assessment of the anecdotal take-up of data journalism in French-speaking Belgium. Digital Journalism, 3(3), 432–446. doi: 10.1080/21670811.2014.976415 Fecher, B., Friesike, S., & Hebing, M. (2015). What drives academic data sharing? PLoS ONE, 10(2), e0118053. doi: 10.1371/journal.pone.0118053 Fink, K., & Anderson, C. W. (2015). Data journalism in the United States: Beyond the “usual suspects.” Journalism Studies, 16(4), 467–481. doi: 10.1080/1461670X.2014.939852 Garrison, B. (1999). Newspaper size as a factor in use of computer-assisted reporting. Newspaper Research Journal, 20(3). Retrieved from http://com.miami.edu/car/baltimore1.htm Kaefer, F., Roper, J., & Sinha, P. (2015). A software-assisted qualitative content analysis of news articles: example and reflections. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 16(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/2123 Knight, M. (2015). Data journalism in the UK: A preliminary analysis of form and content. Journal of Media Practice, 16(1), 55–72. doi: 10.1080/14682753.2015.1015801 Kreibich, C. (2016). scholar.py. Retrieved from https://github.com/ckreibich/scholar.py

MEDIA & DESIGN @julauss 33 #NODA16 Lewis, S. C. (2015). Journalism in an era of big data. Digital Journalism, 3(3), 321–330. doi: 10.1080/21670811.2014.976399 Loosen, W., Reimer, J., & Schmidt, F. (2015). When data become news: A content analysis of data journalism pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff. Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In M. Agosti, J. Borbinha, S. Kapidakis, C. Papatheodorou, & G. Tsakonas (Eds.), Research and Advanced Technology for Digital Libraries (pp. 473–474). Berlin: Springer. Retrieved from doi: 10.1007/978-3-642-04346-8_62 Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the shoulders of giants: undertaking a structured literature review in accounting. Accounting, Auditing & Accountability Journal. Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/ view/1089 Parasie, S. (2015). Data-driven Revelation? Epistemological tensions in investigative journalism in the age of “big data.” Digital Journalism, 3(3), 364–380. doi: 10.1080/21670811.2014.976408 Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted- reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. doi: 10.1177/1461444812463345 QSR International. (2015). Nvivo. Retrieved from http://www.qsrinternational.com/ Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: SAGE. Segel, E., & Heer, J. (2010). Narrative visualization: Telling stories with data. IEEE Trans. Visualization and Computer Graphics, 16(6), 1139–1148. doi: 10.1109/TVCG.2010.179 Stavelin, E. (2013). Computational journalism: When journalism meets programming (Dissertation). University of Bergen, Bergen. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. doi: 10.1111/1467-8551.00375