MEDIA & DESIGN News Production Workflows in Data- driven, Algorithmic Journalism A Systematic Literature Review Julian Ausserhofer1,2 , @julauss Robert Gutounig1 & @sextus_empirico Michael Oppermann2 @oppermann_m 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays

MEDIA & DESIGN *Tool evaluation partners * * * Supported by:

MEDIA & DESIGN Research Interest

MEDIA & DESIGN #DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 4 "experimental use of algorithms, data and social science methods" in journalism (Gynnild, 2014, p. 715)

MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 5 (Cox, 2000) http://www.computerhistory.org/revolution/early-computer-companies/5/102/445 (Meyer, 1973) 1952 #DuMD2015 & #DUMediaDays Computational methods in journalism

MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 6 (Garrison, 1996) #DuMD2015 & #DUMediaDays

MEDIA & DESIGN The datafication of journalism @julauss, @sextus_empirico & @oppermann_m 7 "datafication" of society (Mayer-Schönberger & Cukier, 2013) "increased focus on measurement, outcomes assessment" (Anderson, 2015, p. 363) "journalism's quantitative turn" (Coddington, 2015) … has also affected the "infrastructures of journalism" (Russ-Mohl, 2006, p. 199) #DuMD2015 & #DUMediaDays

MEDIA & DESIGN Research literature on data journalism @julauss, @sextus_empirico & @oppermann_m 8 "internalist tendencies at [... the] early stage of academic research" (Anderson, 2013, p. 1007) ↓ "an explosion in data journalism-oriented scholarship" (Fink & Anderson, 2015, p. 476)* "rapidly growing body" of scientific studies (Lewis, 2015, p. 322)* *cited via (Loosen, Reimer & Schmidt, 2015, p. 2) #DuMD2015 & #DUMediaDays

MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 9 In what way have journalistic workflows and routines changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skill requirements for journalists? #DuMD2015 & #DUMediaDays Research questions

MEDIA & DESIGN Method

MEDIA & DESIGN #DuMD2015 & #DUMediaDays @julauss, @sextus_empirico & @oppermann_m 11 Systematic literature review

MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 12 "to develop insights, critical reflections, future research paths and research questions" (Massaro, Dumay & Guthrie, forthcoming) It adopts "a replicable, scientific and transparent process [...] that aims to minimize bias [...]" (Tranfield, Denyer & Smart, 2003) #DuMD2015 & #DUMediaDays Why a systematic literature review?

MEDIA & DESIGN Undertaking a systematic literature review 13 Adapted from Massaro et al. (forthcoming) Own visualization @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Writing a literature review protocol Developing insights and critique through analyzing the dataset Developing future research paths and questions Determining the type of studies and carrying out a comprehensive literature search Coding data Defining the questions that the literature review should answer

MEDIA & DESIGN 14 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Determining the type of studies and carrying out a comprehensive literature search Defining the questions that the literature review should answer In what way have journalistic workflows and routines changed due to the broad introduction of data in the newsroom? How has this impacted journalistic norms and ethics, and the skill requirements for journalists?

MEDIA & DESIGN 15 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search ● Empirical research on DDJ ● Social science focus, but open to other disciplines ● Published after 1995 Included Journal articles Book sections Conference papers Reports (from industry and research projects) PhD theses Not included Bachelor's and Master's theses Press reports Blog posts

MEDIA & DESIGN 16 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search ● Preliminary search with “data-driven journalism” ● Extracting related terms from the keyword section of research papers

MEDIA & DESIGN 17 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search Search terms algorithmic journalism computational journalism computer-assisted reporting data journalism data-driven journalism data-driven reporting database journalism datajournalism datenjournalismus quantitative journalism No search terms accountability journalism crowdsourced journalism dataviz datavis ddj drone journalism investigative journalism online journalism open journalism

MEDIA & DESIGN 18 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search Scientific Databases ACM Digital Sowiport EBSCO Springer IEEE SpringerLink JSTOR Taylor & Francis Online ProQuest Web of Science Science Direct Wiley Scopus Google Scholar Sociological Abstracts

MEDIA & DESIGN 19 Developing insights and critique through analyzing the dataset Developing future research paths and questions Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Defining the questions that the literature review should answer Determining the type of studies and carrying out a comprehensive literature search 772 search results ↓ Assessment of title, abstract & keywords - by two independently working researchers (Thomas et al., 2004) ↓ 33 research publications

MEDIA & DESIGN 20 Determining the type of studies and carrying out a comprehensive literature search Defining the questions that the literature review should answer Developing future research paths and questions Developing insights and critique through analyzing the dataset Coding data @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays qualitative coding computational analysis

MEDIA & DESIGN Results

MEDIA & DESIGN Literature Corpus

MEDIA & DESIGN 23 Research literature on workflows in DDJ Development of literature over time n=33 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 24 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication type Parasie & Dagiral, 2013 Lewis & Usher, 2013 Diakopolus, 2015 Karlsen & Stavelin, 2014 n=33 node size = number of citations in Google Scholar

MEDIA & DESIGN 25 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication year of referenced papers 1787-2015 n=1151 Constitution of the United States

MEDIA & DESIGN 26 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Publication year of referenced papers 1999-2015 n=1004 2013: 176 cited papers

MEDIA & DESIGN 27 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Research literature on workflows in DDJ Most-cited references

MEDIA & DESIGN 28 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays edges = 165 corpus publications, n=33 references, n = 70 size = nr. of citations from the corpus Research literature on workflows in DDJ Corpus and most-cited references The Journalist as Programmer Precision Journalism Accountability through Algorithm

MEDIA & DESIGN Nr. of studies Survey 3 In-depth interviews 22 (Particpant) observation 4 Method(s) of data collection 29 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN Geographical scope 30 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays Nr. of studies Nr. of studies United States 9 Multinational (>4) 2 United Kingdom 9 Norway 1 Germany 4 Netherlands 1 Sweden 2 Argentina 1 Switzerland 2 Austria 1 Finland 2 Belgium 1

MEDIA & DESIGN Qualitative analysis - News Production Workflow

MEDIA & DESIGN News production workflow Collecting Collecting digitalized information from ● publicly available data sets ● requested data ● own research or data collected through crowdsourcing ● automated research (scraping) ● leaked data (17; Loosen, 2015) 32 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN Type of data source ● government or public bureau ● private corporation ● other organizations like NGOs, research institutes (28, Loosen, 2015) 33 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collecting

MEDIA & DESIGN Access to information in Norway and USA generally considered good (due to freedom of information legislation) Barriers to easy data access ● fees ● lacking competencies for exporting data at public offices ● the form data is often delivered (e.g. as PDF) ● lack of legal legal resources to get access to data that is denied to journalists in the first place (6; 14) 34 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collecting

MEDIA & DESIGN Selecting data Because data journalists don’t have much time in the newsroom, they prefer easy-accessible data, e.g. datasets from public bodies. They are very cautious when it comes to work with data from companies or private sources. (14) 35 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collecting

MEDIA & DESIGN Methods ● Journalists use statistical programs (e.g. MySQL, Excel, Access, SPSS, Google Docs, Maps, Google Refine, Google Fusion Tables etc.) ● Statistical methods and packages including cluster, network & regression analysis --> Journalists are quite cautious with it, since they consider this outside of their field of expertise. (6, 14) 36 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Analyzing

MEDIA & DESIGN Results of DDJ research depend in increasing manner on (graphical) display of (automated) data analysis software Tools and programming languages ● Developers use Python, Ruby on Rails, JavaScript, HTML ● Used third-party software includes MySQL, Access, Excel, Caspio, Tableau, Arcview, TimelineSetter etc. (14, 17, 28) 37 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Building and visualizing

MEDIA & DESIGN DDJ is connected to a special way of representation which consists of ● displaying key messages as graphics and/or ● building interactive web applications Storytelling is done through (1) interactive web applications with continuous update of data (2) interactive application based on a stable data set (3) continuously updated info graphic or (4) static infographics (17) 38 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Building and visualizing

MEDIA & DESIGN ● Working alone at small and medium-sized newspapers, and part of teams at larger news organizations ● New connections between journalists and programmers ● Members of teams usually have complementary backgrounds (6, 9, 14, 18) 39 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collaboration

MEDIA & DESIGN ● Rise of "annotative journalism" ● Regarding the collaboration between newsroom and ICT- department in media corporations data journalists perceive the relation as troublesome. Rather data journalists try to bypass ICT department when installing extra software or setting up databases or servers (6, 28) 40 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Collaboration

MEDIA & DESIGN ● The goal of DDJ activities is by definition the [very accurate] journalistic publication ● Publication of data sources and raw data following open data principles is mentioned too as an integral part of DDJ. (6, 17, Coddington, 2014) 41 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays News production workflow Publishing the product

MEDIA & DESIGN Backgrounds of data journalists

MEDIA & DESIGN Backgrounds of data journalists ● Data journalists are appx. 5 years younger than the average age of German journalists ● All of them have started a university program, 80% have also finished it ● There is no generalizable “data journalism” career path. Background is mostly either social science or informatics. ● Backgrounds include programming, design, typography, info graphics, usability, databases, Web and journalism ● USA: dominance of National Institute for Computer-Assisted Reporting (NICAR) (14, 17, 18; 6) 43 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN Enabling and Hindering Factors

MEDIA & DESIGN Enabling and hindering factors Critical factors ● Skills of the involved journalists ● Support networks (internal and external) important for the development of skills ● Availability of data ● Tools ● Legal resources ● Available resources (6, 14, 17) 45 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN Conclusion

MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 47 #DuMD2015 & #DUMediaDays ● New kind of people arriving in the newsroom ● New skills needed in the newsroom ● Increase of collaborative newswork ● Shift to data-driven approaches (and epistemologies debates connected to that)

MEDIA & DESIGN @julauss, @sextus_empirico & @oppermann_m 48 Scope: ● Practices outside of Europe and North America ● The construction of "facts" ● Gender aspects Methods of data collection: ● Survey ● (Participant) observation ● Content analysis ● Digital methods #DuMD2015 & #DUMediaDays Research gaps

MEDIA & DESIGN 49 Explore the (preliminary) corpus online – it's a protoype: http://tinyurl.com/valid-survey http://www.validproject.at Julian Ausserhofer1,2 , @julauss julian.ausserhofer@fh-joanneum.at Robert Gutounig1 & @sextus_empirico robert.gutounig@fh-joanneum.at Michael Oppermann2 @oppermann_m michael.oppermann@univie.ac.at 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays

MEDIA & DESIGN Tools & Literature 50

MEDIA & DESIGN 51 NVIVO [Computer Software]. (2015). Retrieved from http://www.qsrinternational.com/ Bostock, M., Ogievetsky, V., & Heer, J. (2011). D³ data-driven documents.Visualization and Computer Graphics, IEEE Transactions on, 17(12), 2301-2309. Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In Research and Advanced Technology for Digital Libraries (pp. 473-474). Berlin: Springer. Zotero [Computer Software]. (2015). Retrieved from http://zotero.org/ Tools & software libraries @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 52 Anderson, C. W. (2013). Towards a Sociology of Computational and Algorithmic Journalism. New Media & Society, 15(7), 1005–1021. http://doi.org/10.1177/1461444812465137 Anderson, C. W. (2015). Between the Unique and the Pattern: Historical Tensions in Our Understanding of Quantitative Journalism. Digital Journalism, 3(3), 349–363. http://doi.org/10.1080/21670811.2014.976407 Cox, M. (2000). The Development of Computer-assisted Reporting. Presented at the Newspaper Division, Association for Education in Journalism and Mass Communication, Southeast Colloquium, Chapel Hill, North Carolina. Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the "Usual Suspects". Journalism Studies, 16(4), 467–481. http://doi.org/10.1080/1461670X.2014.939852 Garrison, B. (1996, September). Computer-Assisted Reporting: A National Use Update. Presented at the National Society of Professional Journalists Meeting, Washington, D.C. Lewis, S. C. (2015). Journalism In An Era Of Big Data. Digital Journalism, 3(3), 321–330. http://doi.org/10. 1080/21670811.2014.976399 Loosen, W., Reimer, J., & Schmidt, F. (2015). When Data Become News: A Content Analysis of Data Journalism Pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff. Mayer-Schönberger, V., & Cukier, K. (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think. Boston: Houghton Mifflin Harcourt. Primary Sources @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 53 Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the Shoulders of Giants: Undertaking a Structured Literature Review in Accounting. Accounting, Auditing & Accountability Journal. Meyer, P. (1973). Precision Journalism: A Reporter’s Introduction to Social Science Methods (2nd ed.). Oxford: Rowman & Littlefield. Russ-Mohl, S. (2006). The Economics of Journalism and the Challenge to Improve Journalism Quality: A Research Manifesto. Studies in Communication Sciences, 6(2), 189–208. Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., … Kavanagh, J. (2004). Integrating Qualitative Research with Trials in Systematic Reviews. BMJ : British Medical Journal, 328(7446), 1010–1012. http://doi.org/10.1136/bmj.328.7446.1010 Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British Journal of Management, 14(3), 207–222. http://doi.org/10.1111/1467-8551.00375 Primary Sources @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 54 (34) Aitamurto, T., Sirkkunen, E., & Lehtonen, P. (2011). Trends in Data Journalism. Retrieved from http://virtual.vtt.fi/virtual/nextmedia/Deliverables-2011/D3. 2.1.2.B_Hyperlocal_Trends_In%20Data_Journalism.pdf (2) Ananny, M., & Crawford, K. (2015). A Liminal Press: Situating news app designers within a field of networked news production. Digital Journalism, 3(2), 192–208. http://doi.org/10.1080/21670811.2014.922322 (12) Appelgren, E., & Nygren, G. (2012). Data Journalism in Sweden – Opportunities and Challenges: A Case Study of Brottspejl at Sveriges Television (SVT). In C. Mills, M. Pidd, & E. Ward (Eds.), Proceedings of the Digital Humanities Congress 2012. Sheffield: HRI Online Publications. Retrieved from http://www. hrionline.ac.uk/openbook/chapter/dhc2012-appelgren (11) Appelgren, E., & Nygren, G. (2014). Data Journalism in Sweden: Introducing New Methods and Genres of Journalism into “Old” Organizations. Digital Journalism, 2(3), 394–405. http://doi.org/10.1080/21670811.2014.884344 (18) Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale Methoden in der Kommunikationswissenschaft (Vol. 2, pp. 87–111). Berlin: digitalcommunicationresearch.de. Retrieved from http://nbn-resolving.de/http://dx.doi.org/10.17174/dcr.v2.5 (3) Baack, S. (2013). A New Style of News Reporting: Wikileaks and Data-driven Journalism. In B. Rambatan & J. Johanssen (Eds.), Cyborg Subjects: Discourses on Digital Culture (pp. 113–122). CreateSpace Independent Publishing. Retrieved from http://www.ssoar.info/ssoar/handle/document/40025 (26) Bakker, P. (2014). Mr. Gates Returns: Curation, Community Management and Other New Roles for Journalists. Journalism Studies, 15(5), 596–606. http: //doi.org/10.1080/1461670X.2014.901783 (31) Daniel, A., & Flew, T. (2010). The Guardian Reportage of the UK MP Expenses Scandal: A Case Study of Computational Journalism. In Record of the Communications Policy and Research Forum 2010 (pp. 186–194). Sydney: Network Insight. Retrieved from http://eprints.qut.edu.au/39358/ (35) De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for Data Journalism: A Qualitative Assessment of the Anecdotal Take- up of Data Journalism in French-Speaking Belgium. Digital Journalism, 3(3), 432–446. http://doi.org/10.1080/21670811.2014.976415 (1) Diakopoulos, N. (2015). Algorithmic Accountability: Journalistic Investigation of Computational Power Structures. Digital Journalism, 3(3), 398–415. http: //doi.org/10.1080/21670811.2014.976411 (21) Dick, M. (2014). Interactive Infographics and News Values. Digital Journalism, 2(4), 490–506. http://doi.org/10.1080/21670811.2013.841368 (19) Felle, T. (2015). Digital Watchdogs? Data Reporting and the News Media’s Traditional “Fourth Estate” Function. Journalism, 1464884915593246. http: //doi.org/10.1177/1464884915593246 Corpus 1/3 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 55 (14) Fink, K., & Anderson, C. W. (2015). Data Journalism in the United States: Beyond the “Usual Suspects.” Journalism Studies, 16(4), 467–481. http://doi. org/10.1080/1461670X.2014.939852 (27) Garrison, B. (1999). Newspaper Size as a Factor in Use of Computer-Assisted Reporting. Newspaper Research Journal, 20(3). Retrieved from http://com. miami.edu/car/baltimore1.htm (22) Gynnild, A. (2014). Journalism Innovation Leads to Innovation Journalism: The Impact of Computational Exploration on Changing Mindsets. Journalism, 15 (6), 713–730. http://doi.org/10.1177/1464884913486393 (7) Hannaford, L. (2015). Computational Journalism in the UK Newsroom: Hybrids or Specialists? Journalism Education, 4(1), 6–21. http://doi.org/http://hdl. handle.net/2173/575953 (30) Howard, A. B. (2014). The Art and Science of Data-Driven Journalism (pp. 1–144). New York: Tow Center for Digital Journalism. Retrieved from http: //towcenter.org/wp-content/uploads/2014/05/Tow-Center-Data-Driven-Journalism.pdf (8) Hullman, J., Diakopoulos, N., Momeni, E., & Adar, E. (2015). Content, Context, and Critique: Commenting on a Data Visualization Blog. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1170–1175). New York: ACM. http://doi.org/10.1145/2675133. 2675207 (6) Karlsen, J., & Stavelin, E. (2014). Computational Journalism in Norwegian Newsrooms. Journalism Practice, 8(1), 34–48. http://doi.org/10.1080/17512786. 2013.813190 (13) Knight, M. (2015). Data Journalism in the UK: A Preliminary Analysis of Form and Content. Journal of Media Practice, 16(1), 55–72. http://doi.org/10. 1080/14682753.2015.1015801 (33) Koch, C., & Stollorz, V. (2014). Transparenz oder Konfusion? Methodik und empirische Basis des Datenjournalismus in der Medizin. In V. Lilienthal, D. Reineck, & T. Schnedler (Eds.), Qualität im Gesundheitsjournalismus (pp. 367–387). Wiesbaden: Springer Fachmedien. Retrieved from http://link.springer. com/chapter/10.1007/978-3-658-02427-7_21 (28) Lewis, S. C., & Usher, N. (2013). Open Source and Journalism: Toward New Frameworks for Imagining News Innovation. Media, Culture and Society, 35(5), 602–619. http://doi.org/10.1177/0163443713485494 (5) Lewis, S. C., & Usher, N. (2014). Code, Collaboration, and the Future of Journalism: A Case Study of the Hacks/Hackers Global Network. Digital Journalism, 2 (3), 383–393. http://doi.org/10.1080/21670811.2014.895504 (29) Lugo-Ocando, J., & Brandão, R. F. (2015). Stabbing News: Articulating Crime Statistics in the Newsroom. Journalism Practice, 9, 1–15. http://doi.org/10. 1080/17512786.2015.1058179 Corpus 2/3 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 56 (10) Parasie, S. (2015). Data-driven Revelation? Epistemological Tensions in Investigative Journalism in the Age of “Big Data.” Digital Journalism, 3(3), 364–380. http://doi.org/10.1080/21670811.2014.976408 (9) Parasie, S., & Dagiral, E. (2013). Data-driven Journalism and the Public Good: “Computer-assisted-reporters” and “Programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. http://doi.org/http://dx.doi.org/10.1177/1461444812463345 (32) Radchenko, I., & Sakoyan, A. (2014). The View on Open Data and Data Journalism: Cases, Educational Resources and Current Trends. In D. I. Ignatov, M. Y. Khachay, A. Panchenko, N. Konstantinova, & R. E. Yavorsky (Eds.), Analysis of Images, Social Networks and Texts (pp. 47–54). Cham: Springer International Publishing. Retrieved from http://link.springer.com/chapter/10.1007/978-3-319-12580-0_4 (25) Uskali, T., & Kuutti, H. (2015). Models and Streams of Data Journalism. The Journal of Media Innovations, 2(1), 77. http://doi.org/10.5617/jmi.v2i1.882 (24) Veglis, A. (2005). Modeling Computer Supported Collaborative Work in a Newspaper Editorial Process. In Proceedings of the 9th WSEAS International Conference on Computers (pp. 1–6). Stevens Point: World Scientific and Engineering Academy and Society (WSEAS). Retrieved from http://dl.acm. org/citation.cfm?id=1369599.1369726 (16) Weber, W., & Rall, H. (2012). Data Visualization in Online Journalism and Its Implications for the Production Process. In 2012 16th International Conference on Information Visualisation (IV) (pp. 349–356). Montpellier: IEEE. http://doi.org/10.1109/IV.2012.65 (36) Weber, W., & Rall, H. (2013). “We Are Journalists”: Production Practices, Attitudes and a Case Study of the New York Times Newsroom. In W. Weber, M. Burmester, & R. Tille (Eds.), Interaktive Infografiken (pp. 161–172). Berlin: Springer. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642- 15453-9_9 (17) Weinacht, S., & Spiller, R. (2014). Datenjournalismus in Deutschland: Eine explorative Untersuchung zu Rollenbildern von Datenjournalisten. Publizistik, 59 (4), 411–433. http://doi.org/10.1007/s11616-014-0213-5 (20) Young, M. L., & Hermida, A. (2015). From Mr. and Mrs. Outlier to Central Tendencies: Computational Journalism and Crime Reporting at the Los Angeles Times. Digital Journalism, 3(3), 381–397. http://doi.org/10.1080/21670811.2014.976409 Corpus 3/3 @julauss, @sextus_empirico & @oppermann_m #DuMD2015 & #DUMediaDays

MEDIA & DESIGN 57 Explore the (preliminary) corpus online – it's a protoype: http://tinyurl.com/valid-survey http://validproject.at Julian Ausserhofer1,2 , @julauss julian.ausserhofer@fh-joanneum.at Robert Gutounig1 & @sextus_empirico robert.gutounig@fh-joanneum.at Michael Oppermann2 @oppermann_m michael.oppermann@univie.ac.at 1: FH Joanneum University of Applied Sciences Graz 2: University of Vienna Dubrovnik Media Days, 31.10.15 #DuMD2015 & #DUMediaDays