Tweet
Tweet

Slide 1

Slide 1 text

http://www.flickr.com/photos/wallyg/156079118/ Martin Fenner Technical Lead ALM Project

Slide 2

Slide 2 text

Journal-based metrics correlate poorly with numbers for individual article 2 0 20 40 60 80 100 0.00 0.01 0.02 0.03 0.04 0.05 Many PLOS ONE Papers are Highly Cited Number of Scopus citations for all 4405 PLoS ONE papers published in 2009. Data collected November 8, 2012. Median is 9 citations, 10% of papers have at least 29 citations. Scopus Citations Probability

Slide 3

Slide 3 text

PLOS ONE is not a second-tier journal Combined Scopus citation counts for all PLOS Biology and PLOS Medicine articles published in 2009, as well as top 200 PLOS ONE articles in 2009 3 Citation counts collected November 8, 2012 8849 3263 4341 264 199 200 PLOS Biology PLOS Medicine PLOS ONE Median 15.5 Median 11 Median 51.5

Slide 4

Slide 4 text

The megajournal has changed how we use the journal as a filter 4 Megajournal Keyword/ Subject Area Journal Keyword/ Subject Area Metrics w ork in progress

Slide 5

Slide 5 text

124,985,344 HTML pageviews 27,722,142 PDF downloads 374,958 CrossRef citations 5 0.3% 100% 22.2% Citations are only a small fraction of how a paper is reused and discussed Article-Level Metrics from November 8, 2012 for 63,771 PLOS Papers

Slide 6

Slide 6 text

6 The PLOS Article-Level Metrics project started in 2009 and tracks usage, citations and social web activity for all PLOS articles Usage Citations Social Web PLOS Journals (HTML, PDF, XML) CrossRef Mendeley PLOS Journals (HTML, PDF, XML) Scopus CiteULike PubMed Central (abstract, fulltext, PDF, figures) Web of Science Facebook PubMed Central (abstract, fulltext, PDF, figures) PubMed Central Twitter PubMed Central (abstract, fulltext, PDF, figures) Research Blogging PLOS Comments Wikipedia

Slide 7

Slide 7 text

7 PLoS HTML Views PMC HTML Views CrossRef Scopus PubMed Citations Mendeley CiteULike PLoS Comments Research Blogging Facebook Twitter Wikipedia 100% 98% 63% 62% 49% 71% 25% 12% 2% 27% 13% 6% Proportion of articles covered by source Metrics for 63,771 PLoS journal articles published until November 8, 2012.

Slide 8

Slide 8 text

8 PLoS HTML Views PMC HTML Views CrossRef Scopus PubMed Citations Mendeley CiteULike PLoS Comments Research Blogging Facebook Twitter Wikipedia 100% 91% 17% 21% 3% 51% 48% 16% 4% 76% 92% 17% Proportion of articles covered by source Metrics for 98 PLoS Biology articles published May 8 − November 8, 2012.

Slide 9

Slide 9 text

Why use article-level metrics? 9 Data-driven stories about the post- publication reception of a paper Business intelligence for a journal, university or funder Evaluation of the impact of research and researchers Metrics as a discovery tool

Slide 10

Slide 10 text

Should article-level metrics be used to evaluate the impact of research? • Can numbers reflect the impact of research, across disciplines and over time? • Do the currently available metrics really measure impact or something else? • Does the use of metrics for evaluation create undesired incentives? • How easily can metrics be changed by self-promotion and gaming? 10

Slide 11

Slide 11 text

What do we measure? 11 Impact Attention Self- Promotion

Slide 12

Slide 12 text

12 Gaming and Spam

Slide 13

Slide 13 text

Metrics as a discovery tool 13

Slide 14

Slide 14 text

Data-driven stories about the post- publication reception of a paper 14 0 20 40 60 80 0 10000 20000 30000 40000 50000 60000 PLOS Articles by Type: Review Total views are combined HTML views and PDF downloads from the PLOS website. Bubble size correlates with the number of CrossRef citations, and bubble color with the PLOS journal. Data collected August 23, 2012. Months Total Views

Slide 15

Slide 15 text

15 http://blogs.bbsrc.ac.uk/index.php/2012/09/summertime-reprise/ http://dx.doi.org/10.1371/journal.pcbi.1000204

Slide 16

Slide 16 text

16 0 20000 40000 60000 80000 58 182 251 66 241 265 17 51 90 69 18 97 138 62 44 42 33 29 68 173 15 10 29 Mendeley Ten Simple Rules for Getting Published Ten Simple Rules for Reviewers Ten Simple Rules for Getting Grants Ten Simple Rules for Graduate Students Ten Simple Rules for Doing Your Best Research, According to Hamming Ten Simple Rules for a Good Poster Presentation Ten Simple Rules for Making Good Oral Presentations Ten Simple Rules for a Successful Collaboration Ten Simple Rules for Organizing a Scientific Meeting Ten Simple Rules for Aspiring Scientists in a Low−Income Country Ten Simple Rules for Chairing a Scientific Session Ten Simple Rules for Choosing between Industry and Academia Ten Simple Rules To Combine Teaching and Research Ten Simple Rules for Editing Wikipedia Ten Simple Rules for Organizing a Virtual ConferenceÂAnywhere Ten Simple Rules for Developing a Short Bioinformatics Training Course Ten Simple Rules for Getting Involved in Your Scientific Community Ten Simple Rules for Teaching Bioinformatics at the High School Level Ten Simple Rules for Getting Help from Online Scientific Communities Ten Simple Rules for Building and Maintaining a Scientific Reputation Ten Simple Rules for Providing a Scientific Web\nResource Ten Simple Rules for Getting Ahead as a Computational Biologist in Academia Ten Simple Rules for Starting a Company Popularity of the Ten Simple Rules Collection PDF downloads (open circles), HTML pageviews (closed circles), and Mendeley readers (numbers on the right) for 23 editorials in PLoS Computational Biology. Articles are sorted by descending age. Total Article Views

Slide 17

Slide 17 text

Business intelligence for a journal, university of funder 17

Slide 18

Slide 18 text

18 0 2 4 6 8 10 12 14 0 5000 10000 15000 20000 25000 30000 PLOS Medicine Articles August 2011 to August 2012 Combined HTML views and PDF downloads (PLOS website) for articles published between 08/16/11 and 08/15/12. Bubble size correlates with Scopus citations, bubble color with article type. Data collected 08/24/12. Months Total Views http://bit.ly/alm36

Slide 19

Slide 19 text

19 0 50 100 150 200 250 0 2000 4000 6000 8000 10000 12000 14000 PLOS Articles from the Big Food collection Tweets vs. HTML views at PLOS for articles in the Big Food collection. Circle size correlates with Facebook activity. Data collected August 19, 2012. Tweets HTML Views

Slide 20

Slide 20 text

20 S S M T W T F 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 S S M T W T F 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 S S M T W T F 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 S S M T W T F 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 5 6 7 8 9 10 11 1 2 3 4 31 April S S M T W T F 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 2 3 4 5 6 7 8 1 28 29 30 May S S M T W T F 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 1 2 3 4 5 6 26 27 28 29 30 31 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 6 26 11 7 26 172 19 19 12 119 43 53 June S S M T W T F 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 4 5 6 7 8 9 10 1 2 3 30 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 16 2 2 23 3 2 3 4 1 7 5 5 8 4 2 7 17 9 105 28 20 July S S M T W T F 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 2 3 4 5 6 7 28 29 30 31 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 1 1 3 2 1 5 4 1 12 5 August S S M T W T F 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 1 2 3 4 5 25 26 27 28 29 30 31 September October November December 50 100 150 Tweets about Big Food Collection Data collected August 19, 2012.

Slide 21

Slide 21 text

21 PLOS Search API 2562 Articles CORDIS 1166 Articles Identify Open Access articles published by PLOS, funded by the European Commission Seventh Framework Programme (FP7) Match articles with grant numbers Show the impact of these publications

Slide 22

Slide 22 text

Author locations from the ENGAGE project 22 First authors (blue) and last authors (green). European Network for Genetic and Genomic Epidemiology http://www.openaire.eu/en/component/openaire/project_info/default/530?grant=201413

Slide 23

Slide 23 text

23 0 10 20 30 40 0 5000 10000 15000 PLOS Publications from the EU−funded ENGAGE Project Total views are combined HTML views and PDF downloads from the PLOS website. Bubble size correlates with Scopus citations, and bubble color with the PLOS journal. Data collected September 19, 2012. Months Total Views Scopus

Slide 24

Slide 24 text

24 0 10 20 30 40 0 5000 10000 15000 PLOS Publications from the EU−funded ENGAGE Project Total views are combined HTML views and PDF downloads from the PLOS website. Bubble size correlates with Mendeley bookmarks, and bubble color with the PLOS journal. Data collected September 19, 2012. Months Total Views Mendeley

Slide 25

Slide 25 text

25 Thank you Wordle of altmetrics manifesto by AJ Cann http://www.flickr.com/photos/ajc1/6795008004/