Fear Factor: Use, misuse, and abuse of the impact factor

Slide 1

Slide 1 text

Fear Factor: Use, misuse, and abuse of the impact factor MATTEO CAVALLERI Editor-in-Chief International Journal of Quantum Chemistry Publisher John Wiley & sons, inc. @physicsteo

Slide 2

Slide 2 text

Show of hands • Do you know what the impact factor is, and how it is calculated? • Do you think higher impact factor = better quality journa?l • Do you heavily rely on the impact factor to advertise your journals to editors, authors, reviewers? • Do you know what the h-index is?

Slide 3

Slide 3 text

Impact Factor Definitions (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972) JCR DEFINITION 1972 DEFINITION “Dividing the number of times a journal has been cited by the number of articles it has published during some specific period of time. The journal impact factor reflect an average citation rate per article” -“All citations to the journal to items published in the previous two years divided by the total number of scholarly items published in the journal in the previous two years” 2018 IF TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in 2016 & 2017 # of SCHOLARLY ITEMS (ARTICLES, REVIEWS, PROCEEDINGS) PUBLISHED in 2016 & 2017

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Impact Factor Meaning A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL WHAT IT IS 2018 IF TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in 2016 & 2017 # of SCHOLARLY ITEMS (ARTICLES, REVIEWS, PROCEEDINGS) PUBLISHED in 2016 & 2017 • Intuitive • Universally accepted / imitated • Not uncontroversial • Measure of journal prestige • Field specific • Skewed towards higher cited articles • Open to manipulation J. Am. Soc. Inf. Sci., 60: 27-34. doi:10.1002/asi.20936 arXiv:1906.02660

Slide 6

Slide 6 text

Impact Factor Meaning A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL WHAT IT IS • Intuitive • Universally accepted / imitated • Not uncontroversial • Field specific • Measure of journal prestige • Skewed towards higher cited articles • Open to manipulation WHAT IT ISN’T • Transparent (underlying databased not publicly available) • Measure of quality of individual article / researcher • Measure of prestige of an institution • Predictor of likely citations @DuttonChemistry, Jason Dutton, Assoc. Prof. Latrobe University

Slide 7

Slide 7 text

Impact Factor Misuse ESPECIALLY AS SOLE METRICS • Use in hiring/promotion/tenure decisions • Use in grant evaluations • Employed in University Rankings

Slide 8

Slide 8 text

Impact Factor Abuse WITH GREAT POWER COMES GREAT RESPONSIBILITY • Impact Factor as tool for evaluating researchers’ productivity has altered the research landscape • Association with fraud BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD PNAS, doi:10.1073/pnas.1212247109

Slide 9

Slide 9 text

Slide 10

Slide 10 text

H-Index JE HIRSCH, PNAS, 102(46):16569 -16572 (2005) Metric of the scientific output of an individual, “…incorporating both quantity and visibility of publication” • H-index of 10 means having published 10 articles having at least 10 citations each • Field-dependent • Never decreases • No time-window • Insensitive to citation extremes • Insensitive to author’s contribution to article • Does not correct for self-citations

Slide 11

Slide 11 text

H-Index WITH GREAT POWER COMES GREAT RESPONSIBILITY • Simplistic, but measures the broad impact of one’s work • Should not be used as (sole) criterion to inform decision making in science PHYSICIST B PHYSICIST A H-index @ PhD: 5 10

Slide 12

Slide 12 text

TL;DR DON’T WORRY SO MUCH ABOUT THE IMPACT FACTOR • The absolute value of the impact factor is meaningless • It’s not about people, or articles, it’s about journals • No single metrics should be applied alone to evaluation of journals, research • We have a role to play in avoiding misuses and abuses • Matteo measurably better physicist than Einstein