Fear Factor: Use, misuse, and abuse of the impact factor

Internal presentation (Lunch & Learn, 2019) on the Journal Impact Factor. What it is, and what is not.

Transcript

1. Fear Factor: Use, misuse, and
   abuse of the impact factor
   MATTEO CAVALLERI
   Editor-in-Chief
   International Journal of Quantum Chemistry
   Publisher
   John Wiley & sons, inc.
   @physicsteo
   

   View Slide

2. Show of hands
   • Do you know what the impact
   factor is, and how it is
   calculated?
   • Do you think higher impact
   factor = better quality journa?l
   • Do you heavily rely on the
   impact factor to advertise
   your journals to editors,
   authors, reviewers?
   • Do you know what the h-index
   is?
   

   View Slide

3. Impact Factor Definitions
   (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972)
   JCR DEFINITION
   1972 DEFINITION
   “Dividing the number of times a
   journal has been cited by the
   number of articles it has published
   during some specific period of
   time. The journal impact factor
   reflect an average citation rate per
   article”
   -“All citations to the journal to
   items published in the previous
   two years divided by the total
   number of scholarly items
   published in the journal in the
   previous two years”
   2018 IF
   TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in
   2016 & 2017
   # of SCHOLARLY ITEMS (ARTICLES, REVIEWS,
   PROCEEDINGS) PUBLISHED in 2016 & 2017
   

   View Slide

4. Impact Factor Definitions
   (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972)
   JCR DEFINITION
   1972 DEFINITION
   “Dividing the number of times a
   journal has been cited by the
   number of articles it has published
   during some specific period of
   time. The journal impact factor
   reflect an average citation rate per
   article”
   -“All citations to the journal to
   items published in the previous
   two years divided by the total
   number of scholarly items
   published in the journal in the
   previous two years”
   2.263
   722
   319
   

   View Slide

5. Impact Factor Meaning
   A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL
   WHAT IT IS
   2018 IF
   TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in
   2016 & 2017
   # of SCHOLARLY ITEMS (ARTICLES, REVIEWS,
   PROCEEDINGS) PUBLISHED in 2016 & 2017
   • Intuitive
   • Universally accepted / imitated
   • Not uncontroversial
   • Measure of journal prestige
   • Field specific
   • Skewed towards higher cited
   articles
   • Open to manipulation J. Am. Soc. Inf. Sci., 60: 27-34. doi:10.1002/asi.20936
   arXiv:1906.02660
   

   View Slide

6. Impact Factor Meaning
   A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL
   WHAT IT IS
   • Intuitive
   • Universally accepted / imitated
   • Not uncontroversial
   • Field specific
   • Measure of journal prestige
   • Skewed towards higher cited
   articles
   • Open to manipulation
   WHAT IT ISN’T
   • Transparent (underlying databased
   not publicly available)
   • Measure of quality of individual
   article / researcher
   • Measure of prestige of an
   institution
   • Predictor of likely citations
   @DuttonChemistry, Jason Dutton, Assoc. Prof. Latrobe University
   

   View Slide

7. Impact Factor Misuse
   ESPECIALLY AS SOLE METRICS
   • Use in hiring/promotion/tenure decisions
   • Use in grant evaluations
   • Employed in University Rankings
   

   View Slide

8. Impact Factor Abuse
   WITH GREAT POWER COMES GREAT RESPONSIBILITY
   • Impact Factor as tool for evaluating researchers’ productivity has altered
   the research landscape
   • Association with fraud
   BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD
   PNAS, doi:10.1073/pnas.1212247109
   

   View Slide

9. Impact Factor Abuse
   WITH GREAT POWER COMES GREAT RESPONSIBILITY
   • Impact Factor as tool for evaluating researchers’ productivity has altered
   the research landscape
   • Association with fraud
   BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD
   PNAS, doi:10.1073/pnas.1212247109
   

   View Slide

10. H-Index
    JE HIRSCH, PNAS, 102(46):16569 -16572 (2005)
    Metric of the scientific output of an individual, “…incorporating both quantity and
    visibility of publication”
    • H-index of 10 means having published 10 articles having at least 10 citations each
    • Field-dependent
    • Never decreases
    • No time-window
    • Insensitive to citation extremes
    • Insensitive to author’s contribution to article
    • Does not correct for self-citations
    

    View Slide

11. H-Index
    WITH GREAT POWER COMES GREAT RESPONSIBILITY
    • Simplistic, but measures the broad impact of one’s work
    • Should not be used as (sole) criterion to inform decision making in science
    PHYSICIST B
    PHYSICIST A
    H-index @ PhD: 5 10
    

    View Slide

12. TL;DR
    DON’T WORRY SO MUCH ABOUT THE IMPACT FACTOR
    • The absolute value of the impact factor is meaningless
    • It’s not about people, or articles, it’s about journals
    • No single metrics should be applied alone to evaluation of journals, research
    • We have a role to play in avoiding misuses and abuses
    • Matteo measurably better physicist than Einstein
    

    View Slide