Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fear Factor: Use, misuse, and abuse of the impact factor

Matteo Cavalleri
February 16, 2021

Fear Factor: Use, misuse, and abuse of the impact factor

Internal presentation (Lunch & Learn, 2019) on the Journal Impact Factor. What it is, and what is not.

Matteo Cavalleri

February 16, 2021
Tweet

More Decks by Matteo Cavalleri

Other Decks in Science

Transcript

  1. Fear Factor: Use, misuse, and
    abuse of the impact factor
    MATTEO CAVALLERI
    Editor-in-Chief
    International Journal of Quantum Chemistry
    Publisher
    John Wiley & sons, inc.
    @physicsteo

    View full-size slide

  2. Show of hands
    • Do you know what the impact
    factor is, and how it is
    calculated?
    • Do you think higher impact
    factor = better quality journa?l
    • Do you heavily rely on the
    impact factor to advertise
    your journals to editors,
    authors, reviewers?
    • Do you know what the h-index
    is?

    View full-size slide

  3. Impact Factor Definitions
    (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972)
    JCR DEFINITION
    1972 DEFINITION
    “Dividing the number of times a
    journal has been cited by the
    number of articles it has published
    during some specific period of
    time. The journal impact factor
    reflect an average citation rate per
    article”
    -“All citations to the journal to
    items published in the previous
    two years divided by the total
    number of scholarly items
    published in the journal in the
    previous two years”
    2018 IF
    TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in
    2016 & 2017
    # of SCHOLARLY ITEMS (ARTICLES, REVIEWS,
    PROCEEDINGS) PUBLISHED in 2016 & 2017

    View full-size slide

  4. Impact Factor Definitions
    (EUGENE) GARFIELD, SCIENCE 178:471-479 (1972)
    JCR DEFINITION
    1972 DEFINITION
    “Dividing the number of times a
    journal has been cited by the
    number of articles it has published
    during some specific period of
    time. The journal impact factor
    reflect an average citation rate per
    article”
    -“All citations to the journal to
    items published in the previous
    two years divided by the total
    number of scholarly items
    published in the journal in the
    previous two years”
    2.263
    722
    319

    View full-size slide

  5. Impact Factor Meaning
    A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL
    WHAT IT IS
    2018 IF
    TOTAL CITATIONS in 2018 TO ITEMS PUBLISHED in
    2016 & 2017
    # of SCHOLARLY ITEMS (ARTICLES, REVIEWS,
    PROCEEDINGS) PUBLISHED in 2016 & 2017
    • Intuitive
    • Universally accepted / imitated
    • Not uncontroversial
    • Measure of journal prestige
    • Field specific
    • Skewed towards higher cited
    articles
    • Open to manipulation J. Am. Soc. Inf. Sci., 60: 27-34. doi:10.1002/asi.20936
    arXiv:1906.02660

    View full-size slide

  6. Impact Factor Meaning
    A MEASURE OF AVERAGE CITATION RATE OF A JOURNAL
    WHAT IT IS
    • Intuitive
    • Universally accepted / imitated
    • Not uncontroversial
    • Field specific
    • Measure of journal prestige
    • Skewed towards higher cited
    articles
    • Open to manipulation
    WHAT IT ISN’T
    • Transparent (underlying databased
    not publicly available)
    • Measure of quality of individual
    article / researcher
    • Measure of prestige of an
    institution
    • Predictor of likely citations
    @DuttonChemistry, Jason Dutton, Assoc. Prof. Latrobe University

    View full-size slide

  7. Impact Factor Misuse
    ESPECIALLY AS SOLE METRICS
    • Use in hiring/promotion/tenure decisions
    • Use in grant evaluations
    • Employed in University Rankings

    View full-size slide

  8. Impact Factor Abuse
    WITH GREAT POWER COMES GREAT RESPONSIBILITY
    • Impact Factor as tool for evaluating researchers’ productivity has altered
    the research landscape
    • Association with fraud
    BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD
    PNAS, doi:10.1073/pnas.1212247109

    View full-size slide

  9. Impact Factor Abuse
    WITH GREAT POWER COMES GREAT RESPONSIBILITY
    • Impact Factor as tool for evaluating researchers’ productivity has altered
    the research landscape
    • Association with fraud
    BMJ 2007; 334 doi: 10.1136/bmj.39142.454086.AD
    PNAS, doi:10.1073/pnas.1212247109

    View full-size slide

  10. H-Index
    JE HIRSCH, PNAS, 102(46):16569 -16572 (2005)
    Metric of the scientific output of an individual, “…incorporating both quantity and
    visibility of publication”
    • H-index of 10 means having published 10 articles having at least 10 citations each
    • Field-dependent
    • Never decreases
    • No time-window
    • Insensitive to citation extremes
    • Insensitive to author’s contribution to article
    • Does not correct for self-citations

    View full-size slide

  11. H-Index
    WITH GREAT POWER COMES GREAT RESPONSIBILITY
    • Simplistic, but measures the broad impact of one’s work
    • Should not be used as (sole) criterion to inform decision making in science
    PHYSICIST B
    PHYSICIST A
    H-index @ PhD: 5 10

    View full-size slide

  12. TL;DR
    DON’T WORRY SO MUCH ABOUT THE IMPACT FACTOR
    • The absolute value of the impact factor is meaningless
    • It’s not about people, or articles, it’s about journals
    • No single metrics should be applied alone to evaluation of journals, research
    • We have a role to play in avoiding misuses and abuses
    • Matteo measurably better physicist than Einstein

    View full-size slide