Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A peek inside the black box of scientific publishing

A peek inside the black box of scientific publishing

Publishing the results of one’s research is an integral part of the scientific process, and of scientists’ careers, yet scholarly journals are often seen as black boxes by researchers.

With this talk I aim to reduce the opacity of the inner working of scientific journals and to de-mystify the peer review process from an editor's perspective. I’ll also share my (very happy) experience going from a computational chemistry lab to my current career on the “other side” of scientific publishing, and explore roles for STEM Ph.D.s in the publishing industry.

Matteo Cavalleri

April 30, 2021
Tweet

More Decks by Matteo Cavalleri

Other Decks in Science

Transcript

  1. A peek inside the black box of scientific publishing
    Dr Matteo Cavalleri, Publisher, Materials Science & Physics
    e-Girona, April 27, 2021
    Wiley
    @physicsteo linkedin.com/teowaits
    [email protected]

    View Slide

  2. Who I am
    M.S. in Chemistry (1999)
    PhD in Chemical Physics (2005)
    PostDoc 2005-2008

    View Slide

  3. My career ”on the other side”
    Peer-Review Editor (2008-2010), Berlin
    Associate Editor (2010-2012), NYC
    Editor-in-Chief (2012-2020), NYC
    Rich media objects (data,
    code, notebooks, and
    interactive visualizations)
    integrated in the Version of
    Record
    The “Article of the Future” (IJQC, 2020)

    View Slide

  4. “The Article of the Future”
    Data are connected to the
    figure that describe them.
    They are discoverable,
    indexable, citable, and
    usable
    The code is in the figure, in
    an executable time capsule
    Peer review reports,
    author rebuttals and
    editor notes get a DOI
    and can be cited
    Experience the future: http://bit.ly/IJQC_AotF

    View Slide

  5. My career “on the other side” (2020-)
    New IJQC Editor-in-Chief”: Gian Asara Publisher (Present)
    PhD @ UB w/ Prof. J.M. Ricart and Prof. F. Illas
    • International in-house editorial team of 7, based in East Coast of US
    • Lead US in-house editorial team (co-founded in 2010)
    • Publisher of the Materials Science & Physics team
    • Portfolio includes OA & subscription titles
    !"#$%&

    View Slide

  6. Where I work
    1807 mid 1800s Today
    Charles Wiley opened a print shop in
    New York City, publishing literary fiction
    and non-fiction.
    John Wiley & Sons began focusing on
    science, technical, and engineering
    publishing.
    Seven generations later, Wiley is one of
    the oldest independent publishing
    companies.
    1700+ journals, 400+ Open Access journals, 7.5M articles
    500+ societies, 2M members
    #WorkFromHome

    View Slide

  7. How do journals work?

    View Slide

  8. How do journals work?
    By Nick Kim (www.nearingzero.net); used with permission

    View Slide

  9. How do journals work?

    View Slide

  10. How do journals REALLY work?
    Dissemination
    Spreading the word
    through publishing
    platforms
    But also indexing and
    generally organizing
    knowledge
    Registration
    Precedence of discovery is
    established based on
    article submission date to
    a journal
    Archival
    Safeguarding and
    preserving knowledge
    Publishers play an
    important role preserving
    the scientific record
    Certification
    Peer-review is still the
    gold standard for
    certifying articles
    This is not the same as
    quality-control!
    Peer-review management, Curation, Infrastructure, Ethics, & much, much more. Here’s a list of 96 things publishers do: https://bit.ly/2UW3rKX
    The “Publish or Perish” culture responsible for an environment where (some)
    scientific journals are de facto gatekeepers of science

    View Slide

  11. Author
    submits
    article
    Rejected
    Article
    assessed by
    editor
    Sent to
    reviewers
    Author
    submits
    revised paper
    Revision
    required
    Further
    review
    needed?
    Reviews
    assessed by
    editor
    Rejected
    Accepted
    Publication
    Production
    How do journals REALLY work?

    View Slide

  12. How do journal REALLY work?
    “Peer review is the critical assessment of
    manuscripts submitted to journals by
    experts who are not part of the editorial
    staff”-International Committee of Medical Journals Editors

    View Slide

  13. How did we get here?
    “We (Mr. Rosen and I) had sent you our manuscript for publication and had not
    authorised you to show it to specialists before it is printed. I see no reason to address
    the – in any case erroneous – comments of your anonymous expert.”
    1665: Publication of Journal des Sçavans and Philosophical Transactions of the Royal Society.
    1930s-1940s: Peer-review emerges, not very popular at first
    1960s: Peer-review becomes the institutional standard (Nature, 1967)
    Today: Enjoy overwhelming support despite issues*, alternatives emerge
    * Slow and costly, systemic biases, still misses fraudulent papers

    View Slide

  14. Peer-Review Types
    -SINGLE BLIND: Most common
    -DOUBLE BLIND: Authors undisclosed to reviewers
    -OPEN: Public, community peer-review
    -SIGNED: Non-anonymous referees, BMJ Open
    -TECHNICAL PEER-REVIEW ONLY: PLoS One, Scientific Reports, PeerJ,…
    -MIX OF THE ABOVE: IJQC experiment, “Frontiers In”, EMBO…
    -NONE: Evaluation by community post-publication, arXiv, F1000,…
    -INDIPENDENT FROM JOURNAL: Rubiq, Peerage of Science
    -PORTABLE: Within the same publishing house, portfolio of journals
    -TRANSPARENT: Report published with accepted article
    Credit: Andrew Bissette, @andrewbissette
    -TRIPLE BLIND: Authors undisclosed to reviewers and editors

    View Slide

  15. The editorial office
    EXTERNAL EDITORS
    IN-HOUSE EDITORS
    …all ACS, T&F, OUP, most
    Springer-Nature, Elsevier, Wiley
    journals…
    …+ some APS, RSC, IOP titles,
    Cell Press, Science, PLOS, The
    Lancet…

    View Slide

  16. The editorial office
    EXTERNAL EDITORS
    IN-HOUSE EDITORS
    Have PhDs, often PostDoc
    experience
    Work full time on journal – can
    dedicate more time and
    resources on new
    developments
    General view
    Have own research group
    Expert in specific field
    BOTH: peer-review, decision making, dealing with appeals,
    commissioning, conference participation and lab visits, writing news
    stories, contributing to “input” marketing …

    View Slide

  17. What editors look for?
    MOST JOURNALS
    -Novelty
    -Importance (in specific field / in related disciplines)
    -Interest
    ALL JOURNALS
    -Scope
    -Format (Communication, full paper, review…)
    -Understandability
    -Compliance to guidelines, ethical behavior
    Editors are not always qualified to evaluate the technical merits of manuscripts.
    This is the job of the referees.

    View Slide

  18. How are referees chosen?
    -Editors’ knowledge & experience
    -From related papers:
    - cited manuscripts
    - literature search
    -Additional research:
    - conference/lab visits
    - web search (good ‘ol Google)
    -Reviewer database:
    - keywords, interest, history…

    View Slide

  19. Referee suggestions are welcome
    -Not just the big names, please
    -No collaborators, previous advisors, grant co-applicants, …
    -Tell us about circumstances that may prevent impartial review:
    - close competitors, who may “scoop” you
    - other conflicts
    …within reason…
    Follow the 3W rule when suggesting reviewers: Tell the editor WHO, WHERE, WHY

    View Slide

  20. Accept, reject, or revise?
    -REJECTION
    - Without external referee reports (Editor)
    - Based on reports
    -REVISION
    - Reconsideration or resubmission
    possible after major revisions
    -ACCEPTANCE
    - Without changes (rare)
    - With minor changes
    The decision is the Editor’s job…the reviewer ‘s recommendation is not a vote -- it’s advice!

    View Slide

  21. Revision
    -REVISION
    • Carefully consider referee
    comments
    • Not all changes have to be
    made…
    • …but need convincing
    arguments for changes not
    made
    • Prepare revision
    • Revise manuscript
    • Highlight changes in
    manuscript
    • Point-by-point response to
    all referee criticisms
    • Changes made
    • Why changes not made
    • Response may go back to
    referees!
    • Need to convince editor
    and referees
    The peer-review process is not a private conversation between authors and
    referees. Try to work your answers to the reviewers in the revised manuscript!

    View Slide

  22. Rejection – not the end of the world!
    -REJECTION
    • Most scientists have been rejected– do not take it personally
    • Try to understand why the paper was rejected
    • Note that you have received the benefit of the Editors and reviewers’ time: take their advice seriously!
    • Re-evaluate your work
    • If you resubmit, begin as if you are going to write a new article
    • Consider offers to transfer for your manuscript to another related journal

    View Slide

  23. Reject and Transfer
    -REJECTION
    -Sharing referee reports and communication
    among editors allow research to find the right
    audience
    -Fast publication, often without further peer-
    review (if rejected on reports)
    -40% of submissions/year (~1.3M manuscripts!)
    are rejected after peer-review (Rubriq, 2013).
    Often by the same reviewers.

    View Slide

  24. Appeal
    USUALLY, NO
    -Risk of long time of publication
    -Good papers are found and cited
    -Editors & referees know journal well
    OCCASIONALLY, YES
    -Importance/novelty missed by editor/referees
    -Factual error in referee reports that lead to rejection
    -Need more clarification of decision
    Be calm, argumentative and give scientific justification for reassessment
    It’s false that Editors hate appeals. So should you appeal a rejection?
    Dear Dr. *****:
    Regarding your
    decision on my
    submission **-*****,
    I am afraid you and
    the referees are
    crazy. Please
    reconsider your
    decision.
    Best wishes
    -REJECTION

    View Slide

  25. General structure of a scientific article
    1. Have something to say
    2. Say it
    3. Stop as soon as you have said it.
    Billings, J., An address to our medical literature. British Medical Journal 1881, 262-268

    View Slide

  26. Figures & Tables = Your paper’s storyboard
    Figure 1
    Figure 2
    Figure 3
    …..
    Figure 7
    Figure 8
    Star Wars, Episode V; The Empire Strikes Back

    View Slide

  27. The A,B,C of good scientific writing
    • Avoid vague language and be
    precise/specific
    • Say EXACTLY what you mean and
    avoid over/under statements
    • Make the discussion concise but informative. Focus on the
    important and unexpected results. Not on small details.
    • Use as few words as possible while retaining meaning without
    sacrificing scientific details
    Use simple words and avoid jargon
    Use verb tense consistently throughout the paper
    Where possible, use verbs instead of noun forms

    View Slide

  28. Search Engine Optimization (SEO) for your article
    • Search-Engine friendly Title/Abstract
    • Use keywords throughout the article
    • Be consistent with authors names
    • In-bound links rule Google. Link your article
    across social media, networking and
    institutional sites
    • Network, highlight/elevate your colleagues,
    they will do the same for you!
    • Share data, code. Open science leads to
    greater collaboration, increased confidence
    in results and goodwill between researchers
    • Most journals welcome preprints!
    HELP PEOPLE FIND YOU
    https://authorservices.wiley.com/author-resources/Journal-Authors/Prepare/writing-for-seo.html

    View Slide

  29. One editor’s origin story

    View Slide

  30. Journal Editor: A day in the life*
    OUTREACH
    DAY-TO-DAY MANAGEMENT OF A JOURNAL
    • Read manuscript submitted to the journal
    • Identify and assign manuscript to reviewers (or sub-Editors)
    • Make editorial decisions
    • Solicit manuscripts and special issues
    • Deal and resolve (hopefully!) ethical issues regarding submissions/published papers
    • Collaborate with editorial, production, marketing team
    • Work closely with authors, reviewers, other editors
    • Beta testing new publishing technologies
    • Help authors to disseminate their work further
    • Attend international conferences
    • Workshops, lab/institution visits
    *Work days are pretty reasonable, actually

    View Slide

  31. In-house editors: Life as a pro
    …AND WHAT WE ARE NOT…
    • Active in research (not even part-time!)
    • Lone wolves & hermits
    WHAT WE ARE…
    • Full-time employees of a publisher
    • Former researchers (PhD, often PostDoc experience)
    • Working in small editorial team (1~20 ppl), supported by editorial board
    • Work (mostly, especially initially) from the publisher’s offices*
    • Holding different titles (EiC, associate,..) depending on experience & responsibility
    • Responsible for peer-review process and editorial decisions
    • Responsible for content commissioning
    • The face of the publisher in the community, voice of researchers in the publisher
    *pre-COVID, at least

    View Slide

  32. Publisher*: A day in the life
    LAUNCH/ACQUISITION OF NEW PRODUCTS
    • New journals
    • Journal relaunches
    • Awards, conferences, workshops
    • Webinars, online courses
    STRATEGIC MANAGEMENT AND DEVELOPMENT OF JOURNALS
    • Hire, train, and manage editors and editorial assistants
    • Roll out new publishing initiatives and processes
    • Planning, budgeting, and reporting (Editors don’t deal with finance!)
    • Develop common strategies and goals for portfolio/groups of journals
    • Bring the perspective of the researchers to the company
    • Collaborate closely with editors, marketers, other colleagues
    *”WILEY Publisher”. Titles are not standard in the industry

    View Slide

  33. Other roles: Not just peer-review
    MANAGING EDITOR/JOURNAL PUBLISHING EDITOR
    • Scientific background (sometimes, but not always)
    • Support academic EiC, liaison with production and marketing team
    • Can be involved in making desk rejections
    DEVELOPMENTAL/ACQUISITION EDITOR
    • Scientific backgrounds (often)
    • Commission articles and special issues
    • Responsible for acquisition/launch/improvement of scholarly products
    • Common role in book publishing
    COPY EDITOR
    • Scientific background (often at MS/BS level, not PhD)
    • Responsible for proof-reading manuscripts
    • Mostly present in ”apex” titles, major brands

    View Slide

  34. Other roles: Not just peer-review
    MANAGING EDITOR/JOURNAL PUBLISHING EDITOR
    • Scientific background (sometimes, but not always)
    • Support academic EiC, liaison with production and marketing team
    • Can be involved in making desk rejections
    DEVELOPMENTAL/ACQUISITION EDITOR
    • Scientific backgrounds (often)
    • Commission articles and special issues
    • Responsible for acquisition/launch/improvement of scholarly products
    • Common role in book publishing
    COPY EDITOR
    • Scientific background (often at MS/BS level, not PhD)
    • Responsible for proof-reading manuscripts
    • Mostly present in ”apex” titles, major brands

    View Slide

  35. What’s hot & what’s not
    …AND WHAT I WOULD DO WITHOUT
    • Journal/process development can be slow and frustrating
    • Angry authors are difficult to deal with
    • Fraud/Ethical violations are not uncommon and very exasperating!
    • Sometimes I miss coding, hacking hardware (being a “lab-rat”)
    • Career progression after Editor-in-Chief/JPE not easy
    WHAT I LOVE…
    • It’s a career at the “center of science”
    • Entrusted the knowledge of entire disciplines
    • Bird’s-eye view over science, see best results 1st!
    • Contact with the scientific community
    • Add & participate to the scientific debate and progress
    • Plenty of (international) travel*
    • Real possibility of professional growth
    *pre-COVID, at least

    View Slide

  36. What is a good editor made of?
    … BUT YOU WON’T LOVE IT IF YOU …
    • love being in the lab and doing research
    • enjoy being the world expert in a specific subject
    • don’t like changing topics several times a day
    • hated writing your thesis
    IT MAY BE THE JOB FOR YOU IF YOU …
    • are passionate for science communication
    • recognize the importance of publishing in the scientific process
    • are curious about a broad range of topics & disciplines
    • know the art of diplomacy and have people skills
    • have analytical and decision-making skills
    • are creative, with an eye for detail (and the “next big thing”)
    ENGLISH IS THE LANGUAGE OF SCIENCE
    • Publishing not restricted to native speakers anymore
    • BUT, you need to be fluent in communicating science with it

    View Slide

  37. Dr Matteo Cavalleri @physicsteo linkedin.com/teowaits
    [email protected]

    View Slide

  38. Wiley is wonderful. Really, but other places are available….
    and more…
    In-house editors wanted:
    Other roles:

    View Slide