A peek inside the black box of scientific publishing

Publishing the results of one’s research is an integral part of the scientific process, and of scientists’ careers, yet scholarly journals are often seen as black boxes by researchers.

With this talk I aim to reduce the opacity of the inner working of scientific journals and to de-mystify the peer review process from an editor's perspective. I’ll also share my (very happy) experience going from a computational chemistry lab to my current career on the “other side” of scientific publishing, and explore roles for STEM Ph.D.s in the publishing industry.

Transcript

1. A peek inside the black box of scientific publishing
   Dr Matteo Cavalleri, Publisher, Materials Science & Physics
   e-Girona, April 27, 2021
   Wiley
   @physicsteo linkedin.com/teowaits
   mcavalleri@wiley.com
   

   View Slide

2. Who I am
   M.S. in Chemistry (1999)
   PhD in Chemical Physics (2005)
   PostDoc 2005-2008
   

   View Slide

3. My career ”on the other side”
   Peer-Review Editor (2008-2010), Berlin
   Associate Editor (2010-2012), NYC
   Editor-in-Chief (2012-2020), NYC
   Rich media objects (data,
   code, notebooks, and
   interactive visualizations)
   integrated in the Version of
   Record
   The “Article of the Future” (IJQC, 2020)
   

   View Slide

4. “The Article of the Future”
   Data are connected to the
   figure that describe them.
   They are discoverable,
   indexable, citable, and
   usable
   The code is in the figure, in
   an executable time capsule
   Peer review reports,
   author rebuttals and
   editor notes get a DOI
   and can be cited
   Experience the future: http://bit.ly/IJQC_AotF
   

   View Slide

5. My career “on the other side” (2020-)
   New IJQC Editor-in-Chief”: Gian Asara Publisher (Present)
   PhD @ UB w/ Prof. J.M. Ricart and Prof. F. Illas
   • International in-house editorial team of 7, based in East Coast of US
   • Lead US in-house editorial team (co-founded in 2010)
   • Publisher of the Materials Science & Physics team
   • Portfolio includes OA & subscription titles
   !"#$%&
   

   View Slide

6. Where I work
   1807 mid 1800s Today
   Charles Wiley opened a print shop in
   New York City, publishing literary fiction
   and non-fiction.
   John Wiley & Sons began focusing on
   science, technical, and engineering
   publishing.
   Seven generations later, Wiley is one of
   the oldest independent publishing
   companies.
   1700+ journals, 400+ Open Access journals, 7.5M articles
   500+ societies, 2M members
   #WorkFromHome
   

   View Slide

7. How do journals work?
   

   View Slide

8. How do journals work?
   By Nick Kim (www.nearingzero.net); used with permission
   

   View Slide

9. How do journals work?
   

   View Slide

10. How do journals REALLY work?
    Dissemination
    Spreading the word
    through publishing
    platforms
    But also indexing and
    generally organizing
    knowledge
    Registration
    Precedence of discovery is
    established based on
    article submission date to
    a journal
    Archival
    Safeguarding and
    preserving knowledge
    Publishers play an
    important role preserving
    the scientific record
    Certification
    Peer-review is still the
    gold standard for
    certifying articles
    This is not the same as
    quality-control!
    Peer-review management, Curation, Infrastructure, Ethics, & much, much more. Here’s a list of 96 things publishers do: https://bit.ly/2UW3rKX
    The “Publish or Perish” culture responsible for an environment where (some)
    scientific journals are de facto gatekeepers of science
    

    View Slide

11. Author
    submits
    article
    Rejected
    Article
    assessed by
    editor
    Sent to
    reviewers
    Author
    submits
    revised paper
    Revision
    required
    Further
    review
    needed?
    Reviews
    assessed by
    editor
    Rejected
    Accepted
    Publication
    Production
    How do journals REALLY work?
    

    View Slide

12. How do journal REALLY work?
    “Peer review is the critical assessment of
    manuscripts submitted to journals by
    experts who are not part of the editorial
    staff”-International Committee of Medical Journals Editors
    

    View Slide

13. How did we get here?
    “We (Mr. Rosen and I) had sent you our manuscript for publication and had not
    authorised you to show it to specialists before it is printed. I see no reason to address
    the – in any case erroneous – comments of your anonymous expert.”
    1665: Publication of Journal des Sçavans and Philosophical Transactions of the Royal Society.
    1930s-1940s: Peer-review emerges, not very popular at first
    1960s: Peer-review becomes the institutional standard (Nature, 1967)
    Today: Enjoy overwhelming support despite issues*, alternatives emerge
    * Slow and costly, systemic biases, still misses fraudulent papers
    

    View Slide

14. Peer-Review Types
    -SINGLE BLIND: Most common
    -DOUBLE BLIND: Authors undisclosed to reviewers
    -OPEN: Public, community peer-review
    -SIGNED: Non-anonymous referees, BMJ Open
    -TECHNICAL PEER-REVIEW ONLY: PLoS One, Scientific Reports, PeerJ,…
    -MIX OF THE ABOVE: IJQC experiment, “Frontiers In”, EMBO…
    -NONE: Evaluation by community post-publication, arXiv, F1000,…
    -INDIPENDENT FROM JOURNAL: Rubiq, Peerage of Science
    -PORTABLE: Within the same publishing house, portfolio of journals
    -TRANSPARENT: Report published with accepted article
    Credit: Andrew Bissette, @andrewbissette
    -TRIPLE BLIND: Authors undisclosed to reviewers and editors
    

    View Slide

15. The editorial office
    EXTERNAL EDITORS
    IN-HOUSE EDITORS
    …all ACS, T&F, OUP, most
    Springer-Nature, Elsevier, Wiley
    journals…
    …+ some APS, RSC, IOP titles,
    Cell Press, Science, PLOS, The
    Lancet…
    

    View Slide

16. The editorial office
    EXTERNAL EDITORS
    IN-HOUSE EDITORS
    Have PhDs, often PostDoc
    experience
    Work full time on journal – can
    dedicate more time and
    resources on new
    developments
    General view
    Have own research group
    Expert in specific field
    BOTH: peer-review, decision making, dealing with appeals,
    commissioning, conference participation and lab visits, writing news
    stories, contributing to “input” marketing …
    

    View Slide

17. What editors look for?
    MOST JOURNALS
    -Novelty
    -Importance (in specific field / in related disciplines)
    -Interest
    ALL JOURNALS
    -Scope
    -Format (Communication, full paper, review…)
    -Understandability
    -Compliance to guidelines, ethical behavior
    Editors are not always qualified to evaluate the technical merits of manuscripts.
    This is the job of the referees.
    

    View Slide

18. How are referees chosen?
    -Editors’ knowledge & experience
    -From related papers:
    - cited manuscripts
    - literature search
    -Additional research:
    - conference/lab visits
    - web search (good ‘ol Google)
    -Reviewer database:
    - keywords, interest, history…
    

    View Slide

19. Referee suggestions are welcome
    -Not just the big names, please
    -No collaborators, previous advisors, grant co-applicants, …
    -Tell us about circumstances that may prevent impartial review:
    - close competitors, who may “scoop” you
    - other conflicts
    …within reason…
    Follow the 3W rule when suggesting reviewers: Tell the editor WHO, WHERE, WHY
    

    View Slide

20. Accept, reject, or revise?
    -REJECTION
    - Without external referee reports (Editor)
    - Based on reports
    -REVISION
    - Reconsideration or resubmission
    possible after major revisions
    -ACCEPTANCE
    - Without changes (rare)
    - With minor changes
    The decision is the Editor’s job…the reviewer ‘s recommendation is not a vote -- it’s advice!
    

    View Slide

21. Revision
    -REVISION
    • Carefully consider referee
    comments
    • Not all changes have to be
    made…
    • …but need convincing
    arguments for changes not
    made
    • Prepare revision
    • Revise manuscript
    • Highlight changes in
    manuscript
    • Point-by-point response to
    all referee criticisms
    • Changes made
    • Why changes not made
    • Response may go back to
    referees!
    • Need to convince editor
    and referees
    The peer-review process is not a private conversation between authors and
    referees. Try to work your answers to the reviewers in the revised manuscript!
    

    View Slide

22. Rejection – not the end of the world!
    -REJECTION
    • Most scientists have been rejected– do not take it personally
    • Try to understand why the paper was rejected
    • Note that you have received the benefit of the Editors and reviewers’ time: take their advice seriously!
    • Re-evaluate your work
    • If you resubmit, begin as if you are going to write a new article
    • Consider offers to transfer for your manuscript to another related journal
    

    View Slide

23. Reject and Transfer
    -REJECTION
    -Sharing referee reports and communication
    among editors allow research to find the right
    audience
    -Fast publication, often without further peer-
    review (if rejected on reports)
    -40% of submissions/year (~1.3M manuscripts!)
    are rejected after peer-review (Rubriq, 2013).
    Often by the same reviewers.
    

    View Slide

24. Appeal
    USUALLY, NO
    -Risk of long time of publication
    -Good papers are found and cited
    -Editors & referees know journal well
    OCCASIONALLY, YES
    -Importance/novelty missed by editor/referees
    -Factual error in referee reports that lead to rejection
    -Need more clarification of decision
    Be calm, argumentative and give scientific justification for reassessment
    It’s false that Editors hate appeals. So should you appeal a rejection?
    Dear Dr. *****:
    Regarding your
    decision on my
    submission **-*****,
    I am afraid you and
    the referees are
    crazy. Please
    reconsider your
    decision.
    Best wishes
    -REJECTION
    

    View Slide

25. General structure of a scientific article
    1. Have something to say
    2. Say it
    3. Stop as soon as you have said it.
    Billings, J., An address to our medical literature. British Medical Journal 1881, 262-268
    

    View Slide

26. Figures & Tables = Your paper’s storyboard
    Figure 1
    Figure 2
    Figure 3
    …..
    Figure 7
    Figure 8
    Star Wars, Episode V; The Empire Strikes Back
    

    View Slide

27. The A,B,C of good scientific writing
    • Avoid vague language and be
    precise/specific
    • Say EXACTLY what you mean and
    avoid over/under statements
    • Make the discussion concise but informative. Focus on the
    important and unexpected results. Not on small details.
    • Use as few words as possible while retaining meaning without
    sacrificing scientific details
    Use simple words and avoid jargon
    Use verb tense consistently throughout the paper
    Where possible, use verbs instead of noun forms
    

    View Slide

28. Search Engine Optimization (SEO) for your article
    • Search-Engine friendly Title/Abstract
    • Use keywords throughout the article
    • Be consistent with authors names
    • In-bound links rule Google. Link your article
    across social media, networking and
    institutional sites
    • Network, highlight/elevate your colleagues,
    they will do the same for you!
    • Share data, code. Open science leads to
    greater collaboration, increased confidence
    in results and goodwill between researchers
    • Most journals welcome preprints!
    HELP PEOPLE FIND YOU
    https://authorservices.wiley.com/author-resources/Journal-Authors/Prepare/writing-for-seo.html
    

    View Slide

29. One editor’s origin story
    

    View Slide

30. Journal Editor: A day in the life*
    OUTREACH
    DAY-TO-DAY MANAGEMENT OF A JOURNAL
    • Read manuscript submitted to the journal
    • Identify and assign manuscript to reviewers (or sub-Editors)
    • Make editorial decisions
    • Solicit manuscripts and special issues
    • Deal and resolve (hopefully!) ethical issues regarding submissions/published papers
    • Collaborate with editorial, production, marketing team
    • Work closely with authors, reviewers, other editors
    • Beta testing new publishing technologies
    • Help authors to disseminate their work further
    • Attend international conferences
    • Workshops, lab/institution visits
    *Work days are pretty reasonable, actually
    

    View Slide

31. In-house editors: Life as a pro
    …AND WHAT WE ARE NOT…
    • Active in research (not even part-time!)
    • Lone wolves & hermits
    WHAT WE ARE…
    • Full-time employees of a publisher
    • Former researchers (PhD, often PostDoc experience)
    • Working in small editorial team (1~20 ppl), supported by editorial board
    • Work (mostly, especially initially) from the publisher’s offices*
    • Holding different titles (EiC, associate,..) depending on experience & responsibility
    • Responsible for peer-review process and editorial decisions
    • Responsible for content commissioning
    • The face of the publisher in the community, voice of researchers in the publisher
    *pre-COVID, at least
    

    View Slide

32. Publisher*: A day in the life
    LAUNCH/ACQUISITION OF NEW PRODUCTS
    • New journals
    • Journal relaunches
    • Awards, conferences, workshops
    • Webinars, online courses
    STRATEGIC MANAGEMENT AND DEVELOPMENT OF JOURNALS
    • Hire, train, and manage editors and editorial assistants
    • Roll out new publishing initiatives and processes
    • Planning, budgeting, and reporting (Editors don’t deal with finance!)
    • Develop common strategies and goals for portfolio/groups of journals
    • Bring the perspective of the researchers to the company
    • Collaborate closely with editors, marketers, other colleagues
    *”WILEY Publisher”. Titles are not standard in the industry
    

    View Slide

33. Other roles: Not just peer-review
    MANAGING EDITOR/JOURNAL PUBLISHING EDITOR
    • Scientific background (sometimes, but not always)
    • Support academic EiC, liaison with production and marketing team
    • Can be involved in making desk rejections
    DEVELOPMENTAL/ACQUISITION EDITOR
    • Scientific backgrounds (often)
    • Commission articles and special issues
    • Responsible for acquisition/launch/improvement of scholarly products
    • Common role in book publishing
    COPY EDITOR
    • Scientific background (often at MS/BS level, not PhD)
    • Responsible for proof-reading manuscripts
    • Mostly present in ”apex” titles, major brands
    

    View Slide

34. Other roles: Not just peer-review
    MANAGING EDITOR/JOURNAL PUBLISHING EDITOR
    • Scientific background (sometimes, but not always)
    • Support academic EiC, liaison with production and marketing team
    • Can be involved in making desk rejections
    DEVELOPMENTAL/ACQUISITION EDITOR
    • Scientific backgrounds (often)
    • Commission articles and special issues
    • Responsible for acquisition/launch/improvement of scholarly products
    • Common role in book publishing
    COPY EDITOR
    • Scientific background (often at MS/BS level, not PhD)
    • Responsible for proof-reading manuscripts
    • Mostly present in ”apex” titles, major brands
    

    View Slide

35. What’s hot & what’s not
    …AND WHAT I WOULD DO WITHOUT
    • Journal/process development can be slow and frustrating
    • Angry authors are difficult to deal with
    • Fraud/Ethical violations are not uncommon and very exasperating!
    • Sometimes I miss coding, hacking hardware (being a “lab-rat”)
    • Career progression after Editor-in-Chief/JPE not easy
    WHAT I LOVE…
    • It’s a career at the “center of science”
    • Entrusted the knowledge of entire disciplines
    • Bird’s-eye view over science, see best results 1st!
    • Contact with the scientific community
    • Add & participate to the scientific debate and progress
    • Plenty of (international) travel*
    • Real possibility of professional growth
    *pre-COVID, at least
    

    View Slide

36. What is a good editor made of?
    … BUT YOU WON’T LOVE IT IF YOU …
    • love being in the lab and doing research
    • enjoy being the world expert in a specific subject
    • don’t like changing topics several times a day
    • hated writing your thesis
    IT MAY BE THE JOB FOR YOU IF YOU …
    • are passionate for science communication
    • recognize the importance of publishing in the scientific process
    • are curious about a broad range of topics & disciplines
    • know the art of diplomacy and have people skills
    • have analytical and decision-making skills
    • are creative, with an eye for detail (and the “next big thing”)
    ENGLISH IS THE LANGUAGE OF SCIENCE
    • Publishing not restricted to native speakers anymore
    • BUT, you need to be fluent in communicating science with it
    

    View Slide

37. Dr Matteo Cavalleri @physicsteo linkedin.com/teowaits
    mcavalleri@wiley.com
    

    View Slide

38. Wiley is wonderful. Really, but other places are available….
    and more…
    In-house editors wanted:
    Other roles:
    

    View Slide