Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2021.07.28.454037

Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors

O'Brien, B. C.·Artino, A. R.·Costello, J. A.·Driessen, E.·Maggio, L. A.
View original source ↗Export metadata ↓

Abstract

PurposeRecent calls to improve transparency in peer review have prompted examination of many aspects of the peer review process. Peer review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. MethodsOur dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended "reject," we coded for alignment between reviewers comments to the editor and to authors. We report descriptive statistics. Results49% of reviews contained comments to the editor (n=176). Most of these comments summarized the reviewers impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 48 reviews recommending "reject," the majority of comments to the editor contained content that also appeared in comments to the authors (65%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (85%). ConclusionFindings indicate variability in how reviewers use the confidential comments to editor section in online peer review systems, though generally the way they use them suggests integrity and transparency to authors.

Source connections

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

O'Brien, B. C., Artino, A. R., Costello, J. A., Driessen, E., Maggio, L. A.. 2021-07-28. Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors. https://doi.org/10.1101/2021.07.28.454037

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2023

MolecularWebXR: Multiuser discussions about chemistry and biology in immersive and inclusive VR

MolecularWebXR is a new website for education, science communication and scientific peer discussion in chemistry and biology, based on modern web-based Virtual Reality (VR) and Augmented Reality (AR). With no installs as it is all web-served, MolecularWebXR enables multiple users to simultaneously explore, communicate and discuss concepts about chemistry and biology in immersive 3D environments, by manipulating and passing around objects with their bare hands and pointing at different elements with natural hand gestures. User may either be present in the same real space or distributed around the world, in the latter case talking naturally with each other thanks to built-in audio features. Although MolecularWebXR is most immersive when running in the web browsers of high-end AR/VR headsets, its WebXR core also allows participation by users with consumer devices such as smartphones, possibly inserted into cardboard goggles for deeper immersivity, or even in computers and tablets. MolecularWebXR comes with preset VR rooms that cover topics from general, inorganic and organic chemistry, biophysics and structural biology, and general biology; besides, new content can be added at will through moleculARwebs PDB2AR tool or by contacting the lead authors. We verified MolecularWebXRs ease of use and versatility by people aged 12-80 years old in entirely virtual sessions or in mixed real-virtual sessions at various science outreach events, in courses at the bachelor, masters and early doctoral levels, in scientific collaborations, and in conference lectures. MolecularWebXR is available for free use without registration at https://molecularwebxr.org, and a blog post version of this preprint with embedded videos is available at https://go.epfl.ch/molecularwebxr-blog-post.

scientific communication and education↗
bioRxiv2023

Willingness of Japanese people in their 20s, 30s and 40s to pay for genetically modified foods

The application of genetically modified (GM) technology to food products has increased worldwide. The adaptation has extended to conventional grains and animal products, such as salmon. However, in Japan, the publics acceptance of GM foods is low and experts and policymakers need to know the publics preference for various types of GM foods. Therefore, this study aims to clarify and compare the preferences for various GM foods among Japanese people in their 20s, 30s, and 40s, using the Willingness-to-pay (WTP) indicator. An online survey with 1122 valid responses from people in their 20s-40s was used for analysis. The results showed that the percentages of willingness to purchase various items were as follows - GM blue roses (n = 628; 56%), tomatoes fertilized with GM oil cake (n = 519; 46.3%), potato chips made from GM potatoes (n = 489; 43.6%), chicken thighs fed on GM corn (n = 472; 42.1%), corn flakes made from GM corn (n = 471; 42.0%), GM apples (n = 420; 37.4%), wine brewed with GM yeast (n = 416; 37.1%), GM tomatoes (n = 408; 36.4%), GM chicken thighs (n = 360; 32.1%), GM salmon (n = 349; 31.1%). Comparing the WTP discount rates for animal and plant foods, it was around 20% for animal products (GM chicken thighs = 17.8% and GM salmon = 19.5%), and 15-35% for plant foods (GM corn flakes = 24.1%, GM tomatoes = 17.2%, GM potato chips = 16.4%, GM yeast wine = 36.7%, GM apples = 25.3%). The WTP discount rates were 11.9% for tomatoes fertilized with GM oil cake compared with 17.2% for GM tomatoes, and 21.5% for GM-fed animal products compared with 17.8% for chicken thighs fed with GM corn. Therefore, the WTP values of GM animal foods were lower than GM plant foods and ornamental products.

scientific communication and education↗