Search bioRxivSearch

bioRxiv · 10.1101/479816

Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey of the Biomedical Research Community

Abstract

Scientific peer reviewers play an integral role in the grant selection process, yet very little has been reported on the levels of participation or the motivations of scientists to take part in peer review. AIBS developed a comprehensive peer review survey that examined the motivations and levels of participation of grant reviewers. The survey was disseminated to 13,091 scientists in AIBSs proprietary database. Of the 874 respondents, 76% indicated they had reviewed grant applications in the last 3 years; however, the number of reviews was unevenly distributed across this sample. Higher review loads were associated with respondents who had submitted more grant proposals over this time period, some of whom were likely to be study section members for large funding agencies. The most prevalent reason to participate in a review was to give back to the scientific community (especially among frequent grant submitters) and the most common reason to decline an invitation to review was lack of time. Interestingly, few suggested that expectation from the funding agency was a motivation to review. Most felt that review participation positively influenced their careers through improving grantsmanship and exposure to new scientific ideas. Of those who reviewed, respondents reported dedicating 2-5% of their total annual work time to grant review and, based on their self-reported maximum review loads, it is estimated they are participating at 56%-89% of their capacity, which may have important implications regarding the sustainability of the system. Overall, it is clear that participation in peer review is uneven and in some cases near capacity, and more needs to be done to create new motivations and incentives to increase the future pool of reviewers.

Source connections

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Gallo, S. A., Thompson, L. A., Schmaling, K. B., Glisson, S. R.. 2018-11-28. Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey of the Biomedical Research Community. https://doi.org/10.1101/479816

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education

The Case For and Against Double-blind Reviews

To date, the majority of authors on scientific publications have been men. While much of this gender bias can be explained by historic sexism and discrimination, there is concern that women may still be disadvantaged by the peer review process if reviewers unconscious biases lead them to reject publications with female authors more often. One potential solution to this perceived gender bias in the reviewing process is for journals to adopt double-blind reviews whereby neither the authors nor the reviewers are aware of each others identities and genders. To test the efficacy of double-blind reviews, we assigned gender to every authorship of every paper published in 5 different journals with different peer review processes (double-blind vs. single blind) and subject matter (birds vs. behavioral ecology) from 2010-2018 (n = 4865 papers). While female authorships comprised only 35% of the total, the double-blind journal Behavioral Ecology did not have more female authorships than its single-blind counterparts. Interestingly, the incidence of female authorship is higher at behavioral ecology journals (Behavioral Ecology and Behavioral Ecology and Sociobiology) than in the ornithology journals (Auk, Condor, Ibis), for papers on all topics as well as those on birds. These analyses suggest that double-blind review does not currently increase the incidence of female authorship in the journals studied here. We conclude, at least for these journals, that double-blind review does not benefit female authors and may, in the long run, be detrimental.

scientific communication and education

Using a diabetes discussion forum and Wikipedia to detect the alignment of public interests and the research literature

BackgroundDiabetes is a chronic disease that affects millions of people worldwide. It is therefore unsurprising that there is a high volume of public discussions, resources, and research tackling various aspects of the disease. Over the last decade, more than hundred thousand research articles have been published by researchers and countless of online discussions have taken place on various online platforms. This study is an attempt to identify the areas of public interest, related to diabetes, by looking at online discussion forums and to evaluate their relationship to pages about diabetes found on Wikipedia and to the academic research about the topic. The main aim is to investigate the extent to which researchers are responding to the publics interests and concerns, and to the level of uptake of the research topics in the public sphere.\n\nMethodology/Principal findingsTo detect public interests and concerns in diabetes, we collected posts on a popular diabetes discussion forum (DiabeticConnect) and pages (articles) about diabetes published in Wikipedia. We also downloaded the titles and abstracts of research articles about diabetes from the Scopus database, all between 2008 and 2016. Tags assigned to each post in the discussion forum were used along with the post itself to compute a Labeled Latent Dirichlet Allocation (LLDA) model, which was then used to classify the Wikipedia pages and research articles. The resulting classifications were then used to compare the prevalence of the topics found in the discussion forum with those of the other two sources. The results show that while research articles and Wikipedia pages about diabetes focus on diabetes testing, treatments, and disease control, the public forum discussions focus on Type 2 diabetes, emotional support, and proper diet for diabetic patients. However, for some other topics there was an alignment in the relative rise and fall of interest across the three platforms.\n\nConclusions/SignificanceThe alignment and misalignment in the changes of relative interest over the various topics is evidence that the LLDA modelling can be useful for comparing a public corpus, like a diabetes forum, and an academic one, like research titles and abstracts. The success of using LLDA to classify research articles based on the tags assigned to posts in a public discussion forum shows that this a promising method for better understanding how the scientific community responds to public interests and needs, and, on the flip side, how the public takes up the language and topics discussed by the academic community.

scientific communication and education