Community review: a robust and scalable selection system for resource allocation within open science and innovation communities

Graham, C. L. B.; Landrain, T. E.; Vjestica, A.; Greshake Tzovoras, B.; Masselot, C.; Lawton, E.; Haenel, L.; Blondel, L.; Santolini, M.

doi:10.1101/2022.04.25.489391

bioRxiv · 10.1101/2022.04.25.489391

Community review: a robust and scalable selection system for resource allocation within open science and innovation communities

Graham, C. L. B.·Landrain, T. E.·Vjestica, A.·Greshake Tzovoras, B.·Masselot, C.·Lawton, E.·Haenel, L.·Blondel, L.·Santolini, M.

View original source ↗Export metadata ↓

Abstract

Resource allocation is essential to the selection and implementation of innovative projects in science and technology. With large stakes involved in concentrating large fundings over a few promising projects, current "winner-take-all" models for grant applications are time-intensive endeavours that mobilise significant researcher time in writing extensive project proposals, and rely on the availability of a few time-saturated volunteer experts. Such processes usually carry over several months, resulting in high effective costs compared to expected benefits. Faced with the need for a rapid response to the Covid19 pandemic in 2020, we devised an agile "community review" system to allocate micro-grants for the fast prototyping of innovative solutions. Here we describe and evaluate the implementation of this community review across 147 projects from the "Just One Giant Labs OpenCOVID19 initiative" and "Helpful Engineering" open research communities. The community review process uses granular review forms and requires the participation of grant applicants in the review process. Within a year, we organised 7 rounds of review, resulting in 614 reviews from 201 reviewers, and the attribution of 48 micro-grants of up to 4,000 euros. We show that this system is fast, with a median process duration of 10 days, scalable, with a median of 4 reviewers per project independent of the total number of projects, and fair, with project rankings highly preserved after the synthetic removal of reviewers. We investigate the potential bias introduced by involving applicants in the process, and find that review scores from both applicants and non-applicants have a similar correlation of r=0.28 with other reviews within a project, matching previous observations using traditional approaches. Finally, we find that the ability of projects to apply to several rounds allows to both foster the further implementation of successful early prototypes, as well as provide a pathway to constructively improve an initially failing proposal in an agile manner. Overall, this study quantitatively highlights the benefits of a frugal, community review system acting as a due diligence for rapid and agile resource allocation in open research and innovation programs, with particular implications for decentralised communities.

Source connections

Published article ↗

Explore related subjects

scientific communication and education Preprint

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Graham, C. L. B., Landrain, T. E., Vjestica, A., Greshake Tzovoras, B., Masselot, C., Lawton, E., Haenel, L., Blondel, L., Santolini, M.. 2022-04-26. Community review: a robust and scalable selection system for resource allocation within open science and innovation communities. https://doi.org/10.1101/2022.04.25.489391

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗

bioRxiv2026

Benefits and Challenges of Integrating a Generative AI Assisted Reading Guide in an Undergraduate Journal Club Assignment

Developing scientific reading skills is critical for undergraduate STEM students due to scientific literatures unique formatting and use of specialized jargon. Generative AI tools such as ChatGPT offer students the ability to ask questions about what they are reading interactively. Previously, we reported the development of a ChatGPT-assisted reading guide that combined structured, active reading strategies with using ChatGPT to clarify unfamiliar words and concepts in real time. In the initial study, undergraduates found the use of the ChatGPT-assisted reading guide helpful in their understanding of an abstract and introduction of a journal article. Here, the ChatGPT-assisted reading guide was used in a journal club assignment for an undergraduate chemistry course. ChatGPT transcripts were analyzed for common types of interactions, and students were surveyed about their experience. Overall, students reported that using the ChatGPT-assisted reading guide was helpful in understanding the article and helped them have more productive class discussions. However, some students also expressed skepticism about using AI tools, citing concerns about accuracy of AI-generated information and the effect of using AI on their own learning.

scientific communication and education↗

bioRxiv2026

Evaluation of a One Health mass dog rabies vaccination campaign in a resource-limited urban setting: Evidence from Techiman, Ghana

BackgroundDog-mediated rabies remains endemic in many parts of sub-Saharan Africa despite the availability of effective vaccines. Achieving and sustaining at least 70% vaccination coverage in dog populations is widely recognized as essential for interrupting transmission; however, there is limited operational evidence from municipal campaigns conducted in rapidly urbanizing, resource-constrained settings. We evaluated the implementation and outcomes of a One Health-coordinated free mass dog rabies vaccination campaign conducted in September 2024 in Techiman South Metropolitan, Ghana. MethodologyA mixed-methods post-campaign assessment was undertaken two weeks after campaign completion. Quantitative data were collected from 200 dog-owning households using structured questionnaires, and administrative vaccination records were reviewed to estimate overall coverage. In addition, key informant interviews were conducted with veterinary officers, public health professionals, environmental health personnel, and community leaders involved in campaign delivery. Descriptive statistics were used to summarize household participation and awareness indicators, while qualitative data were analyzed thematically. Principal FindingsAmong surveyed households, 59.5% owned more than one dog and 51.5% reported vaccinating at least one dog during the campaign. Awareness that rabies is fatal increased from 45% (self-reported pre-campaign recall) to 75% after the campaign period, while awareness that rabies is preventable increased from 40% to 70%. Administrative records indicated that approximately 5,600 dogs were vaccinated, corresponding to an estimated 74% metropolitan-level coverage based on projected dog population figures. Stakeholders identified strong multisectoral collaboration and World Rabies Day mobilization as key facilitators. Reported challenges included vaccine supply delays, limited cold-chain capacity, inadequate access for peri-urban communities, and the absence of strategies to vaccinate free-roaming dogs. Conclusions/SignificanceThese findings suggest that short-duration, one-health mass dog vaccination campaigns can achieve target coverage thresholds in resource-limited urban contexts. However, reliance on projected denominators and exclusion of free-roaming dogs may overestimate effective population immunity. Strengthening monitoring systems, decentralizing access, and incorporating structured strategies for free-roaming dog vaccination will be critical for sustaining rabies control and advancing toward elimination goals. AUTHOR SUMMARYRabies is almost always fatal once symptoms begin, yet it can be prevented through vaccination of dogs, which are responsible for nearly all human infections in Africa. Global strategies call for vaccinating at least 70% of dogs each year to stop transmission, but there is limited evidence on how well this works in fast-growing African cities with limited resources. We evaluated a free mass dog rabies vaccination campaign carried out in Techiman, Ghana, using a One Health approach that brought together veterinary services, public health authorities, and community leaders. We interviewed 200 dog-owning households and reviewed official vaccination records. We also spoke with frontline staff involved in the campaign. Official records suggested that roughly three-quarters of the estimated dog population was vaccinated, meeting the recommended coverage threshold. Many households reported improved knowledge about rabies following the campaign. However, not all dog-owning households participated, and free-roaming dogs were not systematically vaccinated. Logistical challenges, including vaccine supply delays and limited access in peri-urban communities, were also identified. Our findings show that coordinated municipal campaigns can reach high vaccination numbers even in resource-limited settings. At the same time, accurate monitoring and inclusion of free-roaming dogs are essential to ensure that reported coverage reflects true protection. Strengthening these components will help Ghana and similar countries move closer to eliminating dog-mediated human rabies.

scientific communication and education↗