Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2024.11.11.623072

Assessing the quality of generative artificial intelligence for science communication in environmental research

Worden, D.·Richards, D.
View original source ↗Export metadata ↓

Abstract

The adoption of Generative Artificial Intelligence (GenAI) tools is drastically changing the way that researchers work. While debate on the quality of GenAI outputs continues, there is optimism that GenAI may help human experts to address the most significant environmental challenges facing society. No previous research has quantitatively assessed the quality of GenAI outputs intended to inform environmental management decisions. Here we surveyed 98 environmental scientists and used their expertise to assess the quality of human and GenAI content relevant to their discipline. We analysed the quality and relative preference between human and GenAI content across three use cases in environmental science outreach and communication. Our results indicate that the GenAI content was generally deemed adequate in quality by human experts, with an average of 82% of respondents indicating a quality of "adequate" or better across the three use cases. Respondents exhibited strong preferences for GenAI over human-only content when using GenAI imageery of future park management scenarios. For the use cases of generating a wetland planting guide and answering a question about invasive species management, preferences were heterogeneous amongst respondents. Our findings raise substantive questions about GenAI content as a complement to human expertise when research is transferred to public audiences.

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Worden, D., Richards, D.. 2024-11-13. Assessing the quality of generative artificial intelligence for science communication in environmental research. https://doi.org/10.1101/2024.11.11.623072

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Perceived Risk and Barriers to Open and Responsible Research Across Fifteen UK Universities

Open research practices are increasingly promoted to improve research transparency, reproducibility, accessibility, and societal impact. Despite growing support from research funders, institutions, and policy initiatives, adoption remains uneven across disciplines and research communities. This study examined perceived risks and barriers associated with 14 FORRT Guideline areas using qualitative responses from the UK Reproducibility Network Open and Transparent Research Practices Survey (N = 2,567), conducted across 15 UK higher education institutions. Free-text responses describing risks and barriers were analysed using inductive thematic analysis. A total of 3,951 relevant comments generated 3,433 coded references to barriers and risks. Three interconnected clusters emerged. Individual concerns included lack of motivation, fear of losing intellectual credit, and concerns about exposing mistakes and criticism. Systemic and institutional barriers included lack of time and resources, inadequate infrastructure, insufficient training and support, lack of incentives and recognition, unclear guidance, disciplinary and methodological challenges, and tensions between open practices and intellectual property requirements. Ethical and quality-related concerns included risks to participant confidentiality and privacy, challenges associated with sensitive data, concerns about inappropriate application of open research practices across different research traditions, perceived impacts on research quality and innovation, and potential effects on public trust in research. Lack of time and resources was the most frequently reported barrier. Across responses, barriers were commonly described as interdependent, with shortcomings in funding, infrastructure, institutional support, incentives, and training reinforcing one another. Respondents largely supported the principles underpinning open research but highlighted substantial practical, professional, and ethical challenges to implementation. These findings suggest that increasing open research adoption requires more than policy mandates or awareness-raising activities. Sustainable uptake will depend on aligned incentives, adequate infrastructure and support, recognition of methodological diversity, and approaches that enable openness to be implemented responsibly across different research contexts.

scientific communication and education↗
bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2026

Undergraduate Biophysical Chemistry Series: Teaching through a Combination of a Purpose-built Textbook, Research-derived Biomolecular Samples and Computer Labs

Biophysics is a rapidly advancing field with an incredible breadth of topics. Thus, undergraduate biophysics instructors have to strategize and decide what topics they will cover in their courses. Educational institutions utilize a variety of biophysics textbooks. A common deficiency of each of the existing texts is that it serves well a given set of topics (theory, illustrations, practice problems) and leaves out other areas. A typical example includes good theory and problems for thermodynamics and kinetics while presenting molecular dynamics and various spectroscopic methods in a lacking or outdated way. The authors of this manuscript teach a capstone Biophysical Chemistry three-quarter series (Western Washington University/WWU, Bellingham, WA) which ideally should resonate with the general and major-specific courses the students take within their major at WWU. To achieve this goal and to enrich the traditional lecture-based delivery, the instructors have developed and brought together key pedagogical elements: purpose-built online textbook with a uniform structure of the academic content and practice problems, a study sample (oligopeptide) of biophysical significance with a growing set of experimental and computational data and student-centric in-class activities including computer labs. Our Biophysical series emphasizes concepts and methods of computational structural biology (Molecular Dynamics) and spectroscopic approaches (IR, UV and NMR). Here we describe the details of our integrative approach, summarize key outcomes and chart ways to advance the biophysical chemistry series further. Our textbook can be found through LibreText.

scientific communication and education↗