Search bioRxiv⌕ Search

bioRxiv · 10.1101/2021.07.26.453749

Information silos distort biomedical research

Abstract

Information silos have been an oft-maligned feature of scientific research for introducing a bias towards knowledge that is produced within a scientists own community. The vastness of the scientific literature has been commonly blamed for this phenomenon, despite recent improvements in information retrieval and text mining. Its actual negative impact on scientific progress, however, has never been quantified. This analysis attempts to do so by exploring its effects on biomedical discovery, particularly in the discovery of relations between diseases, genes and chemical compounds. Results indicate that the probability that two scientific facts will enable the discovery of a new fact depends on how far apart these two facts were published within the scientific landscape. In particular, the probability decreases exponentially with the citation distance. Thus, the direction of scientific progress is distorted based on the location in which each scientific fact is published, representing a path-dependent bias in which originally closely-located discoveries drive the sequence of future discoveries. To counter this bias, scientists should open the scope of their scientific work with modern computational approaches.

Source connections

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Rodriguez-Esteban, R.. 2021-07-28. Information silos distort biomedical research. https://doi.org/10.1101/2021.07.26.453749

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗

Virtually the same? Evaluating the effectiveness of remote undergraduate research experiences

In-person undergraduate research experiences (UREs) promote students integration into careers in life science research. In 2020, the COVID-19 pandemic prompted institutions hosting summer URE programs to offer them remotely, raising questions about whether undergraduates who participate in remote research can experience scientific integration. To address this, we investigated indicators of scientific integration for students who participated in remote life science URE programs in summer 2020. We found that these students experienced gains in their scientific self-efficacy and scientific identity similar to results reported for in-person UREs. We also found that these students perceived high benefits and low costs of doing research at the outset of their programs, and their perceptions did not change despite the remote circumstances. Yet, their perceptions differed by program, indicating that programs differentially affected students perceptions of the costs of doing research. Finally, we observed that students with prior research experience made greater gains in self-efficacy and identity, as well as in their perceptions of the alignment of their values with those of the scientific community, in comparison to students with no prior research experience. This finding suggests that additional programming may be needed for undergraduates with no prior experience to benefit from remote research.

scientific communication and education↗

The role of collegiality in academic review, promotion, and tenure

Review, promotion, and tenure (RPT) processes at universities typically assess candidates along three dimensions: research, teaching, and service. In recent years, some have argued for the inclusion of a controversial fourth criterion: collegiality. While collegiality plays a role in the morale and effectiveness of academic departments, it is amorphic and difficult to assess, and could be misused to stifle dissent or enforce homogeneity. Despite this, some institutions have opted to include this additional element in their RPT documents and processes, but it is unknown the extent of this practice and how it varies across institution type and disciplinary units. This study is based on two sets of data: survey data collected as part of a project that explored the publishing decisions of faculty and how these related to perceived importance in RPT processes, and 864 RPT documents collected from 129 universities from the United States and Canada. We analysed these RPT documents to determine the degree to which collegiality and related terms are mentioned, if they are defined, and if and how they may be assessed during the RPT process. Results show that when collegiality and related terms appear in these documents they are most often just briefly mentioned. It is less common for collegiality and related terms to be defined or assessed in RPT documents. Although the terms are mentioned across all types of institutions, there is a statistically significant difference in how prevalent they are at each. Collegiality is more commonly mentioned in the documents of doctoral research-focused universities (60%), than of masters universities and colleges (31%) or baccalaureate colleges (15%). Results from the accompanying survey of faculty also support this finding: individuals from R-Types were more likely to perceive collegiality to be a factor in their RPT processes. We conclude that collegiality likely plays an important role in RPT processes, whether it is explicitly acknowledged in policies and guidelines or not, and point to several strategies in how it might be best incorporated in the assessment of academic careers.

scientific communication and education↗