Search bioRxivSearch

bioRxiv · 10.1101/133207

A Universal Metric For Evaluating, Optimising And Benchmarking The Performance Of A Research Technology Platform (RTP)

Abstract

Research Technology Platforms (RTPs) exist to facilitate the application and utilisation of specific analytical technologies to the highest possible standard thus delivering reputable data across a broad spectrum of research themes. Specifically, RTPs centralise expertise in a given technology and provide an unparalleled level of continuity and practical knowledge retention that simply cannot be achieved by more organic, ad hoc means of support. As small non profit businesses often tasked with recovering all or a percentage of their running costs, RTPs are under significant pressure to keep pace with rapidly advancing technology and new methodologies against a back drop of dwindling funding for scientific research. At present there are a number of non-trivial issues that make assessing the operational performance of a RTP difficult to determine on a standalone basis let alone attempting to benchmark against other RTPs within the same or different technology fields. Firstly, depending on the technological speciality the RTP may work to one of essentially three operational models. RTPs such as Bio-Imaging or Cytometry provide access to well-maintained analytical systems that can be utilised by trained individuals for a timed access charge. In some cases there will be a requirement for assisted operation of certain instruments by core staff (e.g. cell sorters). Genomics and Proteomics RTPs tend to function on a project basis whereby users will not access the technology themselves rather pay for a full analytical service often with a milestone-based approach for tracking progress. Other RTPs work to a hybrid approach were technical staff provide certain elements of sample preparation for specific projects prior to analysis on core supported, user accessible instrumentation. Secondly the specific operational costs that each RTP is tasked to recover varies significantly on a local, national and international level due to institutional subsidies. These operational costs can include staff salaries, instrument maintenance, associated running consumables, and in some cases instrument depreciation but there is standardised rule as to what each RTP is tasked to recover and to what percentage.\n\nHere we present a generalised mathematical approach to describe the customisable metrics of any given RTP serviceThe general strategy how to increase performance within the framework of this approach has been identified through breaking down these customisable metrics into components and maximising them according to specific requirements. These strategies could be potentially adopted for different operational or local procedures, integrating the specifics related to the institutional or national policies. The approach laid down here should be considered as a trigger for opening a discussion around how to address optimising RTP performance and allow for benchmarking across the full breadth of RTPs.

Explore related subjects

Keep this discovery

BibTeXRIS

Petrunkina, A. M., Filby, A.. 2017-05-02. A Universal Metric For Evaluating, Optimising And Benchmarking The Performance Of A Research Technology Platform (RTP). https://doi.org/10.1101/133207

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education

Fair allocation of healthcare research funds by the European Union?

This study aimed to investigate the distribution of European Union (EU) healthcare research grants across EU countries, and to study the effect of the potential influencing factors on grant allocation. We analysed publicly available data on healthcare research grants from the 7th Framework Programme and the Horizon 2020 Programme allocated to beneficiaries between 2007 and 2016. Grant allocation was analysed at the beneficiary-, country-, and country group-level (EU-15 versus newer Member States, defined as EU-13). The investigated country-level explanatory variables included GDP per capita, population size, overall disease burden, and healthcare research excellence. Grant amounts per 100,000 inhabitants was used as an outcome variable in the regression analyses.\n\nResearch funds were disproportionally allocated to EU-15 versus the EU-13, as 96.9% of total healthcare grants were assigned to EU-15 countries. At the beneficiary level, EU funding was positively influenced by participating in previous grants. The average grant amount per beneficiary was higher for EU-15 organizations. In univariate regression analyses at the country level, higher GDP per capita (p<0.001) and better medical research excellence (p<0.001) were associated with more EU funding, and a higher disease burden was associated with less EU funding (p=0.003). In the multiple regression analysis GDP per capita (p=0.002) and research excellence (p<0.001) had a significant positive association with EU funding. Population size had an inverted U-shaped relationship with EU funding for healthcare research, having the largest per capita funding in second and the third quartiles (p=0.03 and p=0.02).\n\nThe uneven allocation of healthcare research funds across EU countries was influenced by GDP per capita, medical research excellence and population size. Wealthier countries with an average population size and strong research excellence in healthcare had more EU funding for healthcare research. Higher disease burden apparently was not associated with more EU research funding.

scientific communication and education

Authorship inflation and author gender in pulmonology research.

IntroductionHonorary authorship and equal gender representation are two pressing matters in scientific research. Honorary authorship is the inclusion of authors who do not meet the criteria established by the International Committee of Medical Journal Editors (ICMJE) authorship guidelines. The inclusion of honorary authors in the medical literature has led to an increase of the number of authors on studies and a decrease in single author studies in various fields.\n\nMethodsOur primary objective was to assess authorship trends in two major pulmonology journals (selected on the basis of Google Scholar rankings): Thorax and American Journal of Respiratory and Critical Care Medicine. We reviewed all articles published in both journals in the years 1994, 2004, and 2014 using Web of Science and extracted data such as number of authors and gender of the first and last authors.\n\nResultsThe total number of authors steadily increased from 1994 to 2014. The median number of authors grew from about four in 1994 to nearly seven in 2014, which is approximately a 75% increase. When we compiled all the data, we found the percentage of female authors from both journals had increased from 17% to 29.9% during the study period.\n\nDiscussionWe found an increase in the average number of authors on pulmonology publications between 1994 and 2014 as well as an increase in the number of females with a lead or main author position. This may be due to a variety of factors, such as increased team science. However, our data in conjunction with data from other areas of medicine, indicate that honorary authorship may be contributing to the trends we identified.

scientific communication and education