Search bioRxiv⌕ Search

Biology subjects

Marion, M.

Publications and source records attributed to Marion, M..

3 recordsLinked to original sources

LifeSciBench: Evaluating Language Models on Realistic, Expert-Level Tasks in the Life Sciences

We introduce LifeSciBench, a benchmark of 750 expert-authored tasks designed to evaluate whether language models can handle realistic life science research work. The majority of existing life sciences benchmarks have a narrow scope or are purely knowledge-based, and therefore fail to capture the complexity of real-world research, which often involves ambiguities and requires the accurate execution of multiple dependent judgment calls. Additionally, almost all existing benchmarks span at best a small collection of subdomains within the life sciences; there is at present no existing life sciences benchmark with both the requisite breadth and depth required to convincingly measure proficiency in real-world professional research settings. LifeSciBench addresses this gap by spanning seven representative scientific workflows and seven life science domains, with each constituent task paired with a human expert-written rubric. Across five frontier and domain-specialized models, GPT-Rosalind performs best, with a task-weighted mean normalized rubric score of 0.576 and a task-weighted response pass rate of 36.1% (response-level values are first averaged within each task, and the resulting task-level values are then averaged with equal weight). LifeSciBench remains unsaturated, with 171 tasks (22.8%) having no observed passing response from any evaluated model and 261 tasks (34.8%) having a best-model pass rate below 20%. LifeSciBench therefore serves as a high-resolution evaluation of practical scientific reasoning and operational decision-making in the life sciences.

bioinformatics↗

Repertoire-wide contextual mapping reveals signal functions in cooperatively breeding crows

Communication structures society, and is likewise shaped by relationships and shared tasks; yet, for most socially complex species, we know little of their full vocal repertoire and its functions. We investigated how communication structures the who, what, and when of social interactions in cooperative carrion crows - group-living birds who rely on coordinated behaviors, as in chick care. Leveraging machine learning to integrate large-scale data from crow-borne audio-loggers and nest cameras, we charted the vocal repertoire across 24 cooperative groups and mapped all discovered call types to behaviors and social context. We found that crows used a rich repertoire across three domains of joint behavior - flocking, chick care, and territorial display. Relatively quiet call types were abundant and included close-range calls that may coordinate chick care by announcing nest visits. Our study demonstrates how combining continuous-capture data and machine learning can reveal a holistic understanding of how vocalizations function across contexts.

animal behavior and cognition↗

Identification of a regulatory pathway governing TRAF1 via an arthritis-associated non-coding variant

TRAF1/C5 was among the first loci shown to confer risk for inflammatory arthritis in the absence of an associated coding variant, but its genetic mechanism remains undefined. Using ImmunoChip data from 3,939 juvenile idiopathic arthritis (JIA) patients and 14,412 controls, we identified 132 plausible common non-coding variants, reduced serially by SNP-seq, electrophoretic mobility shift, and luciferase studies to the single variant rs7034653 in the third intron of TRAF1. Genetically manipulated experimental cells and primary monocytes from genotyped donors establish that the risk G allele reduces binding of Fos-Related Antigen 2 (FRA2), resulting in reduced TRAF1 expression and enhanced TNF production. Conditioning on this variant eliminates attributable risk for rheumatoid arthritis, implicating a mechanism shared across the arthritis spectrum. These findings reveal that rs7034653, FRA2, and TRAF1 mediate a pathway through which a non-coding causal variant drives risk of inflammatory arthritis in children and adults.

genetics↗