Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.64898/2026.09.19.752880

Scrub Data: A Framework for Reproducible Data Curation with AI Coding Agents

Kay, J.·Bar, S.·Beery, S.
View original source ↗Export metadata ↓

Abstract

Data curation-the process of collecting, cleaning, and joining raw datasets for unified analysis-is a time-consuming yet crucial part of any data science project. With the emergence of powerful agentic coding tools, it is tempting to "vibe curate"-i.e., to prompt AI agents to perform data curation operations and blindly trust their outputs in order to finish work more quickly. However, this can exacerbate two key challenges that already plague manual data curation workflows: reproducibility-the ability to trace the exact set of modifications applied to raw datasets-and verifiability-the ability to audit changes and confirm that data is processed properly. To address these issues, we introduce Scrub Data, a framework that enables the use of AI agents in data curation workflows while transparently maintaining both reproducibility and verifiability. The framework involves an interactive data curation loop between a user and an AI agent, backed by a data provenance graph that tracks and versions all dataset updates. The data graph requires that all updates are formatted as self-contained executable steps, allowing any version of a dataset to be reconstructed from raw data by playing forward the transformations stored in the graph. This workflow takes place within a lightweight web application that is designed to be modified by users and agents in real-time to create custom visualization tools for identifying data curation needs and verifying outcomes. We demonstrate the usefulness of the framework with a real data curation use case from animal movement ecology. Starting with 89M raw GPS coordinates, we use the framework to curate a benchmark dataset of 2.5M coordinates to be used for machine learning or statistical analysis. The framework is available for use and extension at https://github.com/justinkay/scrubdata.

Explore related subjects

ecologyPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kay, J., Bar, S., Beery, S.. 2026-09-23. Scrub Data: A Framework for Reproducible Data Curation with AI Coding Agents. https://doi.org/10.64898/2026.09.19.752880

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Phenotypic diversity acts as a higher-order functional trait of populations

Functional traits are usually assigned to individuals, yet some properties exist only as differences among population members. Whether such higher-order properties predict collective performance remains unclear. Here we used the aquatic plant Spirodela polyrhiza to test whether phenotypic and genetic differences among genotypes predict non-additive population growth. We grew twelve Japanese strains alone and in all pairwise mixtures and quantified their growth dynamics. Mixtures sometimes outperformed the mean of their component monocultures, and some exceeded the better-performing monoculture, although diversity effects were negative on average. Positive effects were most strongly associated with between-strain differences in frond colour and morphology, identifying specific dimensions of phenotypic diversity as population-level predictors of synergistic growth. Genome-wide higher-level association mapping identified loci where between-strain diversity was associated with growth effects, implicating stress responses, secondary metabolism and frond morphology. Loci associated with reduced performance implicated chemical perception and self/non-self recognition, whereas those associated with positive effects reflected functional differentiation in stress responses and physiological niches. Our findings suggest that some traits generate new functions at the population level through variation among individuals, making diversity itself a higher-order functional property.

ecology↗
bioRxiv2026

Developmental cold exposure increases the ability to maintain body temperature during future cold challenges in a free-living altricial bird

Developing in suboptimal temperatures can have widespread negative effects in organisms, but early exposure to thermal challenges can also trigger increased investment in thermoregulatory capacity. Within birds, the effects of cold exposure during incubation may differ between precocial and altricial species due to differences in the ontogeny of thermoregulation. To date, most of the evidence showing that embryonic cold exposure shapes future thermoregulatory capacity comes from a few precocial species in laboratory settings. Less is known about sensitivity to low-temperature incubation in altricial species which only become fully endothermic after hatching. In this study, we tested the effects of cold temperatures during embryonic development by temporarily cooling the nest boxes of free-living tree swallows (Tachycineta bicolor) during late incubation and subsequently exposing nestlings to an acute cold challenge. Cold exposure during early development made nestlings better at maintaining their body temperatures during the acute challenge. Developmentally cold-exposed birds also had higher thyroxine (T4) levels when sampled at lower ambient temperatures, whereas control birds showed the opposite relationship. Among nestlings that failed to maintain their body temperature during the acute cold challenge, developmentally cold-exposed nestlings secreted more corticosterone during the acute cold challenge than control nestlings. Despite these changes, cold-exposed birds did not differ from controls in cold-induced metabolic rates, body mass, or bacteria killing ability. These results are, to our knowledge, the first to show that developmental cold exposure increases the capacity to maintain body temperature during future cold challenges in a songbird. These findings suggest that experiencing even a mild decline in ambient temperatures during embryonic development could prime birds to cope more effectively with harsh or variable environmental conditions later in life.

ecology↗
bioRxiv2026

Winter-run Chinook salmon juvenile recruitment and early life history response to flow in a heavily altered tailwater

Diverse life history strategies allow species to spread risk across a mosaic of habitat conditions. In Chinook Salmon (Oncorhynchus tshawytscha), particularly in the Central Valley of California, this is expressed through varied adult run timings and juvenile outmigration strategies. Anthropogenic impacts, particularly dams, disrupt these fundamental life stage transitions, often forcing managed populations to adapt to homogenized, less stochastic hydrologic regimes. This is exemplified in the Sacramento River, where the conservation of endangered Sacramento River winter run Chinook salmon requires balancing complex water operations with the ecological needs of a population confined to a short stretch of suitable habitat downstream of Shasta Reservoir. Using a 23 year dataset (2002 to 2024), we evaluated the combined effects of flow, temperature, and spawner abundance on juvenile production and life history expression in this habitat. We found that flow and spawner abundance best predicted juvenile abundance at Red Bluff Diversion Dam while temperature had considerably less support. The proportion of juveniles migrating as smolts exhibited significant density dependence that was negatively correlated to both female spawner abundance and peak flows. These results suggest focusing on temperature management alone may be insufficient. Effective management strategies should consider flow variability and habitat restoration to facilitate varying migration strategies and expand upstream rearing capacity. By addressing these physical and hydrologic constraints, managers can better support the full suite of life history strategies necessary for the resilience of winter run Chinook salmon.

ecology↗