Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2023.05.10.540139

TERSE: Efficient compression of the diffraction data

Matinyan, S.·Abrahams, J. P.
View original source ↗Export metadata ↓

Abstract

High-throughput data collection in crystallography poses significant challenges in handling massive amounts of data. Here, we present TERSE, a novel lossless compression algorithm specifically designed for diffraction data. We compare TERSE with the established lossless compression algorithms implemented in gzip, CBF, and HDF5, in terms of compression efficiency and speed, using continuous rotation electron diffraction data of an inorganic compound. Our results show that TERSE outperforms these algorithms by achieving a higher data compression at a speed that is at least an order of magnitude faster. TERSE files are byte-order independent and the algorithm can be readily implemented in hardware. By providing a tailored solution for diffraction data, TERSE facilitates more efficient data analysis and interpretation while mitigating storage and transmission concerns. TERSE C++20 compression/decompression code and an ImageJ/Fiji java plugin for reading TERSE files are open-sourced on GitHub under the permissive MIT license. SynopsisWe present a fast and lossless algorithm for compressing diffraction data, achieving up to 85% reduction in file size while processing up to 2000 512x512 frames per second. This breakthrough in compression technology is a significant step towards more efficient analysis and storage of large diffraction datasets.

Source connections

Explore related subjects

biophysicsPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Matinyan, S., Abrahams, J. P.. 2023-05-12. TERSE: Efficient compression of the diffraction data. https://doi.org/10.1101/2023.05.10.540139

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Design of colipids with controlled lateral distance of head group moieties for targeting strategies in the field of lipid-based nanoparticles

Lipid nanoparticles and liposomes are nanoparticle drug formulations which are firmly established in clinics. In order to expand their application and improve efficacy, surface modifications can be a key strategy, especially for cell specific targeting. Targeting can be achieved by integrating ligand functionalized lipids or lipidoids on the nanoparticle surface. In this case, not solely the targeting ligand itself, but also the structure of the whole targeting lipid/lipidoid can affect the interaction of the targeting ligand with the binding motive on cells. Especially, distal spacer structures between lipid anchor and ligand unit, often composed of polyethylene glycol chains, provide a certain distance of the ligand from the nanoparticle surface. A concept which is currently neglected is the control of the lateral distance between ligand structures on the nanoparticle. Herein we investigated the possibility to design lipidoids with a lateral spacer composed of polyethylene glycol units. We synthesized and characterized different lipidoids which provide a primary amine in the head group for possible ligand coupling. In our model lipidoids the head groups were functionalized with a paramagnetic probe to apply electron paramagnetic resonance (EPR) methods for the characterization of the lipidoid interaction with model nanoparticles, in our case liposomes. Continuous wave EPR was used to study the efficiency of the novel lipidoids to integrate into lipid bilayers of model liposomes, which was shown to depend on lipidoid structure and liposome composition. Furthermore, using double electron-electron resonance (DEER), we observed lipidoid conformational changes occurring upon integration into liposomes.

biophysics↗
bioRxiv2026

Beyond Sequence and Structure: Pretraining PLMs on Continuous Biophysical Fields

We introduce the Biophysical Transformer, a protein language model that adds continuous biophysical fields as a third pretraining signal beside sequence and structure. A Poisson-Boltzmann solver computes the electrostatic potential around each three-residue token, and the model learns to predict it from sequence alone. A rotationally invariant spherical-harmonic descriptor serves as the loss target, eliminating the need for frame alignment during training and for coordinates at inference. Against a matched, one-flag control on the FLIP Meltome thermostability benchmark, the field objective raises Spearman correlation on every trained head of both human splits.The gain on human survives residualizing dipeptide composition under two architecturally different readouts and survives removing the test proteins that overlap the pretraining corpus. No effect survives on the organism-mixed split. At 1.1M parameters, both the Biophysical Transformer and its sequence-only control exceed FLIP's one-hot CNN and untrained-ESM baselines on every split. All results rest on a single pretraining run per arm,and the current design does not separate field content from generic regularization.

biophysics↗
bioRxiv2026

An auditable harness for biomolecular dynamics simulations by AI agents

AI agents now generate biological hypotheses faster than they can be tested. Hypotheses stated at the atomic level can be examined with molecular dynamics (MD) simulations. For new questions without a reference, trust in an agent-run simulation rests on a tool-written process record, not the agent's account. Existing MD agents leave this record to the language model. Here we present MDClaw, a harness whose validating tools and directed acyclic graph (DAG) check stages before execution and record every simulation stage. On MDDataBench, which grades outputs against deposited simulations, four open-weight models with MDClaw passed all three runs on 81-100% of 31 test systems. The DAG complements this outcome check. An ablation without the DAG revealed 11 successful runs that had skipped minimization or equilibration, or simulated a different structure. MDClaw carried questions from human and AI scientists through simulation to trajectories traceable to their inputs and settings.

biophysics↗