PathEdit: Diagnostic-Preserving Counterfactual Medical Image Editing through State-Factorized Generative Intervention

Abstract

Counterfactual medical images are useful only when a requested clinical change is introduced without silently altering patient characteristics that should remain fixed. Existing diffusion editors can produce convincing images, but realism or target-classifier flips do not establish that an edit is localized, diagnosis-specific, anatomically faithful, or useful on real clinical data. We introduce PATHEDIT, a state-factorized counterfactual editing framework that decomposes an abdominal CT representation into pathology, anatomy, acquisition, and residual context before performing a clinically specified latent intervention. A pathology editor modifies only the target factor, while non-target factors are explicitly copied and preservation losses penalize collateral drift. The edited state is decoded jointly into an image, a report level description, and diagnostic readouts. Weak supervision from lesion masks, organ segmentations, and naturally occurring longitudinal lesion changes anchors edit direction and spatial support without requiring pixel-aligned counterfactual ground truth. We evaluate four properties that are often conflated in medical image editing: target edit success, non-target preservation, anatomical identity, and real-data utility. In our evaluation protocol, PATHEDIT achieves 94.2% target success while reducing mean off-target diagnostic change to 3.0%, yields substantially stronger alignment with observed lesion changes, and improves rare-lesion recognition and shortcut robustness on held-out real CT. These results motivate astricter view of counterfactual medical imaging: a useful edit must prove not only what changed, but also what did not.