The devil in citizen science data: observation processes invalidate the causal inference that photovoltaic policy reduces bird diversity

Abstract

Citizen science data are increasingly used to infer biodiversity change, but causal claims based on such data are credible only if sampling effort and its temporal shifts are explicitly modeled. Zhang et al. (1) used citizen science data to conclude that greater photovoltaic policy stringency, measured using the photovoltaic policy stringency index (PSI), reduced county-level bird diversity in China. We reproduced their fixed effects and instrumental variable estimates. However, the observed Shannon diversity derived from pooled citizen science records reflects both bird communities and sampling effort, which the authors' controls do not adequately capture. Accounting for observer count changed the reported statistically significant 2.10% decline in Shannon index to a nonsignificant 0.58% increase (P = 0.288) per one-standard-deviation increase in PSI, and rendered the instrumental variable estimate statistically indistinguishable from zero (P = 0.912). Yet observer count is only one of many sources of sampling bias. PSI was also associated with multiple dimensions of sampling effort, consistent with sampling effort acting as a potential mediator in the PSI-diversity chain. The sampling domain also shifted markedly from 2014 to 2023: recorded county-months increased almost 24-fold, median observer count rose from one to three, and zero-duration records declined from 57.2% to 0.17%. Without adequate adjustment, these shifts confound estimates of temporal change in observed bird diversity. Beyond its inadequate treatment of sampling effort, the original study also misinterpreted its statistical results. Although the reported R{superscript 2} values are high, they are dominated by county and year-month fixed effects, with PSI contributing a partial R{superscript 2} of only 0.048% on observed Shannon index. The PSI-photovoltaic-area correlation is also weak (r = 0.0414) and vanishes after accounting for fixed effects (P = 0.977). Furthermore, the released bird observation data contain many erroneous outliers, raising significant concerns about insufficiently rigorous data preprocessing and quality control. These results show that the released data cannot properly distinguish ecological change from sampling effort change. Robust inference from citizen science data requires checklist-level effort metadata, explicit correction for spatiotemporal sampling shifts, and close collaboration among researchers with complementary methodological and ecological expertise.