Temporal Image Sandwiches Enable Link between Functional Data Analysis and Deep Learning for Single-Plant Cotton Senescence

Abstract

Senescence is a highly ordered degenerative biological process that affects yield and quality in annuals and perennials. Images from 14 unoccupied aerial system (UAS, UAV, drone) flights captured the senescence window across two experiments while functional principal component analysis (FPCA) effectively reduced the dimensionality of temporal visual senescence ratings (VSRs) and two vegetation indices: RCC and TNDGR. Convolutional neural networks (CNNs) trained on temporally concatenated, or "sandwiched," UAS images of individual cotton plants (Gossypium hirsutum L.), allowed single-plant analysis (SPA). The first functional principal component scores (FPC1) served as the regression target across six CNN models (M1-M6). Model performance was strongest for FPC1 scores from VSR (R2 = 0.857 and 0.886 for M1 and M4), strong for TNDGR (R2 = 0.743 and 0.745 for M3 and M6), and strong-to- moderate for RCC (R2 = 0.619 and 0.435 for M2 and M5), with deep learning attention of each model confirmed by activation of plant pixels within saliency maps. Single-plant UAS image analysis across time enabled translatable implementations of high-throughput phenotyping by linking deep learning with functional data analysis (FDA). This has applications for fundamental plant biology, monitoring orchards or other spaced plantings, plant breeding, and genetic research.