Learning using switching synaptic plasticity rules

Abstract

Synapses differ in molecular and ultrastructural state, but the computational role of this heterogeneity remains unclear and computational models do not account for it. Local Hebbian-like plasticity is well supported experimentally, whereas learning complex behaviors most likely requires non-local credit-assignment signals. Recent cortical electron microscopy data suggest that strong and weak synapses occupy distinct structural states, with large synapses often containing a spine apparatus that can alter calcium dynamics and plasticity. Here, we assigned different computational roles to different morphological synaptic types: we asked how networks with state-dependent plasticity operate and whether the mechanisms and structural features they develop are consistent with biological observations. We built a plasticity-switching recurrent neural network (psRNN) in which synaptic state is modeled by recurrent-weight thresholds. Synapses in the low-weight state use a local Hebbian-like rule, whereas synapses in the high-weight state receive an idealized non-local credit-assignment signal implemented with backpropagation through time. psRNNs learned cognitive tasks with temporal and working-memory demands in fewer trials than fully backpropagation-trained RNNs, despite routing BP-based updates to fewer recurrent weights. The advantage arose from three interacting mechanisms: 1) Hebbian-like updates exposed the credit-assignment component to several nearby parameter configurations, 2) the network dynamically built a task-relevant recurrent initialization, and 3) synapses switched into the credit-assignment subset before Hebbian growth destabilized learning. Structurally, trained psRNNs produced more anti-symmetric weights, lower effective rank, and stronger feedforward structure than control RNNs. Our results shed light on the computational role of heterogeneous synapses, which may regulate distinct types of learning signals, and we generate falsifiable predictions for connectomic analyses of recurrent cortical circuits.