Reward-Based Equilibrium Propagation

Abstract

Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation for training neural networks. EP typically relies on free and nudged dynamical phases: during the free phase, the network relaxes toward an equilibrium state, whereas during nudging, the output state is perturbed toward a target using a teaching signal. However, it remains unclear whether the brain has access to such explicit target signals. Inspired by Attention-Gated Brain Propagation (BrainProp), a reward-based learning framework proposed by Pozzi et al. (2020), we introduce a reward-based variant of EP that replaces full target-based nudging with a selected-output binary reward signal. The proposed method updates the network using only the chosen class and whether that choice is correct, without directly revealing the full target vector. We evaluate the method on MNIST, Fashion-MNIST, and CIFAR-10 using both multilayer perceptrons and convolutional neural networks. The proposed reward-based EP achieves performance close to that of conventional EP across all three datasets, although it generally converges more slowly during the early stages of training. Generalization-gap analyses show similar behavior for the two methods on MNIST and Fashion-MNIST, while reward-based EP exhibits a smaller training-test accuracy gap during later training on CIFAR-10. We further investigate the effect of the exploration probability used during stochastic class selection and find that moderate exploration can provide small performance improvements, although its effect is dataset-dependent. These results demonstrate that EP can learn effectively from sparse, action-specific reward feedback rather than a complete supervised target.