Biology subjects

Pawelzik, K. R.

Publications and source records attributed to Pawelzik, K. R..

6 recordsLinked to original sources

bioRxiv2023

Competitive performance and superior noise robustness of a non-negative deep convolutional spiking network

Networks of spiking neurons promise to combine energy efficiency with high performance. However, spiking models that match the performance of current state-of-the-art networks while requiring moderate computational resources are still lacking. Here we present an alternative framework to deep convolutional networks (CNNs), the "Spike by Spike" network (SbS), together with an efficient backpropagation algorithm. SbS implements networks based on non-negative matrix factorisation (NNMF), but uses discrete events as signals instead of real values. On clean data, the performance of CNNs is matched by both NNMF-based networks and SbS. SbS are found to be most robust when the data is corrupted by noise, specially when this noise was not seen before.

neuroscience↗

bioRxiv2021

Synaptic Self-Organization of Spatio-Temporal Pattern Selectivity

Spiking model neurons can be set up to respond selectively to specific spatio-temporal spike patterns by optimization of their input weights. It is unknown, however, if existing synaptic plasticity mechanisms can achieve this temporal mode of neuronal coding and computation. Here it is shown that changes of synaptic efficacies which tend to balance excitatory and inhibitory synaptic inputs can make neurons sensitive to particular input spike patterns. Simulations demonstrate that a combination of Hebbian mechanisms, hetero-synaptic plasticity and synaptic scaling is sufficient for self-organizing sensitivity for spatio-temporal spike patterns that repeat in the input. In networks inclusion of hetero-synaptic plasticity leads to specialization and faithful representation of pattern sequences by a group of target neurons. Pattern detection is robust against a range of distortions and noise. The proposed combination of Hebbian mechanisms, hetero-synaptic plasticity and synaptic scaling is found to protect the memories for specific patterns from being overwritten by ongoing learning during extended periods when the patterns are not present. This suggests a novel explanation for the long term robustness of memory traces despite ongoing activity with substantial synaptic plasticity. Taken together, our result promote the plausibility of precise temporal coding in the brain. Author summaryNeurons communicate using action potentials, that are pulses localized in time. There is evidence that the exact timing of these so called spikes carries information. The hypothesis, however, that computations in brains are indeed based on precise patterns of spikes is debated, particularly because this would require the existence of suitable detectors. While theoretically, individual neurons can perform spike pattern detection when their input synapses are carefully adjusted, it is not known if existing synaptic plasticity mechanisms indeed support this coding principle. Here, a combination of basic but realistic mechanisms is demonstrated to self-organize the synaptic input efficacies such that individual neurons become detectors of patterns repeating in the input. The proposed combination of learning mechanisms yields a balance of excitation and inhibition similar to observations in cortex, robustness of detection against perturbations and noise, and persistence of memory against plasticity during ongoing activity without the learned patterns. The proposed learning mechanism enables groups of neurons to incrementally acquire sets of patterns thereby faithfully representing their which and when in sequences. These results suggest that computations based on spatio-temporal spike patterns might emerge without any supervision from the synaptic plasticity mechanisms present in the brain.

neuroscience↗

bioRxiv2019

Biologically plausible learning in a deep recurrent spiking network

Artificial deep convolutional networks (DCNs) meanwhile beat even human performance in challenging tasks. Recently DCNs were shown to also predict real neuronal responses. Their relevance for understanding the neuronal networks in the brain, however, remains questionable. In contrast to the unidirectional architecture of DCNs neurons in cortex are recurrently connected and exchange signals by short pulses, the action potentials. Furthermore, learning in the brain is based on local synaptic mechanisms, in stark contrast to the global optimization methods used in technical deep networks. What is missing is a similarly powerful approach with spiking neurons that employs local synaptic learning mechanisms for optimizing global network performance. Here, we present a framework consisting of mutually coupled local circuits of spiking neurons. The dynamics of the circuits is derived from first principles to optimally encode their respective inputs. From the same global objective function a local learning rule is derived that corresponds to spike-timing dependent plasticity of the excitatory inter-circuit synapses. For deep networks built from these circuits self-organization is based on the ensemble of inputs while for supervised learning the desired outputs are applied in parallel as additional inputs to output layers.\n\nGenerality of the approach is shown with Boolean functions and its functionality is demonstrated with an image classification task, where networks of spiking neurons approach the performance of their artificial cousins. Since the local circuits operate independently and in parallel, the novel framework not only meets a fundamental property of the brain but also allows for the construction of special hardware. We expect that this will in future enable investigations of very large network architectures far beyond current DCNs, including also large scale models of cortex where areas consisting of many local circuits form a complex cyclic network.

neuroscience↗

bioRxiv2019

Back-propagation learning in deep Spike-By-Spike networks

Neural networks are important building blocks in technical applications. These artificial neural networks (ANNs) rely on noiseless continuous signals in stark contrast to the discrete action potentials stochastically exchanged among the neurons in real brains. A promising approach towards bridging this gap are the Spike-by-Spike (SbS) networks which represent a compromise between non-spiking and spiking versions of generative models that perform inference on their inputs. What is still missing are algorithms for finding weight sets that would optimize the output performances of deep SbS networks with many layers.\n\nHere, a learning rule for hierarchically organized SbS networks is derived. The properties of this approach are investigated and its functionality demonstrated by simulations. In particular, a Deep Convolutional SbS network for classifying handwritten digits (MNIST) is presented. When applied together with an optimizer this learning method achieves a classification performance of roughly 99.3% on the MNIST test data. Thereby it approaches the benchmark results of ANNs without extensive parameter optimization. We envision that with this learning rule SBS networks will provide a new basis for research in neuroscience and for technical applications, especially when they become implemented on specialized computational hardware.

bioinformatics↗

bioRxiv2018

Massively Parallel FPGA Hardware for Spike-By-Spike Networks

While inspired by the brain, currently successful artificial neural networks lack key features of the biological original. In particular, the deep convolutional networks (DCNs) neither use pulses as signals exchanged among neurons, nor do they include recurrent connections which are both core properties of real neuronal networks. This not only puts to question the relevance of DCNs for explaining information processing in nervous systems but also limits their potential for modeling natural intelligence.\n\nSpike-By-Spike (SbS) networks are a promising new approach that combines the computational power of artificial networks with biological realism. Instead of separate neurons they consist of neuronal populations performing inference. Even though the underlying equations are rather simple implementations of such networks on currently available hardware are several orders of magnitude slower than for comparable non-spiking deep networks.\n\nHere, we develop and investigate a framework for SbS networks on chip. Thanks to the communication via spikes, already moderately sized deep networks based on the SbS approach allows a parallelization into thousands of simple and fully independent computational cores. We demonstrate the feasibility of our design on a Xilinx Virtex 6 FPGA while avoiding proprietary cores (except block memory) that cannot be realized on a custom-designed ASIC. We present memory access optimized circuits for updating the internal variables of the neurons based on incoming spikes as well as for learning the connections strength. The optimized computational circuits as well as the representation of variables fully exploit the non-negative properties of all data in the SbS approach. We compare the sizes of the arising circuits for floating and fixed point numbers. In addition we show how to minimize the number of components that are required for the computational cores by reusing their components for different functions.

bioinformatics↗

bioRxiv2017

Open Hardware For Neuro-Prosthesis Research: A Study About A Closed-Loop Multi-Channel System For Electrical Surface Stimulations And Measurements

1Recent progress in neuro-prosthetic technology gives rise to the hope that in the future blind people might regain some degree of visual perception. It was shown that electrically stimulating the brain can be used to produce simple visual impressions of light blobs (phosphenes). However, this perception is very far away from natural sight. For developing the next generation of visual prostheses, real-time closed-loop stimulators which measure the actual neuronal activities and on this basis determine the required stimulation pattern. This leads to the challenge to design a system that can produce arbitrary stimulation-patterns with up to {+/-}70V and with up to 25mA while measuring neuronal signals with amplitudes in the order of mV. Furthermore, the interruption of the measurement by stimulation must be as short as possible and the system needs to scale to hundreds of electrodes. We discuss how such a system and especially its current pumps and input protection need to be designed and which problems arise. We condense our findings into an example design for which we provide all design files (boards, firmwares and software) as open-source. This is a first step in taking the existing open-source www.open-ephys.org recording system and converting it into a closed-loop experimental setup for neuro-prosthetic research.

neuroscience↗