Spike: A GPU Optimised Spiking Neural Network Simulator
Spiking Neural Network (SNN) simulations require internal variables - such as the membrane voltages of individual neurons and their synaptic inputs - to be updated on a sub-millisecond resolution. As a result, a single second of simulation time requires many thousands of update calculations per neuron. Furthermore, increases in the scale of SNN models have, accordingly, led to manyfold increases in the runtime of SNN simulations. Existing solutions to this problem of scale include high performance CPU based simulators capable of multithreaded execution (\"CPU parallelism\"). More recent GPU based simulators have emerged, which aim to utilise GPU parallelism for SNN execution. We have identified several key speedups, which give GPU based simulators up to an order of magnitude performance increase over CPU based simulators on several benchmarks. We present the Spike simulator with three key optimisations: timestep grouping, active synapse grouping, and delay insensitivity. Combined, these optimisations massively increase the speed of executing a SNN simulation and produce a simulator which is, on a single machine, faster than currently available simulators.