Training Spiking Networks

The spiking CNN in From ReLU to Spikes was trained with an ordinary PyTorch loop and one small trick that I postponed explaining. This post explains it: why plain backpropagation cannot see through a spike, the three ways around that, and why the most popular one, surrogate gradients with backpropagation through time, works. Then it trains a spiking network on real event-camera recordings of people making gestures, the data from Seeing in Events, and checks each claim with an experiment.

1. A Gradient That Is Zero

Training any neural network means computing $\partial L / \partial w$ for every weight with the chain rule and nudging the weights downhill. For one spiking neuron the chain is:

\frac{\partial L}{\partial w} = \frac{\partial L}{\partial S} the problem \frac{\partial S}{\partial U} \frac{\partial U}{\partial w}, S = H (U - θ)

$H$ is the Heaviside step: 0 below the threshold, 1 above. Its derivative is 0 everywhere except exactly at $U = θ$ , where it is infinite. So every gradient that has to pass through a spike is either 0 or undefined:

Seen from the loss, this means the loss is a staircase in the weights: a small change of a weight either changes nothing (flat step) or adds or removes a whole spike (a jump). Move the weight below and watch the gradient:

Two related failure modes follow, and it is worth keeping them apart (Eshraghian et al., 2023):

Dead neurons: a neuron that never fires contributes nothing to the output, so the true gradient of every weight that feeds it is 0. It stays silent forever.
Vanishing gradients: the same problem as in deep or recurrent ANNs. Gradients shrink as they are multiplied through many layers or time steps (Section 4).

2. Three Ways Around It

Convert a trained ANN (Cao et al., 2015; Diehl et al., 2015; Rueckauer et al., 2017). Train a ReLU network as usual and copy its weights into integrate-and-fire neurons, whose firing rate equals ReLU (see post 1). No spike is ever differentiated. This works very well on static images, but needs many time steps (often hundreds) for rates to settle, and ignores timing.
Differentiate spike times (Bohte et al., 2002, SpikeProp). The spike itself is a jump, but when it happens moves smoothly with the weights, so take $\partial t_{spike} / \partial w$ instead. The gradient is exact, but a neuron that does not fire has no spike time and therefore no gradient.
Surrogate gradients + BPTT (Neftci, Mostafa & Zenke, 2019). Keep the real spike in the forward pass; in the backward pass only, pretend its derivative is a smooth bump. This is the standard method today and what the rest of this post is about.

3. Surrogate Gradients

The idea is almost embarrassingly simple:

forward: S = H (U - θ), backward: \frac{\partial S}{\partial U} \approx σ^{'} (U - θ)

where $σ^{'}$ is any smooth bump centred on the threshold. The network still computes with real 0/1 spikes; only the learning signal is smoothed. Try the common shapes:

To see it work, here is the staircase neuron from Section 1 again, now trained to fire exactly a target number of times. Two learners start at the same weight: one follows the true gradient, one the surrogate gradient. Press Train:

The true-gradient learner never moves. The surrogate learner gets a useful signal from the time steps whose membrane is close to the threshold (panel 3), and walks down the staircase in a few steps. Each gradient step uses only information the network already has, the membrane values, and a bump function instead of the step.

A neuron whose membrane is near the threshold gets a useful gradient, whether or not it actually spiked; one far below threshold gets almost none. So surrogates rescue neurons that are “almost firing”, but not neurons that are deeply silent. Note also that the weights after a silent neuron still get no update, because $\partial L / \partial w_{out} = S \cdot (\dots)$ and $S = 0$ : a spike has to happen somewhere for the network to learn from it.

The trick has a history in quantised networks. Treating a hard threshold as if its derivative were 1 (the straight-through estimator) was proposed in Hinton’s 2012 lectures and named by Bengio, Léonard & Courville (2013); training binary and quantised networks relies on it (Courbariaux et al., 2016). In spiking networks, smoothed spike derivatives appeared in Bohte (2011), Esser et al. (2016), Zenke & Ganguli (2018, SuperSpike) and Shrestha & Orchard (2018, SLAYER); Neftci, Mostafa & Zenke (2019) gave the method its name.

Surrogate gradients are biased: they are not the gradient of the network actually computed. That they work so well is an empirical finding, which Section 5 tests.

4. Backpropagation Through Time

A spiking layer is a recurrent network (post 1), so it is trained like one: unroll it over the $T$ time steps and backpropagate through the whole unrolled graph.

Three practical details matter:

The shared weight collects gradient at every step. $\partial L / \partial W = \sum_{t} \partial L / \partial W [t]$ , exactly as in an RNN.
Detach the reset. The reset term $- θS [t - 1]$ also contains a spike. Letting the surrogate gradient flow through it as well tends to hurt; most implementations, including the code in this series, cut it out of the backward pass (.detach()). Section 5 tests this too.
Memory grows with $T$ . BPTT stores every step’s state. Long sequences need truncation or the online methods of Section 6.

Here is the whole procedure with real numbers for one neuron over six steps, repeated for three weight updates. Step through each forward pass, backward pass and update, and follow the weight on the loss landscape underneath as it moves into the minimum:

Along the membrane path, each step back in time multiplies the gradient by $β$ :

So an input $n$ steps before the loss is weighted by roughly $β^{n}$ : an exponentially decaying window, just like the timing window of spike-timing-dependent plasticity (STDP) measured in real synapses, where a synapse is strengthened more the shorter the delay between the two neurons’ spikes (Bi & Poo, 1998). The resemblance covers only the potentiation side of STDP; post 4 returns to STDP.

5. Experiments on Real Event Data: DVS Gesture

DVS Gesture (Amir et al., 2017) was recorded with a real 128 × 128 event camera: 29 people performing 11 gestures (hand clap, waves, arm circles in both directions, arm roll, air drums, air guitar, other) under three lighting conditions. The standard split trains on 23 people (1,077 recordings) and tests on the other 6 (264 recordings), so every test score is on people the network has never seen.

Try it yourself The whole experiment (download, preprocessing, training all three networks, confusion matrices, operation and energy counts, and switches for the surrogate experiments) is in a self-contained notebook that runs on Google Colab, in plain PyTorch. In Colab, choose Runtime → Change runtime type → GPU, then Runtime → Run all. The notebook file is also in the repository: dvs_gesture_snn.ipynb.

To keep training on a laptop CPU, I used:

Input: events summed into 4 × 4 pixel blocks (128 → 32 × 32), binned into 60 ms steps, with $T = 30$ steps (a 1.8 s window, random position during training, from 0.6 s into the recording at test time), fed as $lo g (1 + count)$ per polarity.
Network: three convolution + max-pool + LIF layers (16, 32, 64 channels) and a linear LIF output layer; $β = 0.9$ , $θ = 1$ , soft reset (detached); about 43,000 parameters.
Training: prediction = output spike counts as logits, cross-entropy, Adam with cosine decay, 30 epochs, batch 32. Every number below is the test accuracy after the last epoch, not the best epoch.

Two controls use the same layers without spikes. The first sees all 30 steps added into one image; the second runs on every 60 ms frame separately, like a camera at 16.7 frames per second, and averages its 30 answers. The three networks share every layer; only the way time enters is different:

Does the network learn, and does time matter?

	Spiking CNN (BPTT, arctan surrogate)	Same CNN on every frame, no memory	Same CNN on the time-summed event image
Test accuracy, 3 seeds	88.5% ± 0.4 (88.3, 88.3, 89.0)	85.1% ± 0.9 (84.1, 86.4, 84.8)	84.5% ± 0.5 (85.2, 84.1, 84.1)
Clockwise ↔ counter-clockwise mix-ups (of 96 test recordings)	2, 1, 2	3, 0, 3	11, 14, 15
Neurons spiking per step (conv layers 1–3)	13%, 12%, 6%	(not a spiking network)	(not a spiking network)
Training time per epoch (laptop CPU)	24 s	about 19 s	about 1 s

The spiking network learns the task well. For context, published spiking networks with full-resolution input and much larger architectures reach about 97% on this benchmark (Fang et al., 2021); 88.5% is what a 43,000-parameter network on 32 × 32 input gets on a laptop.

The control did better than I expected. A plain CNN that sees all 1.8 s summed into a single two-channel “event image” is only 4 points behind. Its mistakes are telling, though: it confuses clockwise with counter-clockwise 11 to 15 times out of 96, the spiking network only once or twice. Time matters most exactly where two classes differ only in the order of events.

The frame CNN separates the result into its two causes. It has no memory either, yet it confuses the directions only 0 to 3 times: a single 60 ms frame already shows which way the arm is moving, through the ON/OFF layout of the moving edge. What loses the direction is adding the frames up, which mixes the beginning and the end of the movement into one picture. The memory of the spiking network is worth the remaining 3.4 points (85.1% → 88.5%), and the frame CNN pays for its accuracy with 30 passes per recording; the energy of all three is counted in Counting Energy.

Why can a summed image tell the directions apart at all? Because ON and OFF events are counted in separate channels. A moving edge produces ON events on its leading side and OFF events on its trailing side (Seeing in Events), so the layout of the two polarities in the summed image still carries some direction information. A frame camera’s summed image would not have that.

Which surrogate gradient?

Each run uses the same network, data and schedule; only the function used for $\partial S / \partial U$ in the backward pass changes.

No surrogate (the true derivative): the gradient is exactly zero from the first step, and accuracy stays at chance (8.7%; chance is 1 in 11 = 9.1%). This is Section 1, measured.
Straight-through (derivative 1 everywhere): every neuron passes the full gradient at every step, however far its membrane is from threshold. Over 30 steps and 4 layers the gradients pile up (their norm grows from about 24,000 to 3,000,000) and the network fails: 13.3%.
Triangle (zero beyond one unit from threshold): learns, but slowly: 65.9% after 30 epochs.
Arctan and fast sigmoid: 88–89%. Arctan was also robust to its width: 4× wider gives 86.7%, 4× narrower 87.5%.

The reset. Letting the surrogate gradient also flow through the reset term (instead of detaching it) gave 85.2%, against 88.3–89.0% with the reset detached. That is a single seed, so take the exact gap with caution, but it agrees with the common practice of detaching the reset.

Watch it classify

Here is the trained network (seed 0) running in your browser on one test recording per gesture; the browser code reproduces PyTorch’s spike counts exactly. Ten of the eleven are classified correctly; the hand clap is not. The two CNNs run alongside (also seed 0, also reproducing PyTorch): the CNN on every frame gets ten right too, missing the air drums, and the summed-image CNN gets all eleven on this small sample. Press Play to see the events arrive in 60 ms steps, the output spike counts build up, and the frame CNN’s averaged scores change frame by frame. The 3D view below the bars shows the same recording three ways: the raw events as the camera produced them, each a point $(x, y, t, p)$ ; the 30 binned frames that the spiking CNN and the frame CNN receive one per step; and the single summed image the third network receives. Drag it to rotate; it follows the step slider:

6. Learning Without Going Back in Time

BPTT needs the whole history in memory and produces an update only after the sequence. A brain cannot do that, and neither can a small chip that is supposed to learn continuously. The alternative is to carry the necessary information forward:

For a single LIF layer, the influence of a weight on the membrane obeys its own little recursion, $m [t] = β m [t - 1] + x [t]$ , and the gradient is $\partial L [t] / \partial w = (\partial L [t] / \partial U [t]) m [t]$ . This is real-time recurrent learning (RTRL; Williams & Zipser, 1989) specialised to an LIF neuron, and $m [t]$ is an eligibility trace: a decaying memory of recent input at each synapse, waiting for a learning signal. For general recurrent networks exact RTRL is far too expensive (memory of order $n^{3}$ for $n$ neurons), so practical methods approximate it:

e-prop (Bellec et al., 2020): eligibility traces times a learning signal broadcast to the neurons; close to BPTT on several tasks, fully online.
DECOLLE (Kaiser, Mostafa & Neftci, 2020): a local loss at every layer, so errors never travel far.
OSTL (Bohnstingl et al., 2022): separates the spatial and temporal parts of the gradient.

These “three-factor” rules (presynaptic activity × postsynaptic state × a global signal) are also what learning-capable chips such as Loihi implement in hardware (post 4).

7. A Bag of Tricks

Training spiking networks is still more finicky than training CNNs. Things that help, collected from the literature and from the runs above:

Trick	Why
Detach the reset from the backward pass	avoids a second, noisy gradient path through the spike
Tune the surrogate width or the threshold	too few neurons near threshold means too little gradient
Check firing rates per layer	dead layers (no spikes) are the most common reason a model does not train; lowering the threshold is the quick fix
Learnable time constants (PLIF; Fang et al., 2021)	each layer picks its own memory length
Normalisation (batch norm, or threshold-dependent variants)	keeps membrane inputs in the range where the surrogate is non-zero
Residual connections (Fang et al., 2021, SEW-ResNet)	deep spiking networks train for the same reason ResNets do
Spike-count regularisation	trades accuracy for fewer spikes, which is energy (post 4)
Spike counts as logits, cross-entropy loss	simple and robust for classification

8. Common Claims, Checked

Claim	Verdict
”Spiking networks cannot be trained with backpropagation.”	They can, with surrogate gradients. Without one, training fails completely: on DVS Gesture the same network without a surrogate stays at 8.7% accuracy (chance is 9.1%), because its gradient is exactly zero.
”Surrogate gradients compute the true gradient of the spiking network.”	No, they are a biased approximation: the network computes with hard spikes but learns as if they were smooth. It works very well in practice, without a guarantee.
”With surrogate gradients, dead neurons are no problem.”	Partly. Neurons near threshold get gradient even when silent, but deeply silent neurons get almost none, and weights leaving a silent neuron never update, since the spike is 0.
”The choice of surrogate does not matter.”	It matters, within limits. On DVS Gesture, arctan and fast sigmoid reached 88–89%, the triangle 66%, and the straight-through estimator failed (13%) because its gradients exploded. Arctan was robust to a 4× change of width (86.7–89.0%).
”Training a spiking network costs about as much as training an ANN.”	Roughly $T$ times more compute and memory, because BPTT runs and stores every step: on DVS Gesture each epoch took about 24 s for the spiking network against about 1 s for the summed-image CNN.
”Backpropagation through time is the same as STDP.”	Only in part: the exponential $β^{n}$ decay of BPTT updates resembles the potentiation side of the STDP window. The depression side, and STDP’s purely local operation, have no counterpart.
”The straight-through estimator was coined by Hinton.”	The idea was presented in Hinton’s 2012 lectures; the name and its analysis come from Bengio, Léonard & Courville (2013).
”Gesture recognition needs no timing, the event image is enough.”	Not for gestures that differ only in direction: a CNN on the time-summed event image reached 84.5% against 88.5% for the spiking network, and confused clockwise with counter-clockwise 11–15 times out of 96 test recordings against 1–2. The same CNN on every 60 ms frame, without memory, made only 0–3 such mistakes (85.1%): each short frame still shows the direction; summing the whole window is what erases it.

Summary

Idea	In one line
The problem	the spike’s derivative is 0 or ∞, so the loss is a staircase in the weights
Three routes	convert a ReLU network, differentiate spike times, or use surrogate gradients
Surrogate gradient	real spikes forward, a smooth bump $σ^{'} (U - θ)$ backward
BPTT	unroll over $T$ steps; gradients decay by $β$ per step; detach the reset; memory grows with $T$
Online learning	eligibility traces carry the gradient forward: RTRL, e-prop, DECOLLE
On DVS Gesture	88.5% on unseen people with a small network, against 85.1% for the same CNN on every frame and 84.5% on the summed image; without a surrogate it does not learn at all; summing over time is what confuses rotation directions

Next: Counting Energy: what a spike actually costs, measured with NeuroBench on this network, and why the first layer and the number of steps dominate.

References

Training methods

Bohte, S. M., Kok, J. N., & La Poutré, H. (2002). Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing, 48, 17–37.
Bohte, S. M. (2011). Error-backpropagation in networks of fractionally predictive spiking neurons. ICANN.
Esser, S. K., Merolla, P. A., Arthur, J. V., et al. (2016). Convolutional networks for fast, energy-efficient neuromorphic computing. PNAS, 113(41), 11441–11446.
Zenke, F., & Ganguli, S. (2018). SuperSpike: Supervised learning in multilayer spiking neural networks. Neural Computation, 30(6), 1514–1541.
Shrestha, S. B., & Orchard, G. (2018). SLAYER: Spike layer error reassignment in time. NeurIPS.
Neftci, E. O., Mostafa, H., & Zenke, F. (2019). Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine, 36(6), 51–63.
Eshraghian, J. K., Ward, M., Neftci, E. O., et al. (2023). Training spiking neural networks using lessons from deep learning. Proceedings of the IEEE, 111(9), 1016–1054.
Fang, W., Yu, Z., Chen, Y., Huang, T., Masquelier, T., & Tian, Y. (2021). Deep residual learning in spiking neural networks. NeurIPS.
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., & Tian, Y. (2021). Incorporating learnable membrane time constant to enhance learning of spiking neural networks. ICCV.

Conversion

Cao, Y., Chen, Y., & Khosla, D. (2015). Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 113, 54–66.
Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S.-C., & Pfeiffer, M. (2015). Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. IJCNN.
Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M., & Liu, S.-C. (2017). Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience, 11, 682.

Straight-through estimator and quantised networks

Bengio, Y., Léonard, N., & Courville, A. (2013). Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv:1308.3432.
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016). Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or −1. arXiv:1602.02830.

Online learning

Williams, R. J., & Zipser, D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2), 270–280.
Bellec, G., Scherr, F., Subramoney, A., et al. (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11, 3625.
Kaiser, J., Mostafa, H., & Neftci, E. (2020). Synaptic plasticity dynamics for deep continuous local learning (DECOLLE). Frontiers in Neuroscience, 14, 424.
Bohnstingl, T., Woźniak, S., Pantazi, A., & Eleftheriou, E. (2022). Online spatio-temporal learning in deep neural networks. IEEE Transactions on Neural Networks and Learning Systems.

Data and biology

Amir, A., Taba, B., Berg, D., et al. (2017). A low power, fully event-based gesture recognition system. CVPR.
Bi, G.-Q., & Poo, M.-M. (1998). Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18(24), 10464–10472.

Notes

Explorer