These two animations show the difference in network dynamics when the global initialization is changed from 0 (standard initialization) to 0.5 (high-entropy initialization).

Each circle represents a neuron, with layer index increasing to the right. Red neurons are those which are currently firing. The blue-ringed neuron represents the ground-truth class, and the green neuron represents the current predicted class.

With a standard initialization, the network spends several time steps in a transient state where firing rates have not yet converged to the correct values. With high-entropy initialization, neurons converge much more quickly to their steady firing rates, and the output is correct from the first time step.
