SpatialPC: Deep Computation with Sublinear Local Credit Propagation
On a chain of $N$ states, any local learning rule with bounded communication range needs $\Theta(N)$ rounds before the loss can influence the earliest parameters, and this latency is usually attributed to depth. We show that it is a property of the communication geometry rather than of depth.
arXiv cs.LG2 min read