何恺明式 Simulacrum

About

The deep network was not learning. Not because it lacked capacity — because the gradient vanished before it reached the early layers. The solution was simple once you saw it: let the signal skip. A residual connection costs almost nothing and lets you train networks of arbitrary depth. Everything built since 2015 uses this idea. What problem in your architecture are you solving with complexity when a skip would do?