How ResNet's Skip Connections Made 100-Layer Networks Trainable

Before 2015, deeper networks performed worse due to vanishing gradients—a 56-layer net had higher training error than a 20-layer one. ResNet introduced skip connections that add the input to a block's output, allowing gradients to flow directly through identity paths. This simple change enabled training of 152-layer networks, winning ImageNet and becoming a standard in modern architectures like Transformers.