Classic CNN Architectures

The CNN evolution line: LeNet (1998, 5-layer conv+pool+FC, handwritten digits) → VGG (2014, all-3×3 small-kernel stacking, 16/19 layers) → ResNet (2015, residual shortcuts, up to 152 layers) → DenseNet (2017, dense layer-wise concatenation, feature reuse). Design principles: ① small-kernel stacking replaces large kernels — two 3×3 match a 5×5 receptive field (recurrence $2l+1$) with params dropping from $25C^2$ to $2 \times 9C^2$ (~28% less); ② depth over width — VGG trades depth for accuracy, residuals make extreme depth routine; ③ halve resolution, double channels (2× rule), keeping per-layer FLOPs constant; ④ DenseNet reuses features by concatenation instead of re-learning them, with fewer params and smoother gradients.

classic interview archaeology — why VGG uses only 3×3 kernels, what ResNet solves, how DenseNet differs from ResNet; also the default pretrained backbone for transfer learning (feature extraction / fine-tuning).

scaling from shallow to very deep networks: naive deepening hits degradation (VGG's ceiling), residuals make 152 layers trainable, DenseNet improves feature utilization, and ImageNet top-5 error falls from ~7.3% (VGG) to ~4.6% (ResNet-152) with params and FLOPs kept engineering-friendly.

Why stack two 3×3 convs instead of one 5×5? Compare receptive field, parameters and computation.

Outline LeNet→VGG→ResNet→DenseNet; what problem does each generation solve over its predecessor?

Why does halving resolution while doubling channels keep FLOPs roughly constant?

Compare VGG vs ResNet: depth, parameters, training difficulty, accuracy.

Why does depth matter for CNN accuracy? What is the bottleneck of naive deepening (degradation)?