Convolution Basics

Convolution is the fundamental operator of CNNs, with two core properties — parameter sharing and local connectivity. For input $H \times W \times C_{in}$ and kernel $k_h \times k_w \times C_{in} \times C_{out}$, the output size is $H_{out} = \lfloor\frac{H+2p-k}{s}\rfloor + 1$ (similarly for $W$, with padding $p$ and stride $s$); parameters per layer: $k_h k_w C_{in} C_{out} + C_{out}$ (incl. bias). Sharing: the same filter slides over the whole image, so parameter count is independent of input resolution. Locality: each output pixel depends only on a $k_h \times k_w \times C_{in}$ local window.

feature-extraction backbone for image classification/detection/segmentation; interview staples include hand-computing output sizes, parameter counts, the channel-mixing role of 1×1 conv, dilated convolution for larger receptive fields at fixed resolution, and why conv layers have far fewer parameters than fully connected layers.

vs. fully connected layers, parameters drop from $O(HW C_{in} \times HWC_{out})$ to $O(k_h k_w C_{in} C_{out})$ — tens of thousands of times fewer with 3×3 kernels and hundreds of channels — while encoding locality and translation-equivariance priors directly into the architecture, making deep networks on high-resolution inputs trainable.

Hand-compute the output size: 224×224 input, 5×5 kernel, stride 2, padding 1.

Count parameters: 3×3 conv, 64 input channels, 128 output channels, including bias.

What do parameter sharing and local connectivity each solve? Why is conv parameter count independent of input resolution?

A 5×5 conv vs two stacked 3×3 convs have the same receptive field; compare their parameter counts and the saving.

What does a 1×1 conv do? How does dilated convolution enlarge the receptive field without downsampling?