Depthwise Separable Conv

Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one $k \times k$ kernel (no cross-channel mixing); Pointwise — a $1 \times 1$ conv mixes channels linearly. FLOPs ratio: standard conv $= k^2 C_{in} C_{out} HW$ vs separable $= k^2 C_{in} HW + C_{in} C_{out} HW$ ($HW$ = output spatial size), giving $\frac{1}{C_{out}} + \frac{1}{k^2}$ — the first term from pointwise, the second from depthwise. With $k=3$, $C_{out}=256$: $\frac{1}{256} + \frac{1}{9} \approx 0.0039 + 0.1111 \approx 0.114$, i.e. ~11.4% of standard-conv compute, an ~8.8× saving; parameters shrink by the same ratio (identical formula).

the core operator of MobileNetV1 (replacing every standard conv); MobileNetV2 adds inverted residuals + linear bottlenecks. Interviews ask for the FLOPs-ratio derivation, what exactly is saved, and why the 1×1 conv is essential.

decoupling spatial convolution from channel mixing changes compute from proportional to $C_{in} \times C_{out}$ to $C_{in} + C_{out}$; at the cost of only ~1–2 top-1 accuracy points (ImageNet ~70.6% vs ~71.7% baseline), compute and parameters shrink ~8.8×, making vision models run in real time on mobile devices.

Derive the depthwise-separable vs standard-conv compute ratio $\frac{1}{C_{out}} + \frac{1}{k^2}$; where does each term come from?

Hand-compute: 3×3 depthwise separable conv with C_out=256 — what fraction of standard-conv compute is left, and what is the saving?

Why does MobileNet need the 1×1 pointwise conv? What breaks if you keep only depthwise?

What is MobileNetV2's inverted residual (linear bottleneck)? Why does ReLU destroy low-dimensional features?

How do depthwise separable conv and grouped conv (e.g. ResNeXt's 32 groups) differ in compute and expressiveness?