🧠 Deep LearningID: depthwise-separable

Depthwise Separable Conv

Q: Derive the depthwise-separable vs standard-conv compute ratio $\frac{1}{C_{out}} + \frac{1}{k^2}$; where does each term come from?

【Depthwise Separable Conv 核心考点 1】Derive the depthwise-separable vs standard-conv compute ratio $\frac{1}{C_{out}} + \frac{1}{k^2}$; where does each term come from?。技术解析：Definition: Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one k \times k kernel (no cross-channel mixing); Pointwise — a 1 \times 1 conv mixes channels linearly. FLOPs ratio: standard conv = k^2 C_{... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/depthwise-separable

Q: Hand-compute: 3×3 depthwise separable conv with C_out=256 — what fraction of standard-conv compute is left, and what is the saving?

【Depthwise Separable Conv 核心考点 2】Hand-compute: 3×3 depthwise separable conv with C_out=256 — what fraction of standard-conv compute is left, and what is the saving?。技术解析：Definition: Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one k \times k kernel (no cross-channel mixing); Pointwise — a 1 \times 1 conv mixes channels linearly. FLOPs ratio: standard conv = k^2 C_{... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/depthwise-separable

Q: Why does MobileNet need the 1×1 pointwise conv? What breaks if you keep only depthwise?

【Depthwise Separable Conv 核心考点 3】Why does MobileNet need the 1×1 pointwise conv? What breaks if you keep only depthwise?。技术解析：Definition: Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one k \times k kernel (no cross-channel mixing); Pointwise — a 1 \times 1 conv mixes channels linearly. FLOPs ratio: standard conv = k^2 C_{... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/depthwise-separable

Q: What is MobileNetV2's inverted residual (linear bottleneck)? Why does ReLU destroy low-dimensional features?

【Depthwise Separable Conv 核心考点 4】What is MobileNetV2's inverted residual (linear bottleneck)? Why does ReLU destroy low-dimensional features?。技术解析：Definition: Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one k \times k kernel (no cross-channel mixing); Pointwise — a 1 \times 1 conv mixes channels linearly. FLOPs ratio: standard conv = k^2 C_{... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/depthwise-separable

Q: How do depthwise separable conv and grouped conv (e.g. ResNeXt's 32 groups) differ in compute and expressiveness?

【Depthwise Separable Conv 核心考点 5】How do depthwise separable conv and grouped conv (e.g. ResNeXt's 32 groups) differ in compute and expressiveness?。技术解析：Definition: Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one k \times k kernel (no cross-channel mixing); Pointwise — a 1 \times 1 conv mixes channels linearly. FLOPs ratio: standard conv = k^2 C_{... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/DL/card/depthwise-separable

深度可分离卷积

🎯Core Definition

Depthwise separable convolution splits a standard conv into two steps: Depthwise — each channel is convolved independently with one

k \times k

kernel (no cross-channel mixing); Pointwise — a

1 \times 1

conv mixes channels linearly. FLOPs ratio: standard conv

= k^2 C_{in} C_{out} HW

vs separable

= k^2 C_{in} HW + C_{in} C_{out} HW

(

HW

= output spatial size), giving

\frac{1}{C_{out}} + \frac{1}{k^2}

— the first term from pointwise, the second from depthwise. With

k=3

C_{out}=256

\frac{1}{256} + \frac{1}{9} \approx 0.0039 + 0.1111 \approx 0.114

, i.e. ~11.4% of standard-conv compute, an ~8.8× saving; parameters shrink by the same ratio (identical formula).

💡Use Cases

the core operator of MobileNetV1 (replacing every standard conv); MobileNetV2 adds inverted residuals + linear bottlenecks. Interviews ask for the FLOPs-ratio derivation, what exactly is saved, and why the 1×1 conv is essential.

⚡Key Problems Solved

decoupling spatial convolution from channel mixing changes compute from proportional to

C_{in} \times C_{out}

C_{in} + C_{out}

; at the cost of only ~1–2 top-1 accuracy points (ImageNet ~70.6% vs ~71.7% baseline), compute and parameters shrink ~8.8×, making vision models run in real time on mobile devices.

🎯5 High-Frequency Exam Points

Derive the depthwise-separable vs standard-conv compute ratio

\frac{1}{C_{out}} + \frac{1}{k^2}

; where does each term come from?

Hand-compute: 3×3 depthwise separable conv with C_out=256 — what fraction of standard-conv compute is left, and what is the saving?

Why does MobileNet need the 1×1 pointwise conv? What breaks if you keep only depthwise?

What is MobileNetV2's inverted residual (linear bottleneck)? Why does ReLU destroy low-dimensional features?

How do depthwise separable conv and grouped conv (e.g. ResNeXt's 32 groups) differ in compute and expressiveness?

📖 In-depth Guide:📄 cnn-and-vit-architectures →

Updated 2026-08-12

🎯

Test Your Knowledge: Practice Questions for "Depthwise Separable Conv"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardResNet Skip Connections Next Card →ViT & Inductive Bias

🔗 More Deep Learning Knowledge Cards

Activation Functions Adam & AdamW Autograd Compute Graph Batch Normalization