G1Medium★ Core Essential · Industrial BedrockPart G · Architectures & Parameter-Efficient Fine-Tuning

Conv2d from Scratch with Stride & Padding

⚡Flashcard Drill➔

From-scratch 2D sliding-window convolution showing exact output sizing, zero padding, and channel reduction.

⏱️ Time: O(B * C_out * H_out * W_out * C_in * Kh * Kw)
💾 Space: O(B * C_in * (H+2P) * (W+2P)) 填充后的中间张量
🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def conv2d_forward(
x: np.ndarray, # (B, C_in, H, W)
w: np.ndarray, # (C_out, C_in, Kh, Kw)
b: np.ndarray = None, # (C_out,)
stride: int = 1,
padding: int = 0
) -> np.ndarray:
B, C_in, H, W = x.shape
C_out, _, Kh, Kw = w.shape
# 1. 计算输出特征图尺寸
H_out =
W_out = (W + 2 * padding - Kw) // stride + 1
# 2. 空间零填充
if padding > 0:
x_pad = np.pad(x, ((0, 0), (0, 0), (padding, padding), (padding, padding)), mode='constant')
else:
x_pad = x
out = np.zeros((B, C_out, H_out, W_out), dtype=x.dtype)
# 3. 滑动窗口四层循环计算
for h in range(H_out):
h_start = h * stride
h_end = h_start + Kh
for w_idx in range(W_out):
w_start = w_idx * stride
w_end = w_start + Kw
# 截取感受野切片: (B, C_in, Kh, Kw)
x_slice = x_pad[:, :, h_start:h_end, w_start:w_end]
# 沿输入通道与核尺寸乘加规约,输出 (B, C_out)
# x_slice: (B, 1, Cin, Kh, Kw), w: (1, Cout, Cin, Kh, Kw)
val =
if b is not None:
val += b[np.newaxis, :]
out[:, :, h, w_idx] = val
return out
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How does Depthwise Separable Convolution reduce Conv2d FLOPs?
Next: LoRA (Low-Rank Adaptation) →
💡

Core Mental Anchor / Mnemonic

Pad spatial borders, compute integer output shape, slide receptive fields, sum across Cin.

📐 Mathematical Derivation & Core Formula

Hout=⌊Hin+2P−KhS⌋+1,Ob,cout,h,w=∑cin∑i∑jXb,cin,h⋅S+i,w⋅S+jWcout,cin,i,jH_{\text{out}} = \left\lfloor\frac{H_{\text{in}} + 2P - K_h}{S}\right\rfloor + 1, \quad O_{b, c_{\text{out}}, h, w} = \sum_{c_{\text{in}}} \sum_{i} \sum_{j} X_{b, c_{\text{in}}, h\cdot S + i, w\cdot S + j} W_{c_{\text{out}}, c_{\text{in}}, i, j}
### Sliding Window Spatial Projection
2D convolution maps X∈RB×Cin×H×WX \in \mathbb{R}^{B \times C_{\text{in}} \times H \times W} to O∈RB×Cout×Hout×WoutO \in \mathbb{R}^{B \times C_{\text{out}} \times H_{\text{out}} \times W_{\text{out}}} using zero padding PP, stride SS, and CoutC_{\text{out}} kernels of size (Cin,Kh,Kw)(C_{\text{in}}, K_h, K_w).
In optimized BLAS/cuDNN backends, im2col flattens spatial patches into columns so convolution executes via a single GEMM matrix multiplication.

🔄 Tensor Dimensions & Shape Flow

x: (B, Cin, H, W) -> zero pad -> (B, Cin, H+2P, W+2P) -> patch slice * kernel sum -> out: (B, Cout, H_out, W_out)

🛡️ Numerical Stability & Pitfalls Checklist

  • Use floor integer division // for output spatial dimensions (H + 2P - Kh) // S + 1.
  • Pad only spatial axes (H, W), never batch or channel axes.
  • Optimized libraries lower Conv2d to im2col + GEMM matrix multiplication for high throughput.

💻 Reference Code Implementation

import numpy as np

def conv2d_forward(
    x: np.ndarray,          # (B, C_in, H, W)
    w: np.ndarray,          # (C_out, C_in, Kh, Kw)
    b: np.ndarray = None,   # (C_out,)
    stride: int = 1,
    padding: int = 0
) -> np.ndarray:
    B, C_in, H, W = x.shape
    C_out, _, Kh, Kw = w.shape
    
    # 1. 计算输出特征图尺寸
    H_out = (H + 2 * padding - Kh) // stride + 1
    W_out = (W + 2 * padding - Kw) // stride + 1
    
    # 2. 空间零填充
    if padding > 0:
        x_pad = np.pad(x, ((0, 0), (0, 0), (padding, padding), (padding, padding)), mode='constant')
    else:
        x_pad = x
        
    out = np.zeros((B, C_out, H_out, W_out), dtype=x.dtype)
    
    # 3. 滑动窗口四层循环计算
    for h in range(H_out):
        h_start = h * stride
        h_end = h_start + Kh
        for w_idx in range(W_out):
            w_start = w_idx * stride
            w_end = w_start + Kw
            
            # 截取感受野切片: (B, C_in, Kh, Kw)
            x_slice = x_pad[:, :, h_start:h_end, w_start:w_end]
            
            # 沿输入通道与核尺寸乘加规约,输出 (B, C_out)
            # x_slice: (B, 1, Cin, Kh, Kw), w: (1, Cout, Cin, Kh, Kw)
            val = np.sum(x_slice[:, np.newaxis, :, :, :] * w[np.newaxis, :, :, :, :], axis=(2, 3, 4))
            
            if b is not None:
                val += b[np.newaxis, :]
                
            out[:, :, h, w_idx] = val
            
    return out

🧪 Runnable Assertions & Validation

Copy and run directly in Python / Jupyter:

import numpy as np
x = np.ones((1, 1, 3, 3))
w = np.ones((1, 1, 2, 2)) # 2x2 全 1 核
out = conv2d_forward(x, w, stride=1, padding=0)
assert out.shape == (1, 1, 2, 2)
assert np.allclose(out, 4.0), "2x2 全 1 卷积结果应全为 4.0"
print("✓ Conv2d 卷积纯手写自测通过")

🎯 Core Follow-up Inquiries & Reference Answers

Q1:How does Depthwise Separable Convolution reduce Conv2d FLOPs?
It factorizes standard convolution into per-channel spatial filtering (Depthwise) followed by 1×11 \times 1 cross-channel mixing (Pointwise), lowering compute to roughly 1Cout+1KhKw\frac{1}{C_{\text{out}}} + \frac{1}{K_h K_w} of a dense convolution.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
← Prev: Multi-Armed Bandit: epsilon-Greedy & UCB1Back to Part G & All 105Next: LoRA (Low-Rank Adaptation) →