G1Medium★ Core Essential · Industrial BedrockPart G · Architectures & Parameter-Efficient Fine-Tuning
Conv2d from Scratch with Stride & Padding
Industrial-grade implementation and mathematical foundations of Conv2d from Scratch with Stride & Padding.
⏱️ Time Complexity:
O(B * C_out * H_out * W_out * C_in * Kh * Kw)💾 Space Complexity:
O(B * C_in * (H+2P) * (W+2P)) 填充后的中间张量💡
Core Mental Anchor / Mnemonic
Master Conv2d from Scratch with Stride & Padding: enforce numerical stability, check tensor shapes, and eliminate redundant memory allocations.
📐 Mathematical Derivation & Core Formula
### Mathematical Derivation & Theoretical Principles
Detailed first-principles formulation and architectural mechanics for Conv2d from Scratch with Stride & Padding.
Refer to the LaTeX equation above for the core operator definition. The operator is designed to ensure strict numerical bounds, avoiding floating-point overflows and gradient anomalies.
Detailed first-principles formulation and architectural mechanics for Conv2d from Scratch with Stride & Padding.
Refer to the LaTeX equation above for the core operator definition. The operator is designed to ensure strict numerical bounds, avoiding floating-point overflows and gradient anomalies.
🔄 Tensor Dimensions & Shape Flow
x: (B, Cin, H, W) -> 零填充 -> (B, Cin, H+2P, W+2P) -> 窗口slice -> 与核相乘累加 -> out: (B, Cout, H_out, W_out)
🛡️ Industrial Numerical Stability & Pitfalls
- Ensure proper multi-dimensional tensor broadcasting and keepdims retention.
- Enforce numerical guards (eps clamping and overflow thresholds) during exponentiation and division.
- Verify train versus eval mode behavioral distinctions (e.g. frozen running statistics and dropout bypass).
💻 Industrial Code Implementation
import numpy as np
def conv2d_forward(
x: np.ndarray, # (B, C_in, H, W)
w: np.ndarray, # (C_out, C_in, Kh, Kw)
b: np.ndarray = None, # (C_out,)
stride: int = 1,
padding: int = 0
) -> np.ndarray:
B, C_in, H, W = x.shape
C_out, _, Kh, Kw = w.shape
# 1. 计算输出特征图尺寸
H_out = (H + 2 * padding - Kh) // stride + 1
W_out = (W + 2 * padding - Kw) // stride + 1
# 2. 空间零填充
if padding > 0:
x_pad = np.pad(x, ((0, 0), (0, 0), (padding, padding), (padding, padding)), mode='constant')
else:
x_pad = x
out = np.zeros((B, C_out, H_out, W_out), dtype=x.dtype)
# 3. 滑动窗口四层循环计算
for h in range(H_out):
h_start = h * stride
h_end = h_start + Kh
for w_idx in range(W_out):
w_start = w_idx * stride
w_end = w_start + Kw
# 截取感受野切片: (B, C_in, Kh, Kw)
x_slice = x_pad[:, :, h_start:h_end, w_start:w_end]
# 沿输入通道与核尺寸乘加规约，输出 (B, C_out)
# x_slice: (B, 1, Cin, Kh, Kw), w: (1, Cout, Cin, Kh, Kw)
val = np.sum(x_slice[:, np.newaxis, :, :, :] * w[np.newaxis, :, :, :, :], axis=(2, 3, 4))
if b is not None:
val += b[np.newaxis, :]
out[:, :, h, w_idx] = val
return out
🧪 Runnable Assertions & Validation
Copy and run directly in Python / Jupyter to verify correctness:
import numpy as np
x = np.ones((1, 1, 3, 3))
w = np.ones((1, 1, 2, 2)) # 2x2 全 1 核
out = conv2d_forward(x, w, stride=1, padding=0)
assert out.shape == (1, 1, 2, 2)
assert np.allclose(out, 4.0), "2x2 全 1 卷积结果应全为 4.0"
print("✓ Conv2d 卷积纯手写自测通过")🎯 Core Architecture Follow-up Q&A
Q1:What are the key trade-offs and memory bottlenecks when deploying Conv2d from Scratch with Stride & Padding in high-throughput inference?
Memory bandwidth (HBM to SRAM I/O) is the primary latency factor. Fusing element-wise operations and avoiding intermediate tensor materialization significantly outperforms naive implementations.
Q2:How does Conv2d from Scratch with Stride & Padding handle extreme numerical boundaries or precision reduction (FP16/BF16/INT8)?
Under low precision, operations must be upcasted to FP32 during accumulation to prevent underflow/overflow, followed by proper scaling and clamping before converting back to the target format.