M3-098M3: Deep Learning FoundationsConvolution & Vision FoundationsEasy
⚡Flashcard Drill➔Mastery:

Recursive Calculation of Receptive Field in Convolutional Networks

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M3-099 →
📐 Mathematical Definition
rout=rin+(k−1)⋅∏i<lsi;jump=∏isir_{out}=r_{in}+(k-1)\cdot\prod_{i<l}s_i;\qquad \text{jump}=\prod_i s_i
⚡ One-Sentence Core Answer
A feature's receptive field expands recursively via rl=rl−1+(kl−1)⋅jl−1r_l = r_{l-1} + (k_l - 1) \cdot j_{l-1}, where cumulative stride jj exponentially magnifies downsampled kernels.

📌 Key Takeaways

  • •
    Recursive formula: rl=rl−1+(kl−1)⋅jl−1r_l = r_{l-1} + (k_l - 1) \cdot j_{l-1}, with base condition r0=1,j0=1r_0 = 1, j_0 = 1
  • •
    Jump / Effective Stride: jl=jl−1⋅sl=∏i=1lsij_l = j_{l-1} \cdot s_l = \prod_{i=1}^l s_i; strides accumulate multiplicatively
  • •
    Effective vs Theoretical: due to Gaussian-like gradient concentration, the effective receptive field covers only a fraction of the theoretical field

📐 Mathematical Derivations

Mathematical Formulations (Araujo et al., Distill 2019):
Let rlr_l be the receptive field of layer ll, klk_l be kernel size, sls_l stride, and jlj_l cumulative stride (jump).
- Base Initialization: r0=1,j0=1r_0 = 1, \quad j_0 = 1.
- Recursive Step for Layer $l$:
rl=rl−1+(kl−1)⋅jl−1r_l = r_{l-1} + (k_l - 1) \cdot j_{l-1},
jl=jl−1⋅slj_l = j_{l-1} \cdot s_l.
Intuition: A kernel of size klk_l covers (kl−1)(k_l - 1) intervals. Each interval in layer l−1l-1 corresponds to jl−1j_{l-1} pixels in the original input image. Thus, each step expands the coverage by (kl−1)jl−1(k_l - 1) j_{l-1}.
Theoretical vs Effective Receptive Field (ERF):
Luo et al. (NeurIPS 2016) proved that the distribution of impact of an input pixel on a central output feature decays as a 2D Gaussian. The Effective Receptive Field (ERF) (containing 95% of gradient energy) scales with L\sqrt{L}, occupying only a central fraction (20−30%20-30\%) of the theoretical bounding box.

🏭 Production Trade-offs

Design implication for Dense Tasks: In semantic segmentation and object detection, theoretical receptive field must cover the entire input image (r>512r > 512). Architectures achieve this via dilated convolutions (DeepLab) or Feature Pyramid Networks (FPN).
⚠️ Common Interview Pitfalls
  • ✕
    Assuming every pixel inside the theoretical receptive field contributes equally; peripheral pixels contribute near-zero gradient signal
  • ✕
    Forgetting that pooling layers act as kernels with k=pool_sizek = \text{pool\_size} and s=pool_strides = \text{pool\_stride} in receptive field recursion
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does the Effective Receptive Field (ERF) scale with L\sqrt{L} rather than linearly with depth LL?
  • ?
    How do dilated convolutions scale the kernel parameter (k−1)(k-1) in the recursive receptive field formula?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM3-097: Calculating Convolutional Layer Parameter Count and Output Spatial Dimensions📋 Back to BankNext →M3-099: Pooling vs Strided Convolutions: Structural and Functional Trade-offs