M3-100M3: Deep Learning FoundationsConvolution & Vision FoundationsMedium
⚡Flashcard Drill➔Mastery:

Translation Equivariance vs Translation Invariance: Definitions and Roles in CNNs

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M3-101 →
📐 Mathematical Definition
equivariant: f(Tx)=Tf(x);invariant: f(Tx)=f(x)\text{equivariant}:\ f(T x)=T f(x);\qquad \text{invariant}:\ f(T x)=f(x)
⚡ One-Sentence Core Answer
Equivariance means shifting input shifts output feature maps identically (f(Tx)=Tf(x)f(T x) = T f(x)); Invariance means shifting input leaves output completely unchanged (f(Tx)=f(x)f(T x) = f(x)).

📌 Key Takeaways

  • •
    Equivariance: f(TΔx)=TΔf(x)f(T_\Delta x) = T_\Delta f(x); convolution operations are mathematically equivariant due to spatial weight sharing
  • •
    Invariance: f(TΔx)=f(x)f(T_\Delta x) = f(x); global pooling and classification heads enforce invariance to discard spatial coordinates
  • •
    Task requirements: Object detection and segmentation require equivariance; whole-image classification requires invariance

📐 Mathematical Derivations

Mathematical Formulations:
Let TΔT_\Delta be the spatial translation operator: (TΔx)[i,j]=x[i−Δi,j−Δj](T_\Delta x)[i, j] = x[i - \Delta_i, j - \Delta_j].
① Translation Equivariance:
A mapping ff is equivariant with respect to TT if:
f(TΔx)=TΔf(x),∀Δf(T_\Delta x) = T_\Delta f(x), \quad \forall \Delta.
- Proof for Convolution:
(W∗(TΔx))[i,j]=∑u,vW[u,v](TΔx)[i−u,j−v]=∑u,vW[u,v]x[i−Δi−u,j−Δj−v]=(W∗x)[i−Δi,j−Δj]=(TΔ(W∗x))[i,j](W * (T_\Delta x))[i, j] = \sum_{u, v} W[u, v] (T_\Delta x)[i-u, j-v] = \sum_{u, v} W[u, v] x[i-\Delta_i-u, j-\Delta_j-v] = (W * x)[i-\Delta_i, j-\Delta_j] = (T_\Delta (W * x))[i, j].
Convolution commutes with translation. If a cat shifts right by 10 pixels, its feature activations shift right by 10 pixels.
② Translation Invariance:
A mapping gg is invariant with respect to TT if:
g(TΔx)=g(x),∀Δg(T_\Delta x) = g(x), \quad \forall \Delta.
- Global Average Pooling (GAP):
GAP(TΔx)=1HW∑i,j(TΔx)[i,j]=1HW∑i′,j′x[i′,j′]=GAP(x)\text{GAP}(T_\Delta x) = \frac{1}{H W} \sum_{i, j} (T_\Delta x)[i, j] = \frac{1}{H W} \sum_{i', j'} x[i', j'] = \text{GAP}(x).
GAP integrates out spatial dimensions, converting equivariant spatial feature maps into a shift-invariant categorical embedding.

🏭 Production Trade-offs

System design alignment: In semantic segmentation, dense bounding box regression, and keypoint tracking, preserving equivariance across all backbone layers is mandatory. In classification, equivariance is converted into invariance at the final pooling head.
⚠️ Common Interview Pitfalls
  • ✕
    Confusing equivariance with invariance; convolution is equivariant, NOT invariant
  • ✕
    Inserting aggressive pooling early in segmentation networks, destroying spatial equivariance needed for mask delineation
🎯 Interviewer Follow-up Inquiries
  • ?
    Why is Global Average Pooling mathematically invariant to translation while convolution is equivariant?
  • ?
    How does group representation theory generalize translation equivariance to rotation and reflection in Group Equivariant CNNs?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM3-099: Pooling vs Strided Convolutions: Structural and Functional Trade-offs📋 Back to BankNext →M3-101: Architectural Design Philosophies: ViT vs Swin Transformer vs ConvNeXt