M3-100M3: Deep Learning FoundationsConvolution & Vision FoundationsMedium
⚡Flashcard Drill➔Mastery:
Translation Equivariance vs Translation Invariance: Definitions and Roles in CNNs
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Equivariance means shifting input shifts output feature maps identically (); Invariance means shifting input leaves output completely unchanged ().
📌 Key Takeaways
- •Equivariance: ; convolution operations are mathematically equivariant due to spatial weight sharing
- •Invariance: ; global pooling and classification heads enforce invariance to discard spatial coordinates
- •Task requirements: Object detection and segmentation require equivariance; whole-image classification requires invariance
📐 Mathematical Derivations
Mathematical Formulations:
Let be the spatial translation operator: .
① Translation Equivariance:
A mapping is equivariant with respect to if:
.
- Proof for Convolution:
.
Convolution commutes with translation. If a cat shifts right by 10 pixels, its feature activations shift right by 10 pixels.
② Translation Invariance:
A mapping is invariant with respect to if:
.
- Global Average Pooling (GAP):
.
GAP integrates out spatial dimensions, converting equivariant spatial feature maps into a shift-invariant categorical embedding.
Let be the spatial translation operator: .
① Translation Equivariance:
A mapping is equivariant with respect to if:
.
- Proof for Convolution:
.
Convolution commutes with translation. If a cat shifts right by 10 pixels, its feature activations shift right by 10 pixels.
② Translation Invariance:
A mapping is invariant with respect to if:
.
- Global Average Pooling (GAP):
.
GAP integrates out spatial dimensions, converting equivariant spatial feature maps into a shift-invariant categorical embedding.
🏭 Production Trade-offs
System design alignment: In semantic segmentation, dense bounding box regression, and keypoint tracking, preserving equivariance across all backbone layers is mandatory. In classification, equivariance is converted into invariance at the final pooling head.
⚠️ Common Interview Pitfalls
- ✕Confusing equivariance with invariance; convolution is equivariant, NOT invariant
- ✕Inserting aggressive pooling early in segmentation networks, destroying spatial equivariance needed for mask delineation
🎯 Interviewer Follow-up Inquiries
- ?Why is Global Average Pooling mathematically invariant to translation while convolution is equivariant?
- ?How does group representation theory generalize translation equivariance to rotation and reflection in Group Equivariant CNNs?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚