JEPA Joint-Embedding Prediction

JEPA (Joint Embedding Predictive Architecture) does predictive self-supervised learning in latent space: a context encoder $g$ takes the current latent $z_t$ and action $a_t$ and predicts the target representation $f(z_{t+1})$, where the target encoder $f$ is updated by EMA (exponential moving average). The training loss is: $$\ell = \lVert g(z_t, a_t) - f(z_{t+1}) \rVert^2$$ i.e. mean squared error between the predicted and target representations. The EMA target plus the asymmetric structure (no gradient to $f$) prevents representational collapse, so no negative samples are needed.

self-supervised representation learning for images/videos (I-JEPA, V-JEPA), world-model learning; a common interview entry is 'JEPA vs VAE/diffusion reconstruction'.

VAE/diffusion reconstruct in pixel space, wasting compute on unpredictable details (background, texture); JEPA predicts only in abstract latent space and discards unpredictable components — lower compute and more semantic representations, with large gains on downstream linear probing and few-shot classification.

Write JEPA's latent prediction loss and explain $g$, $f$, $z_t$, $a_t$?

What does the EMA target encoder do? Why doesn't JEPA collapse without negative samples?

What is the essential difference between JEPA and VAE/diffusion reconstruction?

How does V-JEPA apply JEPA to video understanding? What representation is predicted?

JEPA vs contrastive learning (CLIP/InfoNCE): differences in objective and negative-sample usage?