Sora Video World Models & Spacetime Patch

Sora-Style Video World Models & 3D Spacetime Latent Patch Architectures transition generative models from static 2D images into interactive physical World Simulators understanding geometry, temporal continuity, and physical dynamics; the 3 architectural pillars include: 1) 3D Spacetime Video Compression VAE: compressing variable-resolution, variable-duration video tensors $(T \times H \times W \times C)$ across both spatial and temporal axes into a unified compact latent manifold; 2) 3D Spacetime Patches: slicing 3D latents into spacetime spacetime patch cubes ($2 \times 16 \times 16$), flattening them into token sequences processed by standard Diffusion Transformers (DiT); 3) Emergence of World Simulators: scaling compute and data induces emergent 3D camera parallax coherence, object permanence during occlusions, and intuitive physics simulation.

Autonomous driving world simulation, long-horizon video generation, and embodied agent synthetic training environments.

2D UNet video generators suffer temporal jitter, morphing objects, and spatial incoherence; 3D Spacetime DiTs deliver seamless 60s coherent physical world simulations.

Diagram the tensor dimension transformations from raw $(T, H, W, C)$ video through 3D VAE compression into 3D spacetime tokens fed to DiT?

Why does the Transformer-based DiT architecture exhibit superior compute-scaling efficiency over convolutional 2D+1D UNets?

How does Sora handle variable-resolution, arbitrary-aspect-ratio videos natively without center cropping via flexible patch packing?

Explain how large-scale video prediction naturally induces 3D parallax geometry and object permanence without explicit 3D inductive biases?

Analyze the current theoretical failure modes of video world models in physical causality inversion and multi-body rigid collision dynamics?