Behavioral Cloning

Behavioral cloning (BC) is the most direct imitation method: treat expert trajectories as labeled data and run supervised learning on $(s,a)$ pairs, minimizing $\min_\pi \mathbb{E}_{(s,a) \sim \pi_E}[-\log \pi(a|s)]$ — maximum-likelihood fitting of the expert policy.

a fast start when expert demos are available but rewards are hard to design — data-driven driving, teleoperated robot demos; LLM SFT (supervised fine-tuning) is BC in disguise.

simple, yet two fatal limitations: (1) compounding error — with independent per-step error probability $\epsilon$, deviation grows $\mathcal{O}(t\epsilon)$ by step $t$, so the total loss accumulates to $\sum_{t=1}^{T} \mathcal{O}(t\epsilon) = \mathcal{O}(T^2 \epsilon)$, growing quadratically in horizon; worse, training only sees states inside the expert's distribution (the i.i.d. assumption), while deployment must cope with the shifted state distribution its own mistakes create (covariate shift); (2) causal confusion — the model may latch onto effects rather than causes, e.g. a driving agent correcting from the ego car's lateral offset (an effect of its own steering) instead of the true cause — plausible in training, but the feedback loop breaks at test time and performance collapses.

Test Your Knowledge: Practice Questions for "Behavioral Cloning"

Single choice pitfall questions with instant feedback and mistake tracking.