Behavioral Cloning

Behavioral cloning (BC) is the most direct imitation method: treat expert trajectories as labeled data and run supervised learning on $(s,a)$ pairs, minimizing $\min_\pi \mathbb{E}_{(s,a) \sim \pi_E}[-\log \pi(a|s)]$ — maximum-likelihood fitting of the expert policy.

a fast start when expert demos are available but rewards are hard to design — data-driven driving, teleoperated robot demos; LLM SFT (supervised fine-tuning) is BC in disguise.

simple, yet two fatal limitations: (1) compounding error — with independent per-step error probability $\epsilon$, deviation grows $\mathcal{O}(t\epsilon)$ by step $t$, so the total loss accumulates to $\sum_{t=1}^{T} \mathcal{O}(t\epsilon) = \mathcal{O}(T^2 \epsilon)$, growing quadratically in horizon; worse, training only sees states inside the expert's distribution (the i.i.d. assumption), while deployment must cope with the shifted state distribution its own mistakes create (covariate shift); (2) causal confusion — the model may latch onto effects rather than causes, e.g. a driving agent correcting from the ego car's lateral offset (an effect of its own steering) instead of the true cause — plausible in training, but the feedback loop breaks at test time and performance collapses.

Why is BC's compounding error $\mathcal{O}(T^2)$? Give the derivation intuition: how does per-step error $\epsilon$ accumulate quadratically along the trajectory?

What is causal confusion? Give a driving/robotics example and its mitigations.

BC vs DAgger? How does DAgger fix covariate shift?

Why does BC look fine in training but fail at deployment? How does the i.i.d. assumption break?

BC vs GAIL/IRL: which imitation approach fits which scenario?