Baum-Welch (HMM EM)

The learning problem estimates HMM parameters $\lambda = (A, B, \pi)$ from observations alone with no state labels; Baum-Welch is exactly EM applied to HMMs. The E-step uses the current parameters $\lambda^{(old)}$ and the forward-backward algorithm to compute posteriors: the state posterior $\gamma_t(i) = P(q_t = i \mid O, \lambda) = \frac{\alpha_t(i)\beta_t(i)}{\sum_j \alpha_t(j)\beta_t(j)}$ (probability of being in state $i$ at time $t$) and the transition posterior $\xi_t(i, j) = \frac{\alpha_t(i)\, a_{ij}\, b_j(o_{t+1})\, \beta_{t+1}(j)}{P(O \mid \lambda)}$ (moving $i \to j$ from $t$ to $t{+}1$); the M-step reestimates parameters as expected counts

iterating E/M until the likelihood $P(O \mid \lambda)$ converges.

unsupervised sequence learning — unlabeled segmentation, speech parameter estimation, bio-sequence modeling; interviews often ask how BW instantiates EM and about its convergence and local optima.

with hidden states, direct likelihood maximization is intractable; EM iteratively builds and raises a lower bound on the likelihood via expected sufficient statistics, guaranteeing that $P(O \mid \lambda)$ never decreases; but the objective is non-convex, so only local optima are guaranteed — practice uses multiple random restarts, model selection for the state count $N$, and scaled forward-backward to avoid numerical underflow on long sequences.

Write the Baum-Welch reestimation formula $\hat a_{ij} = \frac{\sum_t \xi_t(i,j)}{\sum_t \gamma_t(i)}$ and explain the meaning of $\gamma_t(i)$ and $\xi_t(i,j)$.

Why is Baum-Welch a special case of EM? What do its E-step and M-step correspond to?

How do forward-backward computations yield $\gamma_t(i)$ and $\xi_t(i,j)$ (combining $\alpha_t(i)$, $a_{ij}$, $b_j(o_{t+1})$, $\beta_{t+1}(j)$)?

Does Baum-Welch always converge to the global optimum? Why not, and how do you handle local optima?

What numerical problem arises in forward-backward over long sequences, and how is it fixed?