GRU

GRU (Gated Recurrent Unit) is a simplified LSTM variant with only two gates. The update gate $z_t = \sigma(W_z[h_{t-1}, x_t] + b_z)$ decides how much history to keep vs how much new information to write; the reset gate $r_t = \sigma(W_r[h_{t-1}, x_t] + b_r)$ decides how much of the history to discard; the candidate hidden state is $\tilde{h}_t = \tanh(W_h[r_t \odot h_{t-1}, x_t] + b_h)$; and the output is $h_t = (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t$ ($\odot$ elementwise). Compared with LSTM: there is no separate cell state, the forget and input gates merge into the update gate, and weights drop from 4 sets to 3 (roughly 25% fewer parameters). When $z_t \approx 0$, $h_t \approx h_{t-1}$ — the same additive shortcut as LSTM's $f_t \approx 1$ that keeps long-range memory; when $r_t \approx 0$, the candidate ignores history and depends only on the current input, capturing short-term dependencies.

the recurrent unit of choice for parameter-sensitive, small-data, or real-time inference settings (speech, forecasting, lightweight sequence models); in interviews it is a staple comparison question — GRU vs LSTM formulas, parameter count, capacity and training efficiency.

LSTM's four gated components are redundant on most tasks; GRU achieves nearly the same long-term memory with 3 weight sets — about 25% fewer parameters and cheaper per-step computation, so it trains faster and overfits less on small data and edge devices. The cost: no dedicated cell state, so stored information is coupled with the output in one $h_t$, a coarser memory granularity.

Test Your Knowledge: Practice Questions for "GRU"

Single choice pitfall questions with instant feedback and mistake tracking.