Gated Recurrent Unit

The Gated Recurrent Unit (GRU) is a recurrent neural network architecture introduced by Cho et al. in 2014 as part of their work on neural machine translation. It was designed as a simpler alternative to the Long Short-Term Memory network, retaining the core idea of gated control over memory while reducing the number of parameters and computational overhead.

Where LSTM uses three gates and a separate cell state, GRU collapses this into two gates and a single hidden state:

The update gate decides how much of the previous hidden state to carry forward into the next timestep. A value near 1 preserves the existing memory almost entirely; a value near 0 allows the new input to dominate. This gate effectively combines the forget and input gates of LSTM into a single operation.

The reset gate controls how much of the previous hidden state to expose when computing the candidate new state. When the reset gate is near 0, the network ignores the previous state and behaves more like a standard feedforward layer processing only the current input.

Because GRU has no separate cell state and one fewer gate than LSTM, it has fewer parameters per layer, trains faster, and requires less memory. In practice, the performance difference between GRU and LSTM varies by task. A 2014 empirical comparison by Chung et al. found that neither architecture consistently outperforms the other - the right choice depends on dataset size, sequence length, and the specific structure of the problem.

GRUs are widely used in speech recognition, time series modeling, natural language generation, and any sequential task where the efficiency gains over LSTM matter more than marginal accuracy differences. Like LSTM, GRUs have largely been replaced by transformer-based architectures for large-scale NLP tasks, but remain a practical and well-understood tool in settings where sequential structure is important and training compute is constrained.