RNN (Recurrent Neural Network)

Recurrent Neural Networks process sequences one element at a time, maintaining a hidden state that acts as memory of past inputs. This makes them naturally suited for tasks like language modeling, speech recognition, and time series prediction. However, basic RNNs struggle with long sequences due to the vanishing gradient problem, where gradients diminish as they propagate back through many time steps. This limitation led to improved architectures like LSTM and GRU, which use gating mechanisms to better preserve long-term information. Despite these improvements, RNNs have been largely replaced by transformers for most sequence tasks, as transformers can process all positions in parallel and handle long-range dependencies more effectively.