Epoch
Definition
One complete pass through the entire training dataset during model training.
An epoch represents the model seeing every example in the training set exactly once. Training typically requires multiple epochs for the model to converge — simple models might need 10-100 epochs, while some deep learning tasks require hundreds. However, for large language model pre-training, models often train for less than one epoch on their massive datasets, as seeing each document once may be sufficient (and multiple epochs on internet data can cause memorization). Monitoring performance across epochs helps detect overfitting (when validation loss starts increasing while training loss continues to decrease). Early stopping — halting training when validation performance stops improving — is a common regularization technique that depends on tracking epoch-level metrics.
Related Terms
Batch Size
The number of training examples processed together in one forward and backward pass before the model...
Hyperparameter
A configuration value set before training begins that controls the training process itself, as oppos...
Overfitting
When a model learns the training data too well, including its noise and peculiarities, and fails to ...
Training Data
The dataset used to teach a machine learning model, consisting of examples from which the model lear...