Self-Supervised Learning

A training paradigm where models learn from unlabeled data by predicting missing parts of the input, enabling pre-training on massive datasets without human annotation. Self-supervised learning is the foundation of modern LLMs, which learn language patterns by predicting the next token in text sequences.