Encoder

A neural network component that processes input data and produces a compressed representation capturing its essential features. In transformer architectures, encoders use self-attention to create contextual representations of input tokens. BERT is a prominent encoder-only model used for classification and embedding tasks.