Representation Learning
Representation learning discovers features — embeddings, hidden states, or codes — that make downstream tasks easier than working from raw inputs.
How it’s done
Self-supervised objectives (masked/next-token LM, contrastive learning) and supervised multitask training both produce useful latent spaces. Word embeddings, BERT states, and CLIP-style image–text encoders are classic examples.
Why practitioners care
Good representations enable transfer learning: freeze or lightly tune an encoder and train a small head. In RAG and search, representation quality is retrieval quality — evaluate with task metrics, not only intrinsic loss.