Benchmark

A benchmark is a standardized dataset and metric for comparing models (MMLU, HumanEval, GSM8K, etc.). Useful for tracking progress; easy to overfit or misread when the train set leaks into pretraining.

Always pair public benchmarks with private, task-specific tests for product decisions.

← Back to AI Glossary