Prompt Tuning

Prompt tuning adapts a large pretrained language model to a downstream task without updating the full weights. You train a small set of soft prompt embeddings prepended to each input. The base model stays frozen; gradients update only the prompt parameters.

Goals

  • Task adaptation with a tiny trainable footprint
  • One shared frozen backbone, many task-specific soft prompts
  • Less storage and compute than full fine-tuning

Contrast

  • Prompt engineering — hand-written hard prompts; no training
  • LoRA / other PEFT — updates low-rank (or adapter) weights inside layers, usually stronger on hard tasks
  • Full fine-tuning — all parameters move; highest capacity and cost

Prompt tuning shines when you need many cheap task heads on one frozen server model. Soft prompts are not human-readable; treat them like small binary adapters in your deploy path.

← Back to AI Glossary