Prompt Tuning
Prompt tuning adapts a large pretrained language model to a downstream task without updating the full weights. You train a small set of soft prompt embeddings prepended to each input. The base model stays frozen; gradients update only the prompt parameters.
Goals
- Task adaptation with a tiny trainable footprint
- One shared frozen backbone, many task-specific soft prompts
- Less storage and compute than full fine-tuning
Contrast
- Prompt engineering — hand-written hard prompts; no training
- LoRA / other PEFT — updates low-rank (or adapter) weights inside layers, usually stronger on hard tasks
- Full fine-tuning — all parameters move; highest capacity and cost
Prompt tuning shines when you need many cheap task heads on one frozen server model. Soft prompts are not human-readable; treat them like small binary adapters in your deploy path.