Glossary · term60 of 202
distillation
Training a smaller model to mimic a larger one's outputs, letting the smaller model inherit much of its capability at a fraction of the compute cost.
also written as distilling · model distillation
Related terms
LoRA adapterA small add-on set of weights trained separately from a base model to customize its behavior for a specific task, cheaper to produce than retraining the whole model.view termopen-weight modelAn AI model whose trained parameters are published for anyone to download and run on their own hardware, unlike a closed model accessed only through a hosted API.view term
At a glance
- cited by
- 2 posts
- categories
- 2
- first used
- jul 2026
Claude Is a Math Problem (And the Context Window Is the Equation)The Claude context window is an equation, not a chat. Why failed attempts anchor the output, where measured degradation starts, and when to clear and re-derive.
The Open Weights Rules Nobody Can Comply WithEvery open weights rule on the table stops working the moment a download finishes. Sort the three asks by when they act and see which ones still hold.