sparse model

A model built with far more total parameters than it uses on any single request, activating only a fraction of them per query instead of running the whole network every time. This keeps inference cheaper while the model still holds more learned capacity.

also written as sparse architecture