To generate each token, the machine reads the model's weights from memory once. So the speed ceiling is memory bandwidth divided by the bytes read per token. Real machines reach a fraction of that ceiling.
The numbers we use
| Machine type | Share of ceiling | Source |
|---|---|---|
| mac | 80% | Calibrated on a measured run |
| x86-igpu | 60% | Assumption, pending measured runs |
| unified-x86 | 70% | Assumption, pending measured runs |
| unified-arm | 70% | Assumption, pending measured runs |
Memory: a model needs its file size plus about 1.5 GB for context. Macs with 36 GB or less give the GPU about two-thirds of RAM by default; bigger Macs about three-quarters. Mixture-of-experts models read only their active experts per token, so they run faster than their size suggests. Model sizes come from ollama.com (4-bit Q4_K_M). Prices are US list prices where we have a source; otherwise we leave the price blank.
Help make it measured
Every submitted run replaces an estimate with a real number. Submit yours.