EngramState stores one Base State and one Corrective State per tool. For RWKV-7 1.5B, the combined FP16 representation is approximately 24.76 MiB per tool. At 100 tools, that would already exceed 2.4 GiB.
Why channel-wise quantization?
The WKV recurrent state is not a static model weight. It is a dynamically generated activation, and its channels can have very different value ranges. EngramState therefore quantizes channels independently, using a separate range per channel rather than one global scale.
This is also why weight-oriented methods such as GPTQ do not directly transfer: GPTQ relies on statistics for static weights, whereas a recurrent state is produced dynamically from a specific prompt. AWQ's intuition—protect important channels through scaling—is closer to what channel-wise state quantization already exploits.
Four bits: the practical default
RWKV-7 1.5B · Base + Corrective
FP16 : 24.76 MiB / tool
4-bit : 6.19 MiB / toolFour-bit storage gives an approximately 4× reduction. The choice is practical rather than a claim that four bits are the mathematical limit.
What happens at 3 bits?
| Model | Baseline | RTN | Outlier scaling | Channel-wise |
|---|---|---|---|---|
| RWKV-7 0.4B | 71.3 | 6.0 | 4.7 | 70.0 |
| RWKV-7 1.5B | 65.7 | 9.0 | 11.3 | 65.0 |
| RWKV-7 2.9B | 81.0 | 19.0 | 28.3 | 81.0 |
At 3 bits, naïve RTN and simple outlier scaling collapse badly, while channel-wise quantization stays within 1.3 percentage points of the baseline across the reported models.
And at 2 bits?
| Model | Baseline | RTN | Outlier scaling | Channel-wise |
|---|---|---|---|---|
| RWKV-7 0.4B | 71.3 | 3.3 | 1.3 | 68.3 |
| RWKV-7 1.5B | 65.7 | 6.7 | 3.0 | 61.3 |
| RWKV-7 2.9B | 81.0 | 2.3 | 6.0 | 78.3 |
Two-bit channel-wise storage remains dramatically better than the naïve alternatives, but the accuracy drop is larger.
Storage vs runtime memory
The state repository grows linearly with the number of tools, but EngramState does not keep the entire repository resident during inference. The retriever selects candidates and only the chosen state is loaded for the current attempt. Persistent storage therefore scales with the catalog while active state memory remains bounded by the execution path.