A tool specification contains heterogeneous information: tool identity, description, argument schema, value-format rules, examples, failure patterns, and hard cases. Packing everything into one state sounds simple, but small on-device models have limited state capacity. More information does not always improve function-calling accuracy.
Retention-aware boundary calibration
For each fine-tuned RWKV-7 model, we progressively append tool-information sections and evaluate function-calling accuracy on 200 single-call calibration examples. We choose the first point at which calibration accuracy reaches its maximum. Information before that boundary becomes Base knowledge; later sections become Corrective knowledge.
This is deliberately conservative. If later sections no longer improve accuracy—or begin to reduce it—we avoid forcing them into every request.
| Model | Selected boundary | Interpretation |
|---|---|---|
| RWKV-7 0.1B | AS | Keep canonical schema compact |
| RWKV-7 0.4B | AS | Same calibrated common path |
| RWKV-7 1.5B | AS | Base ends at argument schema |
| RWKV-7 2.9B | AF | Larger state retains more information |
The calibration is currently model-level: one boundary is applied across the tool repository for a given backbone. A finer-grained extension could calibrate each tool independently.
What goes into the Base State?
The Tool Name, Tool Description, and Argument Schema are required across the evaluated models. For RWKV-7 1.5B, the calibrated Base boundary is AS, so the canonical execution contract remains in the fast path.
Base State
├── system instruction
├── tool name
├── tool description
└── argument schemaWhat goes into the Corrective State?
Later information—format rules, difficult values, hard cases, and corrective examples—can remain available without burdening the first attempt. Validator feedback determines whether execution terminates, retries the Corrective State, or advances to another candidate.
User query
↓
Retrieve candidate
↓
Load Base State
↓
Generate function call
├── valid → execute
└── error → load Corrective State → retryCompilation is only a forward pass
Once the boundary is chosen, compilation is straightforward. The selected prompt sections are tokenized and fed to a frozen backbone. After the final token, EngramState saves the remaining recurrent state and discards the logits. There is no gradient computation, backpropagation, or parameter update.
For RWKV-7, one saved state includes three tensor families from every layer:
- time-mixing token-shift state,
- WKV recurrent state,
- channel-mixing token-shift state.
For RWKV-7 1.5B, one complete uncompressed state is about 12 MiB; the Base + Corrective pair is then compressed for storage.
Why not merge Base and Corrective?
Larger models may eventually retain enough information in one state that the split becomes less important. The 2.9B calibration already shifts to a later boundary, suggesting increased capacity. But for small on-device models, separating common and rare information gives a better latency/robustness trade-off.