Why retries matter
EngramState validates generated calls and may try a Corrective State or another retrieved candidate. This makes average latency only part of the story. A deployment-oriented evaluation should ask how bad the tail can get.
Single-call bound
Under Top-4 retrieval, a single-call query can make at most five total attempts: the initial Base attempt plus up to four retries. Execution terminates after the Corrective path for the final candidate.
Multi-call bound
For multi-call execution, each candidate's Base and Corrective states are attempted at most once across calls. Under Top-4 this yields at most eight total attempts.
| Retry count | ES TTFT / attempt | ES cumulative E2E | Prompt TTFT / attempt | Prompt cumulative E2E |
|---|---|---|---|---|
| Initial | 0.203 s | 1.595 s | 7.677 s | 9.047 s |
| 1 | 0.212 s | 3.202 s | 8.847 s | 19.248 s |
| 2 | 0.206 s | 4.812 s | 8.852 s | 29.494 s |
| 3 | 0.204 s | 6.411 s | 8.839 s | 39.697 s |
| 4 · ES single-call worst case | 0.205 s | 8.204 s | 10.965 s | 56.540 s |
| 7 · ES multi-call worst case | 0.206 s | 13.126 s | 10.913 s | 94.347 s |
| 23 · stress reference | 0.206 s | 39.38 s | 14.555 s | 318.490 s |
Why the per-attempt TTFT stays flat
Each EngramState retry loads a compact candidate state and runs from the short live query. The prompt baseline repeatedly pays the cost of long tool-conditioning context. This is why repeated retries amplify the gap rather than erasing it.
What “on-device” means here
The mobile evaluation uses the Galaxy S25 Ultra as the inference target. The model execution and state-loading path measured for these results run locally on the device; the numbers are not remote API round-trip times.
For a product claim, we prefer this wording over the looser phrase “server-free”: fully on-device model inference with no remote model serving in the measured execution path.
What remains outside this benchmark
These measurements characterize model/state execution and retry behavior. A production app can add unrelated costs such as Android intent dispatch, app startup, sensor access, UI animation, or network-bound tools. Those should be measured separately rather than folded into model TTFT.