Model behaviour: training shows in the tool calls
Resolve rate hides how a model works. The trajectories do not. This page reads the 78 paired OpenCode attempts of Laguna xs 2.1 and MiniMax M2.5 — same harness, same system prompt, same tools, same repositories, same limits — and counts what each model did with its calls. Every number is from the stored trajectory and the per-call ledger, computed by one script for both.
The counts
| per 78 attempts (OpenCode, plain prompt) | Laguna xs 2.1 | MiniMax M2.5 | ratio |
|---|---|---|---|
| LLM calls | 6,514 | 2,527 | 2.6× |
| tool calls | 6,433 | 2,464 | 2.6× |
read calls | 1,556 | 614 | 2.5× |
| whole-file reads (no offset or limit) | 503 | 97 | 5.2× |
| characters pulled in by whole-file reads | 6.0 M | 0.67 M | 9.1× |
| re-reads of a file that had not changed | 728 | 245 | 3.0× |
grep / glob searches | 675 | 432 | 1.6× |
bash calls | 3,832 | 1,194 | 3.2× |
| test runs | 1,167 | 476 | 2.5× |
| redundant test runs (same command, nothing changed) | 38 | 3 | 12.7× |
| edits | 360 | 212 | 1.7× |
| call at which the first edit happened, mean (median) | 24 (15) | 9.6 (6) | 2.5× |
| calls per attempt, mean (median) | 83.5 (69.5) | 32.4 (23) | 2.6× |
| prompt tokens per call | 76.9 K | 26.4 K | 2.9× |
| context carried across the attempt (characters) | 13.9 M | 3.2 M | 4.4× |
| spend after own tests were green | $4.39 | $1.45 | 3.0× |
| resolved | 58 | 59 | — |
What the pattern is
- Laguna reads before it searches; MiniMax searches before it reads. Laguna opened files whole five times as often and re-read unchanged files three times as often; its ratio of searches to reads is 0.43 against MiniMax's 0.70. The first edit came at call 24 on average against call 10.
- Laguna tests more and keeps testing after green. Two and a half times the test runs, thirteen times the redundant re-runs of an unchanged test, and three times the spend after its own tests had already passed. Both models keep going after the fix; Laguna keeps going longer.
- Almost all of Laguna's tokens are cache reads (98.5%) — the cost is not new text, it is the same growing context re-sent on every one of 2.6× as many calls.
- The outcomes are the same. 55 instances resolved by both, 16 by neither; the extra work bought no extra resolves on this subset.
What the judge attributed it to
The rubric-v3 judge pass over the Laguna pilot scored a token efficiency dimension on 84 attempts, with a fixed list of causes, each requiring cited turns. How often each cause was flagged, and its mean share of the attempt's avoidable tokens:
| cause | attempts flagged (of 84) | mean share of avoidable tokens |
|---|---|---|
| repeated reads of unchanged files | 57 | 16.5% |
| redundant test runs | 49 | 15.9% |
| probing without searching first | 39 | 11.7% |
| unbounded file reads | 27 | 6.1% |
| looping | 24 | 5.1% |
| verbose reasoning | 13 | 2.6% |
| unbounded command output | 9 | 1.7% |
| other | 7 | 1.5% |
The judge's causes and the trajectory counts above were computed independently and agree on the order: reads, then tests, then probing.
What is instructable
Three of these behaviours respond to a prompt and two do not. With five working rules appended to the same harness (the prompt experiment), Laguna's whole-file reads fell from 503 to 276 and its cost by 36%; the same rules cut MiniMax's by 28%. The tail after green tests shrank but did not close on either model, and the one reasoning-only stop with no edit appeared only under the rules. The reading habits are a prompt problem; the stopping habit is a harness-enforcement problem; the empty-patch stop is a model problem.
Caveats
One harness, one prompt, 78 django-heavy instances, one run per model, OpenRouter with the provider pinned. The counters are heuristics over trajectories (a "re-read of an unchanged file" is a read of a path whose content hash has not changed since the last read; a "redundant test run" is the same command with no intervening edit), applied identically to both models. The judge's attribution is a model's reading with cited turns, not ground truth. None of this ranks the models on anything but this measurement.
- run
- 01788910552328915713-21ebff89 · 2026-09-08
- model
- openrouter/poolside/laguna-xs-2.1 (alias laguna-xs-2.1-opencode)
- harness
- OpenCode · CLI 1.18.18 (pinned in the harness image build at this framework commit)
- framework
- dd365d8ff1197276992135731e67d81b17fb6e94
- harness image
- sha256:5f326fc00f2decd4f0592b10ac0c36defe26cfed3ae27e0eb24d3804d2df944f
- benchmark
- SWE-bench/SWE-bench_Verified @ 78f471bf655a · swebench 5.0.2 · images SWE-bench_Verified-78f471bf655a.json
- network
- isolated
- explore
- this run in the Explorer: every instance, its trajectory and patch, every inference call, the judge’s verdicts
- download
- this run's artifacts (patches, trajectories, call ledgers, eval reports, judge passes) · preds.jsonl · all runs
- run
- 01788991399526532442-883d2571 · 2026-09-09
- model
- openrouter/minimax/minimax-m2.5 (alias minimax-m2.5-opencode)
- harness
- OpenCode · CLI 1.18.18 (pinned in the harness image build at this framework commit)
- framework
- 04a187d444a3f917b8e8642301f45cc28ed4769c
- harness image
- sha256:b1963fc83e1be63c3b25ca7688fac05a5ed9e6a7518333c7fe3cf6da03880dbb
- benchmark
- SWE-bench/SWE-bench_Verified @ 78f471bf655a · swebench 5.0.2 · images SWE-bench_Verified-78f471bf655a.json
- network
- isolated
- explore
- this run in the Explorer: every instance, its trajectory and patch, every inference call, the judge’s verdicts
- download
- this run's artifacts (patches, trajectories, call ledgers, eval reports, judge passes) · preds.jsonl · all runs