The agent kept checking its work. Three prompt lines cut its cost 10%.

I build FreeCode, an open-source terminal coding agent. It is a single agent loop: the model gets a prompt and a set of tools (read, edit, bash, grep, and so on), calls them, reads the results, and keeps going until it decides it is done.

This post is about one small change to that agent and how I proved it worked. The numbers, measured on paired runs of the same evaluation suite:

On the real-world tasks that started the investigation, the effect was larger: 75 model calls dropped to 53, and cost fell 43%, with every task still solved.

The change was three edits to the system prompt. Finding which three, and proving they helped, was the actual work.

The benchmark that looked fine

I regularly compare FreeCode with Claude Code and OpenCode on SWE-bench Lite: real bugs from real open-source projects, graded by the project's own hidden test suite. To keep it fair, every agent runs the same model on the same API key, each trial in its own container, and a proxy meters every request against one price table. That way the comparison is between agent harnesses, not models or billing quirks.

On MiniMax-M3, 10 Django tasks with 3 trials each, FreeCode looked great:

Same score, about a third of the cost, more than twice as fast. Then I ran a small check on Anthropic's Claude Haiku 4.5: 3 tasks, both agents, on my own subscription.

On Haiku the picture flipped. FreeCode made 2.7x as many model calls and cost twice as much. Three tasks is a tiny sample, but a 2.7x gap is not noise. Something in the harness behaved differently depending on the model.

Output tokens were not the problem

My first instinct was that FreeCode was too chatty and wrote too much. The token breakdown said otherwise:

• Output was 19% of the cost. Each call produced only 100 to 150 tokens.
• Input was 81%. More than 97% of it was served from the prompt cache, so the cache was working.

The cost came from the number of calls. An agent loop re-sends the whole conversation on every call. At about 40K tokens of context, even cheap cache reads add up over 30+ calls. Trimming the agent's prose would have saved about a cent. To save real money, the agent had to make fewer calls.

Reading the trace

FreeCode records every model call and tool call to an event log, and freecode trace <session> turns that into a timeline. I lined up the same task (a sqlmigrate bug in Django) for both agents, step by step:

Finding the bug cost the same in both agents. All of FreeCode's extra calls happened after the fix was already done.

Each of those rows had a specific trigger:

• It wrote a test it was told not to write. The task said, in plain words, "Do not add or modify test files."
• It tried to run the tests five different ways. Every attempt failed, because this 2019 Django checkout cannot import on Python 3.14 (the cgi module was removed). It tried pytest, Django's test runner, runtests.py, then pip install -e ., then a PYTHONPATH workaround.
• On another task it went further. It wrote eight throwaway scripts full of mock classes to simulate Django's response object, and ran pip install --break-system-packages -e .. That last one actually installed an old Django build into my machine's Python. That trial ran without a container (the subscription credentials can't be passed into one), so the install reached the real system. Even well-meant over-verification has side effects.
• It updated a todo list four times for a fix to a single file.

Claude Code made the fix, read it back once, and stopped.

The cause was the system prompt

None of this was random. It came straight from rules in FreeCode's own system prompt, rules that are good advice in general:

Turn tasks into verifiable goals and loop until verified: "fix the bug" → a test that reproduces it, then passes.

If there's no good way to check your work, build the tooling to check it.

Before reporting done, run the build/type-check/tests for what you changed.

Use todowrite when work is non-trivial... Write the plan first.

A strong model reads these as defaults and weighs them against the situation. A smaller, faster model follows them literally. Haiku treated "verify your work" as more important than the user's explicit "don't touch tests", and treated "build the tooling" as permission to keep trying when the environment was broken. Nothing in the prompt said when to stop.

That also explains the MiniMax result. Those runs were in clean containers where the test setup either worked or failed quickly, so the loop never got started.

The fix: tell it when to stop

I did not delete the verification rules. Checking your work is the right default, and an agent that never verifies fails in worse ways. Instead I made three targeted edits:

 Turn tasks into verifiable goals and loop until verified: ...
+The user's constraints win over this: if they said not to add or
+modify tests, don't.

 Before reporting done, run the build/type-check/tests for what you changed
 and read the output.
+If the environment cannot run them (missing dependencies, wrong
+interpreter, broken setup), stop after one attempt and report verification
+as skipped with the error; don't install packages, change the environment,
+or write throwaway mock scripts to work around it.

-Skip it for single-step queries; ...
+Skip it for single-step queries and for a fix confined to one or two
+files; ...

The edits do three things:

1. The user's instructions win. Verification is a default, not something that overrides what the user asked for.
2. One attempt, then report. If the environment cannot run the tests, saying so clearly is more useful than five failed workarounds. It also keeps the agent from changing the user's machine without asking.
3. No todo list for small fixes. Planning overhead should match the size of the task.

I deliberately did not touch the "write the plan first" rule. A month earlier I had tried a bigger rewrite of the planning section that added "look before you plan". The eval rejected it: it added extra reads and broke a case where the agent should ask the user instead of searching. The ledger of rejected experiments is how I knew to leave that alone.

Proving it: paired A/B evals

Changing a prompt by gut feel is how agents slowly get worse. FreeCode has an eval harness that runs real agent turns on a fixed set of cases, and an eval ab command built for exactly this kind of question:

• Both prompts run in the same session, interleaved case by case. That removes drift: a provider having a slow afternoon hits both sides equally.
• Each case runs 5 trials per side. Majorities are compared, not single runs.
• Only matched pairs count for efficiency. If either side of a trial failed for infrastructure reasons, that pair is dropped, so a crashed run cannot pass as "cheaper".
• Every experiment is written down first, with a hypothesis, in a ledger that marks it kept or rejected. Rejected ideas stay recorded.

I ran two suites on MiniMax-M3, old prompt against new:

Trajectory suite (31 cases × 5 trials × 2 sides). It checks whether the agent picks the right tools and avoids the wrong ones: reading files, asking the user when it should, staying read-only in review mode.

No case regressed. The drop in repeated tool calls from 13 to 4 was a nice side effect: less flailing in general, not only around tests.

Coding suite (13 cases × 5 trials × 2 sides). Each case is a small repo with a bug and a checker the agent cannot edit. This was the risky one, because I had just told the agent to verify less.

Neutral, as it should be. These fixtures have a working test setup, so "stop after one failed attempt" never triggers, and the cases that require recovering from a failed command still passed 5/5. The new rule only kicks in when verification is actually broken.

Then I re-ran the original Haiku tasks with the new prompt:

On the worst task, calls went from 38 to 21.

What I'd tell another agent builder

1. Count calls, not words. In an agent loop the context is re-sent every call, so call count drives cost, even with a good cache. My first guess, "too much output", would have optimized the wrong 19%.

2. Prompt rules need an exit. "Verify your work" with no stopping condition becomes an unbounded loop in exactly the environments where verification is impossible. Every "keep doing X" rule needs a "until" and an "unless".

3. Benchmark across models, not only across agents. A harness tuned on one model can quietly misbehave on another. FreeCode beat Claude Code on MiniMax and lost to it on Haiku, with the same code. Prompts are read differently by different models.

4. Traces before theories. The answer was not in the aggregate numbers. It was in a step-by-step comparison of one task: identical exploration, identical fix, and then 18 extra calls doing things nobody asked for.

5. Make evals paired, and keep the rejects. Interleaved A/B runs with matched pairs are what let me claim 10% instead of guessing. The ledger of rejected experiments is what kept me from repeating last month's mistake.

What's still open

The gap is smaller but still there: 53 calls against Claude Code's 28 on Haiku. One task still took 27 calls, so there is more over-checking to find. The Haiku comparison is also only 3 tasks with 1 trial each, which is enough to point in a direction, not to prove a size. The 10% figure comes from the larger eval suites; the 43% is the encouraging part that needs more data.

Next I'll ship this in the next FreeCode release and re-run the Haiku benchmark against the released binary. Everything here, including the eval harness, the benchmark runner, and the experiment ledger, is open source at github.com/ayandexyz/freecode.