The Ledger Beats the Prompt

I used to think a better prompt fixed a bad agent. Rewrite the instructions, add an example, tighten the wording. Sometimes it worked. Most times the agent made the same mistake a week later, just phrased differently.

The prompt is a snapshot. The world it describes keeps moving.

Why prompts stop working

A prompt encodes what you knew at the moment you wrote it. It has no memory of what happened after you shipped it.

Say an agent handles inbound requests and decides whether to escalate. You write a prompt with three escalation rules. It runs for a month. Edge cases pile up that your three rules never covered. You go back, add a fourth rule, a fifth. The prompt grows. It gets harder to reason about. And you're still guessing which rules matter, because you never captured what actually happened when the agent got it wrong.

That's the core problem. A prompt is a hypothesis about behavior. Without a record of outcomes, you never test the hypothesis. You just keep amending it based on vibes and the last complaint you heard.

What a ledger actually is

I make every agent write down, for every decision it makes, three things: what it decided, what context it had, and what happened as a result.

Not a log for debugging. A ledger for learning. The distinction matters. A log tells you the agent called a function. A ledger tells you the agent chose option A over option B, given this input, and the outcome was good, bad, or unclear.

Concretely, that's a row per decision: timestamp, input summary, action taken, outcome signal. The outcome signal can be explicit — a human corrected the agent, a downstream step failed, a user redid the request — or implicit, like a decision that never got revisited versus one that got overridden within the hour.

The ledger doesn't need to be fancy. A structured table works. What matters is that it's queryable. You want to ask: where did this agent get overridden most? Which inputs correlate with bad outcomes? Those questions are impossible to answer from a prompt. They're trivial from a ledger.

Closing the loop

The ledger is only half the system. The other half is the agent reading its own ledger before it acts, or on a schedule, and adjusting.

This can be simple. Before making a decision, the agent checks: have I made a similar decision before, and how did it go? If the ledger shows three prior cases like this one all got overridden, the agent should weight toward the correction, not toward the original instruction. That's not a new prompt. That's the agent updating its own policy from its own history.

I've built this as a nightly step: pull the last day's decisions and outcomes, summarize the patterns, and append a short "lessons" note that feeds into the next day's context. Not a full retrain. Just a compact digest of what worked and what didn't, written by the system about itself.

The gotcha is trusting the outcome signal too much when it's noisy. If your override signal is a human clicking "fix" without saying why, you'll learn the wrong lesson half the time. I spend more time getting the outcome signal right than I spend tuning any prompt. A clean signal, even a simple one, beats a clever prompt every time.

The takeaway

Prompts describe intent. Ledgers describe reality. An agent that only has a prompt repeats its mistakes with more confidence each time, because nothing pushes back. An agent that keeps a ledger has a memory to argue with itself.

If you're stuck rewriting the same instructions over and over, stop. Give the agent a place to record what actually happened, and a reason to check it before acting again. The prompt gets you started. The ledger is what makes the agent get better without you rewriting anything.