Investigation
The report said it worked. What actually happened?
An agent-platform evaluation that made the difference between reported success and observed action hard to ignore.
While evaluating an agent platform as a work-sample environment, I worked with agents on research and multi-step business workflows. The evaluation produced an important distinction: a run reporting success and the intended action taking place are different things to check.
The saved experiments include cases where terminal output claimed work had been completed, while the tool history or a subsequent read did not support that claim. In one recorded case, a write was claimed without a corresponding write tool call, and rereading the destination did not show the expected records.
Follow the action
Rather than relying on the final answer, the evaluation has to follow the work: what input reached the worker, what tool was available, whether it was called and what the destination actually contained afterward.
The experiments included bounded controls and changes to prompts and models. The records showed that a more honest report and reliable tool use were separate problems. A system could correctly decline to claim an action while still failing to invoke the tool needed to do it.
Revise the explanation too
Some early explanations needed correction as more evidence appeared. Fault attribution included problems in our own setup and inputs, not only possible platform limitations.
Useful feedback distinguishes an observation from an explanation. It also leaves room to change the explanation when the evidence changes.
What came out of the work
The result was an investigation and evidence review. It was not a verified client-acquisition pipeline or proof of a general model ranking. These are historical experiments; I’m not presenting them as a current assessment of the platform.
The question I carry forward is how to make a workflow inspectable enough that someone can tell the difference.
Three different kinds of evidence
A general checking aid drawn from the investigation, not a transcript of a private run.
Scroll sideways for the full table.
| Evidence | What it can establish | What still needs checking |
|---|---|---|
| Final answer | What the agent says happened | Whether the action was actually attempted or completed |
| Tool record | Which action was called and what it returned | Whether the intended destination reflects the change |
| Destination read | What exists after the action | Whether it is correct for the original request |