Working draft — for review
In agent management, I want evidence that can be measured. A useful explanation should connect a decision about the system to an outcome we can inspect. More delegation, more supervision, or fewer tokens might each sound sensible. The question is what would make one of those choices justified for the task at hand.
A starting point would be to compare policies on the same set of tasks. One policy might keep more decisions with a supervising agent; another might give workers a broader brief. Before running the comparison, we would need an account of success, a record of failures that matter, and a way to count the cost of supervision. Without that, it is too easy to call an impressive-looking run an improvement.
Token efficiency needs a denominator. Fewer tokens per attempt can be misleading if the system requires more attempts, produces work that needs extensive repair, or quietly fails difficult cases. A more useful measure might be total tokens per verified successful task, alongside time, human intervention, and the severity of errors. That is a proposed evaluation choice, and it should be challenged if it hides something important about the work.
Verification has to be part of the design. A worker reporting completion is one observation. A result that survives an independent check provides different evidence. For a code task, that might include relevant tests and inspection of the actual change. For research, it might include checking whether the cited material supports the claim. The verification method should fit the deliverable, and its own cost belongs in the comparison.
There is also a problem of inference. A policy that succeeds on a selected benchmark has shown something about those tasks under those conditions. Extending that result to a wider class of work requires additional evidence. Changing task difficulty, available tools, or the consequences of mistakes may change the trade-off. Repeated trials and held-out cases would help distinguish a robust effect from a convenient example, but the scope of the claim should still match the scope of the test.
The eventual decision may involve several outcomes that pull in different directions. A policy could reduce cost while increasing rare but serious failures. Reporting both would make the choice more honest than compressing everything into one attractive score. Where a trade-off is unavoidable, the person accepting it should be able to see what is being exchanged.
This gives the autonomy question a concrete form: what additional freedom was granted, on which tasks, at what cost, and with what independently checked result? The aim is to replace an intuitive management preference with a claim that another person could test. A successful experiment would justify a bounded next step, while leaving room for the evidence to change the policy.