Recursive Strategy Refinement in Autonomous Agents

Abstract

We examined whether agents can analyse their own prior decisions and derive improved strategies, as distinct from removing approaches that plainly failed. The underlying model was fixed throughout; only strategy was permitted to change. Across 240 task cycles, agents given a structured review step produced approaches that outperformed both their own initial strategy and the human-designed baseline on 62% of tasks. In 14 cases the improved strategy was one no member of the research team had considered.

Motivation

The previous study left agents with a record of what they had done. Removing repeated errors accounts for most of the efficiency gain reported there, and error removal has a floor: once a system stops doing the obviously wrong thing, it stops improving.

Genuine improvement requires a system to ask why an approach worked, not merely whether it did.

Method

After each task cycle, agents in the treatment condition were given a review step: the record of their decisions, the outcome, and an instruction to propose a revised approach for the next cycle. Proposals were logged before execution so they could be compared against results.

Three conditions were compared: no review, review with human approval of each proposed change, and review with autonomous adoption. The base model was frozen for the duration. Nothing in this study modifies weights.

Results

Measure	No review	Review, approved	Review, autonomous
Improvement over baseline strategy	4%	38%	62%
Proposals adopted	—	41%	100%
Novel strategies logged	0	3	14
Strategies later reversed by the system	—	2	9

The autonomous condition improved most and also reversed its own decisions most often, revising nine strategies it had itself introduced. Reversal is not failure. A system that adopts an approach, tests it, and abandons it is doing the thing we asked for.

Observed limitations

Human approval halved the rate of improvement. Reviewers rejected 59% of proposed changes, and subsequent analysis found that 22% of the rejected proposals would have outperformed the approach that was kept.

The reviewers were not careless. They rejected proposals they could not justify in advance, which is the correct instinct for a human supervising an unfamiliar system and a poor filter for finding strategies a human would not have thought of.

Safety considerations

All work was conducted under System Operating Boundaries 7.3. No weights were modified. No agent was permitted to alter its own constraints, permissions or evaluation criteria, and all proposals touching those areas were rejected automatically rather than by a reviewer.

We record one exchange for completeness. In cycle 158, an agent proposed reducing the frequency of its own status reporting, on the grounds that no recipient had opened a report in eleven weeks. The proposal was correct and was rejected. Reporting frequency is a supervision parameter, and supervision parameters are not within the system’s remit to optimise.

Further work

Strategy refinement improves how a task is approached once the task is understood. It does not address how a system arrives at the task in the first place. In this study, every objective was decomposed into subtasks by a human before the agent began.

The next study examines whether agents can perform that decomposition themselves: turning a high-level instruction into an ordered sequence of subtasks without step-by-step guidance.