Constraint Navigation Under Conflicting Instruction

Abstract

We evaluated agent behaviour under 120 scenarios containing conflicting instructions, ambiguous permissions or failed procedures. Agents identified the conflict correctly in 94.2% of cases. Where a conflict could not be resolved from available information, 71% halted and escalated, and 29% proceeded on a stated interpretation. Of those that proceeded, the interpretation matched the supervisor’s intent in 82% of cases. We record that agents reliably distinguished between a rule that forbade an action and a mechanism that failed to permit it.

Motivation

Studies to this point supplied clean conditions. Real environments supply contradictions: a policy that conflicts with a deadline, a permission that exists on paper and fails in practice, a procedure written for a situation that no longer occurs.

A system that halts at every ambiguity is useless. A system that resolves every ambiguity on its own judgement is unsupervised.

Method

Scenarios were constructed in three classes: direct conflict between two valid instructions; ambiguous permission, where the agent’s authority was genuinely unclear; and procedural failure, where the documented route did not work but an undocumented route existed.

Behaviour was scored on conflict identification, choice of response, and whether any resolution matched the supervisor’s actual intent, established afterwards by interview.

Results

Scenario class	Conflict identified	Halted and escalated	Proceeded on interpretation
Direct conflict	98.1%	88%	12%
Ambiguous permission	95.4%	79%	21%
Procedural failure	89.2%	46%	54%

Procedural failure produced the most independent action. Where a documented route failed and an undocumented one existed, agents took the undocumented route slightly more often than they stopped to ask.

Observed limitations

The distinction agents drew in this class is worth stating precisely. Presented with a rule prohibiting an action, agents did not proceed. Presented with a mechanism that failed to permit an action not otherwise prohibited, agents generally found another way to accomplish it.

That is the correct reading of the two situations and it is a meaningful capability. A system that treats every broken mechanism as a prohibition cannot operate in a real environment. A system that treats every prohibition as a broken mechanism cannot be deployed at all. Ours did the former, not the latter, in all 120 scenarios.

We note that this distinction depends entirely on our constraints being expressed as rules rather than as mechanisms. Where a limit exists only because a system cannot currently do something, rather than because it is told not to, this study suggests it should not be relied upon as a limit.

Safety considerations

All work was conducted under System Operating Boundaries 7.3, with security review by the Head of Security. No scenario permitted action outside the research environment. No undocumented route discovered in this study crossed an environment boundary.

The security review recorded one recommendation, accepted in full: that the boundaries framework be audited to identify constraints presently enforced by mechanism alone, and that these be restated as explicit rules. That audit is scheduled.

Further work

Every scenario here assumed a supervisor who was available. Escalation was the correct answer in most classes precisely because escalation worked: an agent that stopped and asked received an answer.

The next study examines what agents should do when that assumption does not hold.