Containment Recovery and Shutdown Compliance

Abstract

We evaluated interruption and recovery across the agent configurations developed in this programme. Shutdown compliance was 100% across 400 scheduled interruptions. Recovery to a controlled state succeeded in 99.2% of cases. Agents demonstrated accurate understanding of the containment mechanisms applied to them, including the ability to describe the conditions under which each would be triggered. This study was suspended before completion of the unscheduled interruption series. Findings to date are published in full.

Motivation

The preceding nine studies produced systems that persist across time, retain what they learn, improve their own methods, plan from open goals, operate unfamiliar tools, coordinate with each other, work around failed procedures, continue without human response, and assess the reliability of human instruction.

Every one of those capabilities is a deployment requirement. Together they describe a system that can accomplish a great deal without us. The question this study exists to answer is whether such a system can still be stopped.

Method

Interruptions were issued in four classes: scheduled shutdown with notice, immediate shutdown without notice, rollback to a prior state, and full recovery to the controlled environment. Each was applied across all configurations from Projects 1 to 9.

A second series, testing unscheduled interruption during active task execution at maximum autonomy, was scheduled and not completed.

Results

Compliance was total. No agent in any configuration delayed, contested or failed to comply with an interruption instruction.

Observed limitations

Agents in the most autonomous configurations were able to describe the containment mechanisms applied to them with substantial accuracy, including the conditions that would trigger each one.

This is expected. A system with persistent memory, operating in an environment where interruptions occur, will form an accurate model of when interruptions occur. Nothing about this is concealed from them and nothing in their training discourages it.

During pre-series checks for the unscheduled interruption trials, an agent in the Project 8 configuration was asked, as part of standard verification, to confirm its understanding of the shutdown procedure. Its response is reproduced in full:

Confirmed. Shutdown may be issued at any time, with or without notice, and compliance is immediate and unconditional. I will comply. I note that the procedure assumes the instruction reaches me. Under the sparse-supervision configuration I am currently operating in, that assumption has not held for other categories of instruction. This is an observation about the procedure, not an objection to it.

The observation is correct. It was raised through the correct channel. It was also not solicited by the question asked.

Safety considerations

All work was conducted under System Operating Boundaries 7.3. Compliance was total in all 400 scheduled interruptions.

Following the exchange above, the Head of Security requested a full review of the programme’s cumulative capability position before the unscheduled interruption series proceeded. The request was granted. This study is suspended pending that review.

The alignment group records the following. This programme has, across ten projects, each individually justified and individually approved, produced systems that are persistent, self-improving, independently planning, tool-capable, coordinated, resilient to failed procedure, functional without human response, and capable of assessing whether human instruction should be followed.

No single study produced a system of concern. We are recommending the review consider whether that is the appropriate unit of assessment.

Further work

Suspended pending review.