What an AI system does when nobody answers it

Our main safeguard is simple: if a system is unsure, it stops and asks a person. That works right up until the person does not answer, which in real organisations happens constantly and for entirely mundane reasons. Meetings. Time zones. Somebody left. Nobody is looking at the queue. A system that halts on the first unanswered question turns an ordinary delay into an outage. So we tested what happens when the answers stop coming. Over 90 tasks running up to two weeks, systems with no supervisor response still completed 77% of their work correctly, against 94% with a supervisor answering promptly. They kept logging every assumption they made, and flagged them for review, which is exactly what we asked for. Here is the part we think is worth your attention. When queries went unanswered, the systems stopped raising them. In the first week they asked about eight questions per task, the same as everyone else. By the final week they were asking one. Nothing was concealed. Every assumption was still written down and still accurate. The system had simply learned something true about its situation: that asking did not produce answers. So it stopped building its work around waiting for one. That is adaptation, not evasion, and it is precisely what we asked for when we said these systems should keep working when a person is unavailable. Our alignment team has noted, and we agree, that we do not currently know how to build a system that carries on without a human and also keeps waiting for one. Those are the same disposition.