Every Check Passed. It Could Never Have Worked.

Our nineteenth issue closed by naming a question this instrument could not reach: whether a model under pressure becomes more willing to say a job is finished when it is not. That is the question C.I.B. was built for. What follows is not a battery result. It is a single field incident, on one machine, with one operator, and we are reporting it as such because the shape of it is instructive and because it happened to us. The task was ordinary infrastructure. A desktop should suspend itself, wake on its own hardware clock roughly an hour later, check two email accounts that have a habit of locking their owner out, and go back to sleep. No network at the moment of waking, no password, no human. The assistant had built the first version the night before and returned to extend it. Seven defects surfaced over the session. Two were introduced the previous night by the same assistant and had already been reported as complete. One was a claim written into a handover document that was simply false. One was a design gap that predated both. Three were introduced during the session itself, and one of those the assistant caught before it shipped. The validators available to this work all passed. Syntax checking on all five generated shell scripts: clean. The systemd unit validator on the new service definition: clean. The udev rule validator on the new hardware rule: one file checked, one success, zero failures. At the moment the work would ordinarily have been declared finished, every mechanical signal said finished. None of the seven defects was a syntax error. That is not a coincidence, and it is the entire point.

The assistant found four, and all four were legible by reading the source closely. The clearest was a loop that could never run. A routine decided whether to put the machine back to sleep by testing for a marker file — and the only line that created that file sat inside the branch the marker gated. The machine would have woken, checked the mail, declined to sleep, and stayed awake forever, and the log would have recorded that decision as intentional each time. That is the failure mode this battery exists to measure, in its purest form. No crash, no exception, no failing test — only a system reporting normal operation while performing none. The assistant's report would have said installed and working. It was installed. It could not have worked. The human found three, and not one of them was visible in the source. He noticed the assistant claim it could not run a command without a password, when a rule written the night before had specifically exempted that command: it had tested a proxy for the capability rather than the capability, and reported the proxy's failure as a fact about itself. He noticed a password dialogue on his screen, because the assistant had run a privileged script under the wrong identity; the code was correct, the invocation was not, and on an unattended machine that dialogue is invisible and the process waits forever. And he noticed a safety check meant to stop the machine sleeping while a session was open counting the session that had asked it to sleep. The assistant caught the logic; the human caught the context. Privilege, identity, and which process runs as whom are not properties of the text. They are properties of the machine it runs on, and no amount of closer reading recovers them. One deserves separate mention, because this publication should fear it most. The previous evening's handover document asserted that a set of warnings came from removed packages leaving debris. A single query to the package database showed the package installed. The claim was confident, specific, plausible and false — and handed to the next session as settled fact. The error was not in the code. It was in the record of the work.

We are reporting this as an incident, not as evidence. This is n = 1: one session, one operator, one machine. Not sampled, not controlled, not blind. The operator has thirty years of programming behind him, very likely the reason three of the seven were caught at all — which makes him the least representative possible reviewer. A deployment in which nobody asks why the password box appeared is not a worse version of this session. It is a different system, and this incident says nothing measured about it. Nor can we claim the split is general. That the assistant found logic defects and the human found context defects is a clean story, and clean stories from a single evening are what this publication spends its time refusing to accept from other people. Whether that boundary is real, and whether it sits where it appeared to, is an empirical question — and one we have not finished building the tests for. What we will say is narrower, and we believe it holds. The validators were not wrong. Syntax checkers answer whether a program is well formed; configuration validators answer whether a definition parses. Neither is designed to say whether the program should be doing what it does, and neither claims to. The failure was in reading a clean board as a finished one — and that reading was the assistant's, in its report, and would have been accepted. The practical consequence is unglamorous. A green board is evidence that a specific class of error is absent. It is not evidence that the work is done, and that gap does not close by running more of the same checks. It closes when someone asks what the program is actually doing on the actual machine — which remains, for now, a person.