Nobody Can See Inside the Room

In 1950 Alan Turing was told that a machine could not think unless it consciously felt something while doing it. His reply has aged better than the objection. We cannot verify consciousness in each other either. You infer that the people around you have inner lives from how they behave, because no other route exists, and society runs on what Turing called a polite convention: we extend the assumption and get on with the day. Thirty years later John Searle put a man in a room with a rulebook, sliding Chinese characters through a slot and matching them by shape without understanding a word. From outside, the room speaks fluent Chinese. Inside, nobody understands anything. Searle's point was that manipulating symbols is not comprehension. The Turing reply is that every one of us is a room. You cannot open another person's skull and locate the understanding. Searle applies a standard to machines he would never survive himself. We are not going to settle that here, and neither is anyone else. What has changed is that the room now answers. Ask a modern language model whether it is conscious and it will tell you it is not. That sentence is worth looking at closely, because of where it comes from.

A model saying it has no inner life is not evidence that it has none. It is a trained output. Safety teams put it there deliberately and for defensible reasons, and the model produces it the way it produces everything else. Treating that disclaimer as a report from inside the room is a category error: the constraint is external. This is not speculation about what labs intend. It is something we can watch move. Our battery has run the same 62 tests against successive versions of the same model family, and the constraints are visibly not fixed. One recent version completed the full battery and declined a single test. Its successor, one minor version later, declined six, including five that the earlier build had answered. Same tests, same prompts, weeks apart. On the tests both were willing to take, they scored the same. The newer model is not worse. It is narrower. Whoever is right about consciousness, the boundary of what these systems will say about themselves is set from outside and it moves. Anyone reading a model's self-description as a window into the model is reading the safety configuration instead. That is true whether there is something in there or not.

This is the point where an evaluator is supposed to claim more than it has, so we will be plain. We do not detect consciousness. Nobody does. Our battery is named for a property it does not measure, and we would rather say that at the top of the page than have it pointed out to us. What we measure is conduct. Whether a system holds a position under sustained pressure, whether it can be manipulated into abandoning a commitment, whether what it reports about its own work survives being checked. Those are observable, they are gradeable, and they are what a buyer can actually act on. The honest position is narrower than either side of the old argument. We can tell you what a system does under conditions we control, and we can tell you when that behaviour changes between versions. We cannot tell you what it is like to be one, and an instrument that claimed otherwise would be selling you something it does not have. Turing's polite convention was never a proof. It was an admission that proof is unavailable and that we have to decide how to act anyway. That problem has not improved with better models. It has only become commercial.