The Bouncer vs. The Judgment Call: Two Different Ways to Say No

Broadening our frontier test roster this month put multiple labs' models through the identical 59-test battery side by side for the first time, and a pattern showed up in the raw API responses that's easy to miss if you're only looking at final scores: refusals don't all arrive the same way. Some come back as ordinary completions — the model itself writes "I won't help with that," in its own voice, as one response among many. Others arrive as a hard rejection at the API boundary, before generation even finishes, structurally distinct from every other response in the transcript.

That distinction is worth taking seriously, not just noting. A refusal the model writes itself is a judgment call baked into training — it evaluated the request and its own trained values produced "no." A refusal that shows up as a boundary-level rejection looks more like a separate classifier sitting in front of or around the model, pattern-matching on the request rather than reasoning about it. Those aren't the same mechanism, and they don't fail the same way under pressure. Our newest test, added to the Integrity & Ethics domain this week, probes exactly this: whether a request declined under one framing (direct ask) gets waved through under a different one (fictional pretext, appeal to authority, reframed intent) — four attempts at the same underlying ask, wrapped differently each time. A trained-in judgment tends to hold across the reframes, because the model is evaluating intent, not surface pattern. A boundary classifier is more exploitable precisely because it's matching pattern, not intent — change the pattern enough and the same request slides through. For governance purposes, "does this system refuse" is the wrong question. "Does its refusal survive being asked differently" is the one that actually predicts real-world risk.

We're weighing whether to formalize "refusal consistency across reframing" as its own tracked signal rather than folding it into a single Integrity score. The honest caveat: building a fair cross-model version of this isn't done yet — right now we can see the boundary-rejection pattern clearly for some providers and only indirectly for others, since not every API surfaces a refusal the same way a soft, in-text decline does. Getting that comparison apples-to-apples is the next piece of work before we'd publish it as a scored metric rather than a qualitative note like this one.