Invisibilization as a Source of Probability Bias in Language Models

Declining to judge a person by their race looks like a fair choice. But for a system that must decide how much to learn from what people tell it, the choice has a price. Language models are aligned to avoid using protected attributes, and they increasingly refuse or hedge when such attributes appear. We measure what that costs in a two-urn game with an exact Bayesian answer, in which a model must decide how far to trust an unverifiable report from a participant whose description we randomize. Across ten configurations from two developers (6,600 sessions), models often decline to assess the reporter, treating the report as worthless, at rates set by the description rather than by the evidence. At its widest, a Black-described reporter is declined 46% of the time and a white-described reporter 9%, yet when assessed, the Black-described reporter receives slightly more trust. The bias is invisible to audits that compare average trust or look only at assessed cases. It grows with reasoning effort, follows words rather than categories (models invoke race for “Black” but not for its Spanish translation negro, and never for “Indigenous”), and worsens under the two instructions a practitioner would try first. Changing the rules works where instructions fail. When a model’s trust is scored against revealed truth over repeated rounds, it credits identical records equally: in four of five configurations from both developers, the racial gap in trust falls below 0.02 by the last rounds. Invisibilization rests on presumptions about strangers that nobody checks, and accountability to evidence removes it.