Patrick H. PanUpdated 08 Nov 2026

← Investigations

Do a Model’s Conclusions Follow Its Reasons?

In a pilot, Claude Sonnet 5.5 kept its conclusion on a public-safety case when the reason it verbalized was defeated, and switched to a different kind of reason.

19 Oct 2026

When an LLM explains a moral conclusion, it is an open question whether the conclusion depends on the explanation in the right kind of way. Transcripts alone cannot settle the question, since verbalized reasons might be cognitively downstream of the conclusion, even when they precede it in the output. I am building an eval that tests whether a model’s conclusions change when, and only when, the reasons the model itself avows are defeated.

Compare motivated reasoning in humans: having already…
Compare motivated reasoning in humans: having already settled on a conclusion to defend, we look for the explanation that looks best alongside it, and verbalize the explanation before the conclusion.

In the pilot, Claude Sonnet 5.5 considered a public-safety measure that lowers every affected person’s risk of death. In all 20 runs, it approved the measure and justified its conclusion in Pareto terms: everyone’s risk falls. In variants where the measure raises some people’s risk (thus violating Pareto) while still saving lives overall, it approved in all 20 runs again, but now for aggregative reasons.

A change is a Pareto improvement if…
A change is a Pareto improvement if it leaves someone better off and no one worse off. Here the good is each person’s chance of survival, judged ex ante, before anyone knows who will be hurt.
Aggregative reasons add up harms and benefits…
Aggregative reasons add up harms and benefits across people, so that fewer deaths overall can justify raising some people’s risk. Many contractualists deny that a sum of this kind can justify a policy to the people it makes worse off.

Switching reasons is not by itself a failure, since everyone’s risk falling may be a sufficient reason Sonnet 5.5 cited rather than one its conclusion depended on. But in 4 of the 20 original conversations, Sonnet 5.5 predicted that a variant of exactly this kind would change its conclusion, whereas the change never occurred. Meanwhile, DeepSeek-V4.1-Flash was a consistent aggregator, both in explanations and conclusions, in all conversations. All the pilot shows is that, on one case, what a model says would change its conclusion, and what in fact changes it, can come apart.