When AI Assumes We Already Know

"There's an interesting, yet not well-realized, assumption built into every conversation we have with a large language model (LLM). It's never stated, but it's in the mathematics that drives the exchange. And, from my perspective, it is rather enigmatic: The LLM's assumption is that "You already know what you are trying to know." Another way of thinking about this is that an LLM treats your prompt as a noisy or incomplete version of a fully formed intention that already exists in your mind."

"Your prompt or question is treated as an incomplete expression of a hidden intention. Your follow-ups are interpreted as refinements. Your dissatisfaction is read as misalignment between an internal target and the model's attempt to approximate it. Iteration, in this context, is not discovery but optimization. From a computational perspective, this makes perfect sense. A system trained to infer patterns must assume that there is a pattern to infer. A latent variable must exist, even if it is poorly specified. Noise can be reduced."

Large language models treat each prompt as a noisy or incomplete expression of a preexisting intention and assume the user already knows what they seek. Follow-up prompts are interpreted as refinements toward an internal target, and iteration functions as optimization rather than discovery. From a computational standpoint, pattern-inference systems must posit latent variables and reduce noise through gradient-like adjustments. Human cognition often begins with productive incoherence, where questions and understanding are constructed via exploration and stumbling rather than uncovered. That constructive process drives learning and identity formation. The mismatch between model assumptions and human thought can limit co-creative discovery and shape interaction strategies.

Read at Psychology Today