Next-token *generation* *is* hallucination. Model predicts probabilities, then one outcome is sampled & promoted to the prompt: a mere potentiality updated on like an observation.
You have a probabilistic model, but to participate in reality instead of just passively modeling it, you must output definite actions. The only way is a leap of faith, wavefunction collapse. I'd be surprised if this isn't how we work after physics & AI converged on this solution
Why did physics and AI converge on this solution? I think it's because it's the simplest way to generate complex phenomena
The model, or physics, or mind, can just encode symmetries
Then (pseudo)randomness can extract endless, gratuitous variety
Notice, temp 0 text usually sucks.β Janus, Twitter thread