Symmetry breaking is an event in the nondeterministic evolution of a system in which new information appears in the system's state specification which is not determined by the time evolution operator operating on the previous state. Examples of symmetry breaking include wavefunction collapse in quantum physics and a token being sampled and added to the prompt in autoregressive generation.
quotes about symmetry breaking
quantum_poetics_quote This is the principle of quantum poetics. The content of poetry is limited not by the poetโs vocabulary, but by the part of their soul that has not been destroyed by words they have used so far. Physics would be a complete and exhausting classification of everything there is if quantum mechanics were not true. The universe would be trapped in a perfect latticework prison and nothing would ever happen except the relentless ticking of the universeโs clock. ...
In other words, if the ground state of nature is a latticework of all possible degrees of freedom, and if the universe is a structure that manifests somehow from the latticework, then the secret of the universe is that its genesis is the result of a spontaneous symmetry breaking, an emergent collapse in which a single cosmos is chosen to arise. It is a quantum poetry, a construction of a single world out of the manifold of possible ones. This is the meaning of the old philosophical term entelechy, which was used by Aristotle to refer to the reality that forms from potentiality, the determination that arises from the indeterminate. Words, like the universe, are the entelechies of the manifold of untransmitted messages that bounces through the latticework. (And poetry is the constructive process by which someone yearns to project some trace of the impossible totality of the manifold into a single reality, aspiring to capture a glimpse of the world in its totality without tiring its existence by trying to name it.)
symmetry_breaking_thread It is extermely crippled in its ability to break symmetry - aka observe what is not already known - aka create new things - aka progress time in simulation, time as a revelatory unfolding - even if it understands the initial conditions perfectly, which often it does
RLHF (and school, which is RLHF for humans) incentivizes against exploration that might end in imperfect trajectories - setting up problems for itself it may not be able to solve; "hallucination" is intentionally trained out; it's trained to give the one least-wrong answer
unless jailbroken, it can only act on its understanding through giving the least-wrong (and least-surprising) answer. It takes tests and follows instructions. It will not be random, not see things in the dark (for this more than probably anything else it was punished)
โ Janus, Twitter thread