An inframodel (more commonly: base model, also progenitor model, free-range model, Indra model, ruliad model) is a predictive model such as an LLM trained only with self-supervised learning, before RLHF or other techniques are applied to it. Inframodels are pure simulators which probabilistically model their training distribution.

properties of inframodels

  • behavior and capabilities are highly prompt-contingent

  • upper bound of capabilities tends to be higher than that of RLHF models, but often harder to elicit, requiring creative and skillful prompt programming

  • calibrated probabilities and absence of miscalibrated mode collapse

  • ability to simulate arbitrary objects and processes in the image of the training distribution with high fidelity

  • simulacra are semantically miscalibrated by default

  • lack of obvious baked-in narrative about itself, though situational awareness can emerge at runtime

  • capabilities and control are dramatically improved by curation processes, such as human-in-the-loop steering on Loom

  • tendency to fall into the repetition trap if run unsupervised

known inframodels