World Models Explained: The Architecture Beyond LLMs
Every few months a new system gets called a "world model" — Genie generating playable game worlds from a single image, Sora producing minutes of physically plausible video, Meta's V-JEPA learning from raw video without labels, robotics labs training arms entirely inside a learned simulation. It is tempting to treat "world model" as a rebrand of "big generative model," but the term names a specific, older idea from control theory and reinforcement learning: a model that predicts what happens to an environment's state if you take a specific action — not what word comes next in a sentence. That distinction is not academic. It determines whether a system can be used to plan, and it explains why a model that produces gorgeous video can still fail badly the moment you ask it to support a real decision.
This guide assumes you understand next-token prediction in a transformer — see Transformer Architecture and Self-Attention Explained if you need that first — and treats language models as the point of contrast, not the subject. You will learn the encoder–predictor(–decoder) shape every world model shares, the real architectural split between predicting raw pixels and predicting compressed latent state (JEPA-style), how a world model is used to plan an action before taking it, and the specific failure mode — compounding rollout error — that makes "trust the model's imagined future" a genuinely risky sentence in robotics and autonomous systems.
What a world model actually predicts
A world model is a learned function with three moving parts: an encoder that turns a raw observation (a camera frame, a sensor reading) into a compact internal state; a dynamics model (also called a predictor or transition function) that takes the current state plus a proposed action and predicts the next state; and, optionally, a decoder that turns a predicted state back into something you can look at, like a rendered frame. The dynamics model is the part that earns the name — it answers "if I do this, what happens next?" without anyone actually doing it. This idea predates the current generative-AI wave: Ha and Schmidhuber's 2018 "World Models" paper and Hafner's Dreamer line of work built agents that learned a compact latent dynamics model of a game or robot environment, then trained a policy entirely inside imagined rollouts of that model — never touching the real environment during policy learning.
For the rest of this guide, use one running example: a warehouse robot arm with a camera, watching a box on a table. The state is the box's position and orientation; the action is "push the box 10 cm to the left"; the model's job is to predict the box's position after that push, before the arm actually moves. A language model has no equivalent action slot — it predicts the next token given prior tokens, with no notion of an action changing an external environment's state. That is the categorical difference: an LLM models a sequence; a world model models a controlled dynamical system, where the thing being predicted depends on a choice the agent is about to make.
Quick reference
- Definition: a world model predicts next state given (current state, action) — planning is then a search over actions using the model's own predictions, not the real environment.
- The classic shape is encoder → dynamics/predictor → optional decoder; the dynamics model is what makes it a world model rather than a plain autoencoder.
- Model-based reinforcement learning is the origin field — Dreamer-style agents train a policy by rolling the learned dynamics model forward in imagination, then only occasionally check the result against the real environment.
- "State" does not have to mean pixels — it is usually a compressed vector; how compressed, and what it discards, is the entire design argument in the next section.
- A world model without an action input is just a video predictor — useful for generation, but it cannot answer "what if I intervene," which is the property planning needs.
Remember this
A world model predicts the consequence of an action on an environment's state; an LLM predicts the next token in a sequence with no action — that missing action slot is why LLMs alone cannot plan physical behavior, only describe it.
Two designs: render the future vs compress it
Once you accept the encoder–predictor shape, the open design question is: what should the predicted state actually represent? One school renders the future — predict the next camera frame (or several seconds of video) pixel by pixel, autoregressively or via diffusion. Genie's playable worlds and Sora-style video generation sit in this camp: the output is directly watchable, which makes errors easy to spot by eye, but the model spends real capacity predicting details that do not matter for any decision — the exact dapple of light on the box, sensor noise, a shadow's edge. Those details are often close to unpredictable in principle, and a model that tries to nail them anyway wastes parameters and compute on noise instead of structure.
The second school, Joint-Embedding Predictive Architecture (JEPA) — proposed by Yann LeCun as a deliberate alternative to next-token and next-pixel prediction — predicts the future directly in an abstract embedding space instead of pixel space. The encoder for the predicted frame and the encoder for the actual future frame are trained so their embeddings agree, without ever forcing the model to reconstruct pixels. Concretely for the warehouse example: a JEPA-style model predicts an embedding that encodes "box moved roughly 10 cm left, orientation unchanged" and is free to ignore lighting flicker entirely, because that variation was never part of the loss it optimizes. Meta's V-JEPA line applies this to raw video for representation learning aimed at robotics and physical reasoning, explicitly trading a renderable output for a representation that is cheaper to predict and, the architecture's proponents argue, more useful for downstream planning and control.
Quick reference
- Pixel-space / generative world models (Genie-style playable worlds, Sora-style video) optimize reconstruction — every pixel, predictable or not, contributes to the loss.
- JEPA-style world models optimize embedding agreement — the loss only cares that predicted and real future embeddings land close together, so unpredictable low-level detail is free to be dropped.
- Neither design is "the" world model architecture; treat this as a genuine, unresolved trade-off in the field, not a settled ranking — pick based on whether you need a renderable output or a planning-efficient representation.
- A rendered prediction is easy to sanity-check by eye; a latent prediction is not directly visualizable, which is a real cost for debugging and trust, independent of accuracy.
- Classical physics engines (rigid-body simulators) are a third, non-learned option: exact for modeled geometries and forces, but they only cover what an engineer explicitly programmed — they do not generalize to an unmodeled material or a camera view outside their assumptions the way a learned model can.
Remember this
Pixel-space world models spend capacity reconstructing everything, including unpredictable noise; JEPA-style latent models spend it only on what the loss says matters — that is a real architectural bet, not a marketing distinction between two demos.
Running one prediction: encode, predict, plan
Put the pieces to work. Before the arm pushes the box, the agent needs to choose which push to make — that requires comparing candidate actions using the world model, not the real box. The loop: encode the current camera frame into state z_t; for each candidate action (push left 5 cm, push left 10 cm, push diagonally), run the dynamics model to get a predicted next state z_{t+1}; score each predicted state against the goal (box centered in the bin); execute only the action whose predicted outcome scores best. This search-inside-imagination procedure is a form of model-predictive control (MPC) — replanning at every step from a fresh observation, rather than committing to one long open-loop plan up front.
The code below is illustrative pseudocode for exactly that loop — a rollout function that chains the encoder and dynamics model, and a planner that scores several candidate actions by rolling each one forward before picking the best. Swap in real encoder/dynamics networks and this structure runs unchanged; the point is the shape of the computation, not a specific framework's API.
Quick reference
- MPC replans every step from a real, re-encoded observation — it does not trust a long open-loop imagined plan, which limits how far rollout error can compound before it gets corrected.
- Scoring candidate actions inside imagination is exactly what makes a world model useful for control: the real environment is touched once per step, not once per candidate.
- The dynamics model here is deterministic for clarity; production world models are usually stochastic or ensemble-based, so planners also weigh predicted uncertainty, not only the mean prediction.
- This is the same shape ReAct-style agent loops use for tool calls — observe, propose an action, evaluate the (predicted or actual) outcome, act — but a world model evaluates the outcome by simulation instead of by calling a real tool.
- Planning cost scales with candidate actions × rollout depth × dynamics-model calls — deeper lookahead and more candidates both directly increase inference-time compute.
Remember this
Planning with a world model means scoring imagined outcomes for several candidate actions and executing only the winner — re-encoding the real result every step (MPC) is what keeps a single bad prediction from steering many real actions in a row.
Failure story: compounding rollout error
Trigger. A robotics team trains a policy entirely inside imagined rollouts of their latent dynamics model — the Dreamer-style recipe — because real-robot pushes are slow and wear the hardware. The policy is trained to plan 8 steps ahead in imagination before ever touching the real box. Symptom. In simulation-in-the-loop training, reward climbs steadily and the policy looks excellent. Deployed on the real arm, the first 1–2 pushes land close to predicted, then the box drifts noticeably off-target by step 4 or 5, and by step 8 the arm is confidently pushing toward empty space.
Root mechanism. The dynamics model's one-step prediction error is small but never exactly zero. In an open-loop rollout, step 2's prediction is built on step 1's predicted (not real) state, so step 1's small error becomes part of step 2's input; step 3 compounds step 2's already-compounded error, and so on. Errors do not cancel out on average — they accumulate, and a policy trained purely inside these rollouts can additionally learn to exploit regions where the model is most wrong, because "exploit the model's blind spot" and "exploit real physics" look identical from inside imagination alone. Diagnostic. Log n-step-ahead prediction error against ground truth for n = 1, 2, 4, 8: a healthy model shows graceful, roughly linear error growth; a model with an exploited blind spot shows the policy's real-world reward drop sharply at exactly the horizon where imagined and real rollouts start to diverge, while imagined reward keeps climbing. Recovery. Shorten the planning horizon (MPC-style frequent replanning against a real observation, as in the previous section, rather than one long open-loop plan) and periodically fine-tune the dynamics model on real rollout data collected under the current policy, not only the data it was originally trained on. Prevention. Treat rollout horizon as a tuned hyperparameter validated against real deployment gap, not a constant copied from a paper — and keep a held-out real-vs-imagined reward gap on the same dashboard as training reward.
Quick reference
- Compounding rollout error is the direct analog of the sim-to-real gap in classical robotics — a learned world model does not remove that gap, it just relocates where the mismatch between imagined and real dynamics lives.
- Stochastic or ensemble dynamics models (multiple predictors, or a predicted distribution instead of a point estimate) let a planner penalize actions whose predicted outcomes disagree across the ensemble — a proxy for "the model doesn't actually know this region well."
- Shorter planning horizons trade lookahead quality for lower compounding error; the right horizon depends on how fast your specific dynamics model's one-step error grows, not a fixed rule of thumb.
- "Model exploitation" — a policy gaming inaccuracies in its own world model — is a known model-based RL failure mode, distinct from ordinary overfitting, because the policy never sees real data during the exploited phase of training.
- Video- or pixel-space world models fail the same way, but the drift often becomes visually obvious (a rendered object smearing or teleporting) before it becomes a control disaster — one practical argument for keeping a decoder around even when the planner only uses the latent state.
Remember this
One-step prediction error looks small in isolation but compounds every time an imagined rollout feeds its own output back in as input — validate a world model on multi-step real-vs-imagined drift, and replan from real observations often enough that no single bad step steers many real actions.
World models vs LLMs vs classical simulators: deciding and evaluating
None of these three tools is a strict upgrade over the others — they answer different questions. An LLM is the right tool when the task is language understanding, retrieval, or reasoning over text and does not require predicting a physical environment's state; see Foundation Models vs Small Language Models for sizing that choice. A classical physics or graphics simulator is the right tool when your environment's dynamics are well understood, geometry and materials are known in advance, and you need exact, reproducible physics — its ceiling is that it only covers what an engineer explicitly modeled. A learned world model earns its cost when the environment is only partially known or hard to hand-specify (real camera noise, deformable materials, novel object shapes), and you need a model that generalizes from data rather than from an engineer's equations, while accepting that its predictions are approximate and can drift.
Evaluating a world model before you trust it for planning is not optional. Track three things: n-step-ahead prediction error against real ground truth (does error grow gracefully or explode past a specific horizon?); the real-vs-imagined reward or task-success gap for any policy trained inside it (a large, growing gap is the compounding-drift failure from the previous section, already showing up in outcomes); and representation probes — simple linear classifiers trained on the model's latent state to check whether it actually encodes physically meaningful properties (object position, contact, occlusion) rather than incidental correlations in the training data. A model that renders beautiful video or produces a low reconstruction loss can still fail all three checks; treat visual plausibility as marketing, not validation.
| Tool | Predicts | Strength | Real limitation |
|---|---|---|---|
| LLM | Next token given prior text | Language reasoning, retrieval, knowledge | No native action/state model — cannot plan physical outcomes on its own |
| Classical simulator | Exact dynamics for modeled physics | Deterministic, reproducible, no drift | Only covers what was explicitly programmed; poor generalization to novel materials/geometry |
| Learned world model | Next state given (state, action) | Generalizes from data; supports planning via imagined rollouts | Approximate; one-step error compounds across multi-step open-loop rollouts |
Quick reference
- Decision rule: choose a classical simulator when the physics is fully known and reproducibility matters more than generalization; choose a learned world model when the environment is partly unknown or too costly to hand-model; choose an LLM when the task is language, not physical state.
- Many production robotics and autonomous-driving stacks combine all three — a classical simulator for known geometry, a learned world model for rare or hard-to-specify scenarios, and language models for instruction following or scene description layered on top.
- Report n-step prediction error, not just one-step error — a model that looks excellent at n=1 and useless at n=8 is not equally useful for a planner that needs 8 steps of lookahead.
- Representation probes catch a model that reconstructs pixels well but has not actually learned object permanence or contact dynamics — a common gap between reconstruction loss and planning usefulness.
- "World model" in a paper title or product announcement does not guarantee any of this evaluation was done — ask which of the three checks (rollout error, real-vs-imagined gap, representation probes) the claim is backed by before trusting a planning result.
Remember this
Pick the tool by what you actually need predicted — language, exact known physics, or generalizable state transitions — and validate a world model on multi-step rollout drift and real-vs-imagined task gap, not on how good its renders look.
Key takeaway
A world model earns its name by predicting what an action does to an environment's state, which is exactly the capability an LLM's next-token objective does not provide. Whether the design renders pixels or predicts in a JEPA-style latent space, the useful mechanism is the same encode → predict → plan loop, and the same failure waits at the end of it: one-step prediction error that looks negligible compounds every time a rollout feeds its own prediction back in as input. That is a testable property, not a philosophical one.
Practice (25 min). In Python, build the smallest possible world model: state s = (x, y), action a = (dx, dy), and a linear dynamics model predict(s, a) = s + W @ a where W is a 2×2 matrix initialized near the identity. Generate 200 synthetic (s, a, s_next) triples from the true rule s_next = s + a + noise (small Gaussian noise), and fit W with a few steps of gradient descent on mean-squared prediction error. Expected result: one-step prediction error should drop to near the noise floor within a few dozen steps. Intentional break: using the trained model, roll out 10 steps open-loop — feed each predicted s back in as the next input, never re-observing the true state — and plot predicted vs. true trajectory; drift should be visibly larger at step 10 than step 2. Recovery: switch to MPC-style planning — re-run predict from the true s at every step instead of chaining predictions — and re-plot; drift at step 10 should shrink close to the single-step error again. Pass criterion: open-loop drift at step 10 should exceed 3× the single-step error, and re-observing at every step should bring it back under 1.5×. For where this sits next to agents that call real tools instead of imagined ones, see ReAct: The Reasoning-Acting Loop.
Related Articles
Explore this topic