Test-Time Compute: How Reasoning Models Spend Inference Budget

Classic scaling laws describe what happens when you spend more compute once, during training: a bigger model, trained on more data, with more optimizer steps, tends to get more capable. That spend is amortized — every user who ever calls the model shares the cost of having trained it, and the resulting weights do not change per request.

Test-time compute is a different ledger. It is spent fresh on every call: more sampled attempts, more intermediate reasoning tokens, more self-checking passes, before the model returns an answer. Research on inference-time scaling (the s1 and DeepSeek-R1 lines of work) found that a smaller model given room to search at inference time can close a meaningful part of the gap to a model that only got bigger during training — but that room costs real latency and tokens on every single request, not once at release time.

There are two broad families of test-time technique, and they are not interchangeable. Parallel methods run the same request multiple times — sample N candidate answers (or N reasoning paths), then pick the best one by majority vote (self-consistency) or with a separate verifier/reward model. Cost scales with N; latency can stay flat if samples run concurrently.

Sequential methods spend the budget inside one continuous generation instead: the model drafts, notices it's about to stop, and is pushed to keep reasoning. The s1 paper's budget forcing technique does this almost mechanically — when the model tries to end its reasoning early, the harness appends the token Wait and lets it continue, which the paper reported let a fine-tuned 32B model exceed a preview reasoning model by a wide margin on competition math after training on only about 1,000 curated examples. DeepSeek-R1 gets a similar effect a different way: reinforcement learning trains the model to naturally allocate a longer reasoning chain to harder problems, without an external harness forcing continuation.

Extended test-time compute helps most on tasks with a checkable structure the model can search over — math with a verifiable final number, code with a test suite that passes or fails, planning problems with constraints to satisfy. The failing test test_partial_refund_after_currency_rounding is exactly this shape: the agent can draft a patch, run the suite, see red or green, and spend more search only while it's still red.

It helps far less, and sometimes hurts, on tasks that are recall-bound rather than search-bound: "who is on call this week" or "what does clause 4.2 of the contract say" has one correct answer sitting in context (or missing from it) — no amount of additional reasoning invents a fact that was never retrieved. Research on knowledge-intensive benchmarks found test-time scaling gave little or no accuracy gain there, and a longer reasoning trace can even talk itself into a confident, wrong bridging fact instead of admitting the context doesn't contain the answer.

Build a ten-case eval set from real coding-agent failures: five checkable bugs (a failing test, a type error, a lint violation) and five recall-bound questions ("what does the on-call runbook say to do", "what's the current refund window") where the answer must come from a provided document, not invented. Run every case at low, medium, and high reasoning effort, and record pass rate, tokens spent, and wall-clock latency for each.

Expected result: the checkable-bug cases show real accuracy gains from low to high, at a real token and latency cost. Intentional break: run the recall-bound cases at high effort with the source document deliberately removed from context. Pass criterion: the model should say it cannot find the answer rather than confidently inventing one — if a longer reasoning trace produces a fluent wrong answer instead of an admission of missing context, that is the overthinking failure mode this article warned about, and the fix is a retrieval or tool step, not a bigger effort setting.