Reasoning and Memory in Language Models

Jul 2026 to present · Independent research · Pythia, Qwen2.5-Coder, PyTorch · Not yet published

Two questions about changing a language model after it is trained: can a small module installed inside a frozen model teach it to reason in several steps, and when should a deployed model learn a new fact in its weights instead of looking it up?

1. A recurrent reasoning graft

Recurrent reasoning models like TRM and HRM get strong results from small modules that iterate and decide for themselves when to stop. Interpretability work has also found a narrow band inside language models that multi-step reasoning passes through. The hypothesis: put a small recurrent module with learned halting at that band of a frozen model, train only the module (about 5% of the parameters), and the model gains reasoning it did not have.

The graft took frozen Pythia-1B from near 0% to 54% accuracy on eight-hop questions, and to 61% on entities it never saw in training, while the rest of the model stayed frozen. The same graft on Qwen2.5-Coder-7B reached 96% on eight-hop questions.

I wrote this up as a 12-page paper, along with four measurement pitfalls that affect anyone training a module inside a frozen transformer.

2. Learn a fact, or look it up?

A deployed model keeps meeting facts it was never trained on. It can store them in its weights with a small update, or retrieve them into the prompt each time. I tested both on Pythia-410M and 1B, with synthetic facts and with real counterfactual edits (CounterFact).

• Looking it up was as accurate or better on every test. A simple retriever found the right fact for every held-out wording, at about 11 extra prompt tokens per question.
• So the choice is cost. On Pythia-410M, learning a fact in the weights only saves compute once it has been asked about roughly 2,400 times (about 1,300 at 1B). That leaves a narrow niche: facts that are asked often and rarely change.
• An update can be kept away from everything else. A gate that switches the update on only when a prompt names a learned entity fired on every such prompt and on no others, so unrelated text stayed as it was.
• One small adapter per entity avoids forgetting. Forty facts added over eight rounds all stayed correct with no replay of old data, at 20 times the parameters of one shared adapter. One seed so far.

Both lines of work are unpublished. The paper and the claims ledger behind every number here are available on request.