Can Models Answer Without Retrieval? IAR Splits Document Internalization into Three Stages

Introduction

Retrieval-augmented generation is the usual answer when a model must work with a bounded document collection. But what if the documents are not provided at inference time? The paper Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization studies this setting as document knowledge internalization: converting a fixed corpus into usable parametric knowledge for question answering without retrieved passages.

The central issue is that “seeing” a document during training and “being able to answer from it” are not necessarily the same capability. A model may absorb information but fail to access it in a question-answering format. Conversely, aggressive domain fine-tuning may improve document QA while weakening instruction following and other general skills.

A three-stage training design

IAR assigns a different responsibility to each stage:

• Inject turns source documents into continuation, rewrite, and instruction-conditioned reconstruction objectives. These tasks provide dense and varied exposure to the corpus, rather than relying only on conventional supervised QA examples.
• Align applies answer-only QA supervision to the injected model. The goal is to make the internalized information accessible through the expected question-answering behavior.
• Recover merges the domain-adapted checkpoint with the original instruction model. This step is intended to restore general capabilities that may have degraded during domain adaptation.

The separation is the paper’s main conceptual contribution. Instead of treating document exposure, answer generation, and capability retention as one optimization target, the framework handles them sequentially and then uses model merging to balance specialization with generality.

What the reported results show

The experiments cover two document corpora, Common Corpus and CCI, as well as the Llama, Phi, Qwen, and SmolLM model families. Relative to Vanilla SFT, IAR improves all four reported metrics in seven of eight dataset-model settings. On average, the framework raises domain QA accuracy by 3.6 percentage points and improves the mean general performance across IFEval, MMLU, and MSBench by 12.1 points.

The extended Common Corpus comparisons add an important qualification. LoRA and FAPM can lead on individual general-purpose metrics. However, among methods that also deliver leading or near-leading document internalization, IAR remains one of the stronger overall trade-offs. The result therefore should not be read as “IAR wins every metric,” but as evidence that staged optimization can improve the domain-general frontier.

Why it matters

The study offers an alternative lens on the boundary between RAG and fine-tuning. Parameter-based internalization may be attractive when the corpus is relatively stable, retrieval is costly, or inference must operate without exposing source passages. It also suggests that evaluations should separately measure document exposure, QA accessibility, and retention of general abilities.

There are practical caveats. Once information is stored in parameters, updating facts, tracing their sources, and correcting mistakes can be harder than with an external retriever. The reported gains are also tied to the selected corpora, model families, baselines, and evaluation protocol. IAR is therefore better viewed as a structured method for studying retrieval-free knowledge acquisition, not as a universal replacement for retrieval.

Source: Hugging Face Daily Papers