The Narrative Gap: Can LLMs Navigate Diverse Views Across Languages?
Introduction
Large language models are becoming an important gateway to news and public information. On contested topics, however, a model does more than retrieve facts: it also determines which explanations, actors, and perspectives become visible. Narratives that are common in one region or language may be absent, framed differently, or treated as secondary elsewhere. If a model consistently favors a narrow set of accounts, an apparently helpful answer can become another layer of information filtering.
The study The Narrative Gap examines whether LLMs can help people navigate such differences. Its authors introduce NarrativeBench, a benchmark designed to test whether models provide meaningful narrative diversity rather than merely producing summaries that sound neutral or balanced.
Key points
- Diverse information seeking involves six capabilities. Three foundational capabilities concern identifying narratives that should be included, narratives that should be excluded, and narratives missing from the available material. Three higher-level capabilities concern presenting diverse accounts, preserving coverage when users ask confirmatory questions, and maintaining information parity across languages.
- The benchmark represents a multilingual news environment. NarrativeBench includes 1,333 news articles written in 23 publication languages and covering 23 contested topics. It also provides 460 Reddit-grounded queries. The study evaluates 14 LLMs using 24 query languages.
- Asking for diversity does not solve the problem. According to the study summary, existing models still lack the capabilities needed to navigate competing narratives reliably. A prompt that explicitly requests multiple perspectives may not prevent important accounts from being omitted.
- Language and framing affect what users can access. The same topic can receive different narrative coverage depending on how a question is phrased and which language is used. The result may be more than an imperfect answer: users may be placed in different information environments.
Why it matters
The contribution of this work is to move beyond the broad question of whether an answer “sounds balanced.” A model can add cautious phrases such as “some people argue” without identifying the most relevant missing account. Likewise, translating an answer into another language does not guarantee that readers receive the same range of evidence or interpretations. Narrative diversity therefore needs to be evaluated as coverage, omission, and consistency—not just tone.
For model developers, NarrativeBench offers a way to test behavior closer to real information seeking. Systems should be examined for what they leave out, whether they narrow an answer when a user signals a preferred conclusion, and whether query-language differences create systematic disparities. For search products, news interfaces, and retrieval-augmented systems, the findings suggest that source language, narrative coverage, and unresolved gaps may need to be made more visible.
If LLMs become part of the public information infrastructure, answering a question is only the starting point. A more demanding standard is helping users understand which accounts are represented, which are contested, and what they may still be missing.
Source: Hugging Face Daily Papers
Comments
Checking sign-in status...
Loading comments...