Can LLM Translation Move Beyond Sentences? PAT Tests a RAG-Based Approach

Introduction

Most translation technology, from CAT tools to machine translation systems, still treats translation as a sequence of sentence-level decisions. That design makes alignment, editing, and quality control easier, but it can flatten the larger features that make a text feel natural in another language: discourse organization, rhetorical emphasis, paragraph flow, and pragmatic expectations. The arXiv paper “Can an Old Dog Be Taught New Tricks? Taking LLMs Beyond Sentence Level Translation” asks whether large language models can be guided away from that inherited paradigm.

Key points

• The goal is not just better sentence translation. The study focuses on whole-document draft translation for professional verification, especially where U.S. English texts must be reshaped for Latin American and Mexican Spanish contexts.
• The paper introduces PAT, or Pragmatic Auto-Translator. PAT is a RAG-based system that combines user-configured translation specifications with retrieved examples from authentic longform texts.
• The retrieved context works above the sentence level. Instead of only showing the model local sentence pairs, PAT provides paragraph-, section-, and document-level examples intended to guide discourse structure, rhetorical style, and pragmatic norms.
• The evaluation uses a customized MQM typology. Six automatic translations of essays about generative AI were assessed across three projects by two trained evaluators working from U.S. English into LATAM and Mexican Spanish.
• The findings are cautious. A limited prompt produced no meaningful reformulation. Specifications and corpus-informed translations sometimes led to substantial restructuring, but those changes were not always successful.

Why it matters

The paper is valuable because it reframes LLM translation as a discourse and localization problem, not merely a lexical or syntactic transfer task. In professional settings, a faithful sentence-by-sentence rendering may still fail if the target text does not match the expectations of its audience. Argument order, explanatory density, paragraphing, and tone can all differ across language communities.

At the same time, the study avoids overclaiming. Encouraging a model to reformulate more freely does not automatically improve quality. The usefulness of PAT depends on how translation specifications are written, how the comparable corpus is built, what kinds of passages are retrieved, and how evaluators define successful reformulation. In that sense, the system is best understood as a way to generate drafts for expert review, not as a replacement for professional judgment.

The broader implication is clear: high-quality LLM translation systems may need to combine generation, retrieval, and explicit translation briefs. This work shows that LLMs can be nudged beyond the sentence-by-sentence model, but also that controlled and effective document-level reformulation remains an open challenge.

Source: arXiv