Why Signed Knowledge Packs Can Still Steer Offline RAG

Introduction

Offline retrieval-augmented generation is increasingly attractive for edge devices, disconnected environments, and systems that cannot continuously refresh data from the cloud. A typical update arrives as a signed knowledge pack containing documents and precomputed embeddings. That signature answers an important provenance question—who published the package—but it does not prove that each vector actually represents the text next to it. The resulting text-vector integrity gap gives an attacker a way to alter retrieval without visibly rewriting the source document.

What the study found

• Tolerance becomes an attack surface. A verifier may recompute an embedding and accept it when its cosine similarity to the supplied vector exceeds a threshold. An attacker can then search within that allowed error to move a document in embedding space. The practical exposure is governed by retrieval margins: whenever the gap between competing candidates is smaller than the tolerated perturbation, ranking can change.
• Exact reproducibility is difficult. Floating-point pipelines and dynamic INT8 inference can vary with batch size or hardware architecture. Requiring bit-for-bit equality is therefore unrealistic for many deployments, but relaxing the requirement without measuring its consequences can be dangerous.
• The measured risk is substantial. Across five corpora and two embedding models, a tolerance near floating-point noise exposed as many as 5.5% of queries; after removing exact duplicate documents, the maximum for any corpus was 1.2%. A threshold calibrated from CPU INT8-FP32 differences exposed 98–100% of queries. In an end-to-end Raspberry Pi 5 experiment with a local language model, the attack reduced the correct-answer rate from 0.75 to 0.39.
• Compression does not automatically restore integrity. In a FAISS product-quantization index containing 57,638 documents, tampered vectors that passed the audit still placed the attacker’s document first for 25% of targets.

A different verification boundary

The paper’s proposed direction is to verify quantization codes rather than relying only on reconstructed floating-point vectors. The design combines verifier-decided handling of ambiguous cases, a calibrated guard band, and canonicalization. In the reported samples across three platforms, every honest vector was accepted, while the publisher could no longer exploit an open tolerance to steer rankings. The authors also report preregistered negative results for abstention, context pruning, and semantic screening, suggesting that post-retrieval fixes alone are not a complete defense.

Why it matters

The broader lesson is that publisher authentication, vector consistency, and retrieval-behavior integrity are separate properties. An offline RAG supply chain should therefore audit more than signatures and file hashes. It should specify what representation is verified, how cross-platform numerical variation is bounded, and who resolves quantization ambiguity. A canonical, verifier-controlled representation can narrow the gap between a package that is genuinely trustworthy and one that is merely signed.

Source: Hugging Face Daily Papers