LatentPort Tests Cross-Model Memory Handoffs Beyond KV Cache

Introduction

When an inference workload moves from one language model to another, the usual options are expensive. The receiving model can reread the entire prefix, or it can receive an attention KV cache produced by the first model. The first option repeats computation; the second may discard information accumulated by recurrent components in a hybrid architecture. LatentPort asks whether a model can instead pass a broader snapshot of its live memory to another model, allowing the receiver to continue without replaying the historical tokens.

What was tested

The paper evaluates a geometry-matched Qwen3.5 4B-to-9B Base-model pair. The 4B model first processes a 4,096-token context. The handoff then supplies the 9B model with translated attention keys and values, together with state from its Gated DeltaNet (GDN) recurrent modules and its convolutional components. The target model receives zero historical prefix tokens and is evaluated on the continuation.

The main findings are:

• KV translation alone leaves a substantial performance gap.
• Adding the persistent GDN state reduces teacher-forced negative log-likelihood by 0.747 nats per token, with a 95% paired-document bootstrap confidence interval of 0.6921–0.8047. All 64 PG19 documents improve.
• Direct reuse of recurrent and convolutional state beats the learned GDN maps tested in the study, suggesting that some state coordinates retain partial functional compatibility across the two model sizes.
• A further 434,176-parameter correction improves the selected handoff on 64 fresh web documents. Its continuation loss is 0.076 nats per token above native 9B, with a JS divergence of 0.022 and a native-context-recovery score of 0.918.

The corrected 9B handoff also significantly outperforms simply continuing inference with the 4B model, despite processing no historical prefix tokens. This makes the result more interesting than a cache-compression trick: it suggests that a model upgrade could inherit part of an earlier model’s computational history.

Why it matters—and what it does not prove

The conceptual contribution is to broaden the unit of transfer. In a hybrid model, useful context may live not only in attention KV entries but also in persistent recurrent and convolutional states. A practical state interface could eventually support cheaper model switching, cascaded inference, dynamic serving, or cooperation between specialized models. It would also shift part of the systems discussion from token transport toward representation compatibility.

The evidence is still narrow. The experiments cover one transfer direction, one architecture-matched model pair, and 4K teacher-forced continuation. The near-native gating branch failed, the 16K branch was not run, and the reported material does not establish equivalence under free generation. It is therefore too early to treat LatentPort as a general cross-model memory protocol. Compatibility across unrelated architectures, training versions, and deployment settings remains an open question, as do isolation and privacy risks when internal states cross model boundaries.

Source: Hugging Face Daily Papers