Share Symbols, Not Vectors: SYNC Connects Heterogeneous Language Models

Introduction

Communication is central to multi-agent language-model systems. One model may observe the scene, another may answer questions, and a third may combine the evidence. Text is the easiest common language, but it consumes context and can introduce unnecessary verbal overhead. Exchanging hidden states looks more compact, yet representations are usually tied to a model’s architecture, tokenizer, dimensions, and layer organization. In practice, cross-model state transfer often requires a learned translator or assumptions about architectural compatibility.

A team from Nanyang Technological University proposes SYNC, or Symbol-Indexed Neural Communication. The method takes a middle path: agents share indices for words in a common list, while each receiver interprets those symbols through its own internal representation space.

How SYNC works

• The sender transmits indices, not vectors. From a description it can observe, the sender selects words that represent an object’s attributes and sends the corresponding indices for each object. It does not see the questions used later in evaluation, which helps test whether the message contains reusable facts rather than answer-specific hints.
• The receiver uses local word directions. The researchers fit directions for the shared words once from the receiver’s activations. Incoming symbols are then written into the receiver’s hidden states using those local directions. No task-specific fine-tuning or cross-architecture translator is introduced.
• The write depth is selected early. A suitable layer is chosen on four development receivers and then fixed for evaluation. This design tests whether the interface can transfer to new receivers instead of being retuned for every model.
• Selective use is tested directly. The benchmark swaps one attribute between two objects and asks for four answers: the changed and unchanged attributes, both before and after the swap. A successful system must use the communicated fact selectively rather than merely follow a global prompt pattern.

Results and limitations

Across three senders and nine newer receivers from five model families, four-answer success on new scenes ranges from 72.0% to 98.3%. Compared with writing into a middle layer, the improvement reaches as high as 96 percentage points. The two largest receivers perform on par with plain text communication. A task-free probe also predicted which writing depths each new receiver could use, correctly covering all 45 evaluated cases.

The interface remained effective when messages listed every fact and the layouts changed, with success ranging from 89.6% to 100% without retuning. That result suggests the method is not simply memorizing one spatial arrangement. However, the picture is less positive for a landmark task using multi-token symbols: success was 23.9% to 64.0%. This is far above the 0% result without a message, but still below text. SYNC therefore appears strongest when facts can be expressed as discrete, shared attributes, not when communication requires richer compositional language.

Why it matters

SYNC suggests that a useful interoperability layer does not have to expose or translate full neural states. A shared symbolic interface can provide a small public protocol, while receiver-local directions preserve the differences between model families. This could lower the setup cost for heterogeneous agent teams that exchange structured object states or other compact facts.

The work does not establish that symbols can replace language in general. The experiments focus on controlled fact communication, and open-vocabulary relations, long-range reasoning, ambiguity, and safety behavior remain open questions. A practical architecture may combine SYNC for compact state transfer with text for explanations, exceptions, and uncertainty. Even with those limits, the result is a notable step toward training-free communication across model boundaries.

Source: Hugging Face Daily Papers