OCP-CT: Fine-grained CT vision-language pretraining with organ-conditioned pattern tokens

Introduction

Vision-language pretraining for computed tomography has a clear appeal: hospitals already produce large amounts of paired CT volumes and radiology reports, and these pairs can be used to train models without exhaustive manual labeling. Yet CT is a difficult setting for standard multimodal learning. A single scan can contain multiple organs, varied abnormalities, and unevenly distributed visual evidence, while a report may summarize findings at different levels of detail.

Most existing approaches rely on global scan-report contrast. This strategy is scalable and stable, but it treats the entire CT volume and the entire report as the main alignment units. As a result, organ-specific or pattern-specific evidence may be diluted. Direct organ-level alignment is also not sufficient, because the same anatomical region can exhibit many different radiological appearances.

The arXiv paper “Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding” proposes OCP-CT to address this gap. Its central idea is to align image and text not only at the scan or organ level, but at the level of organ-conditioned radiological patterns.

Key ideas

• A stable global branch remains in place. OCP-CT does not discard global CT-report contrastive learning. Instead, it preserves this branch as a foundation for broad semantic alignment.
• An organ pattern interface adds finer structure. The framework introduces a module that organizes image and text tokens according to latent radiological patterns conditioned on anatomy.
• Sparse MoE routes tokens by latent patterns. A sparse Mixture-of-Experts mechanism routes visual and textual tokens to expert pathways, helping separate heterogeneous evidence rather than mixing it into a single representation.
• Learnable slots produce pattern tokens. Learnable slots query the routed tokens and convert them into continuous pattern tokens that represent radiological appearances in both image and text spaces.
• Paired token contrast performs fine-grained alignment. Image-side and text-side pattern tokens are aligned with structured soft targets derived from report-based clinical similarity, rather than relying only on hard pair labels.

Reported results

The paper evaluates OCP-CT on the public CT-RATE and RAD-ChestCT benchmarks for zero-shot abnormality diagnosis. According to the abstract, the method achieves average AUROCs of 84.5% and 69.9% on these benchmarks, respectively. Compared with the strongest previously reported results cited by the authors, this corresponds to absolute AUROC gains of 6.7 and 0.8 percentage points.

These numbers are notable because the evaluation focuses on zero-shot diagnosis. In medical imaging, zero-shot transfer is especially relevant: label sets, report styles, and abnormality definitions often vary across datasets and institutions. A model that learns more structured image-report correspondences may generalize better when the target task is not identical to the pretraining setup.

Why it matters

OCP-CT highlights an important direction for medical multimodal AI: scaling data alone is not enough; the unit of alignment also matters. Global contrast provides a useful training signal, but CT interpretation is inherently compositional. Radiologists reason across organs, patterns, and clinical context. Modeling organ-conditioned radiological patterns is a step toward representations that better reflect this structure.

The work is still presented as an arXiv preprint, and the reported gains will need further validation through independent reproduction and broader clinical evaluation. Even so, the approach points to a practical path for CT foundation models: combine robust global alignment with finer pattern-level supervision extracted from reports.

Source: arXiv