ColBERT Late Interaction

ColBERT (Contextualized Late Interaction over BERT) is a multi-vector retrieval and re-ranking architecture that fuses the speed of Bi-Encoders with the precision of Cross-Encoders; instead of compressing documents into a single dense vector, ColBERT maintains independent contextualized token embeddings for all tokens; during scoring, it bypasses heavy cross-attention layers, computing relevance via a lightweight MaxSim operator $S(Q, D) = \sum_{i \in |Q|} \max_{j \in |D|} (E(q_i) \cdot E(d_j))$ that sums maximum cosine alignments for each query token across document tokens.

Ultra-low-latency (sub-10ms) precision re-ranking, multi-vector end-to-end search (RAGatouille), and visual multi-modal page indexing (ColPali).

Heavy Cross-Encoders ($O(N \cdot L^2)$) induce 100ms+ latency, blocking high-concurrency scaling; single-vector Bi-Encoders lose fine-grained token alignments; ColBERT's MaxSim reduces scoring to batched matrix dot products, delivering Cross-Encoder quality at millisecond speeds.

Test Your Knowledge: Practice Questions for "ColBERT Late Interaction"

Single choice pitfall questions with instant feedback and mistake tracking.