Pre-Ranking Lightweight Architecture

The Pre-Ranking (Match-Filtering) Architecture is the critical intermediate layer bridging multi-channel retrieval and heavy ranking; within a strict 5ms latency budget, it scores and filters $\sim 10,000$ candidates from multi-channel recall down to $\sim 1,000$ high-potential items for heavy ranking; standard architectures include: 1) Vector-based Pre-Ranking (dual-tower dot products); 2) Distilled Lightweight DNNs (e.g., Alibaba COLD: a compact 2-3 layer MLP sharing lower embedding tables with the heavy ranker); 3) Feature-Separated Cross Networks (FSCD).

High-concurrency recommendation and ad systems where multi-channel recall yields 50,000+ candidates exceeding heavy ranker compute budgets.

Sending 10,000+ items directly to heavy rankers triggers massive SLA timeouts and infrastructure cost explosions; naive truncation destroys multi-channel diversity; pre-ranking achieves 90%+ heavy-ranker overlap at only 5% compute cost.

Compare Two-Tower Pre-ranking vs Lightweight Interactive Pre-ranking (COLD) across expressive power and compute latency?

When pre-ranker shares embedding tables with the heavy ranker, how to prevent pre-ranker gradients from corrupting embedding precision?

How does Knowledge Distillation train the lightweight pre-ranker using the heavy ranker's soft prediction scores as teacher targets?

Why does pre-ranking evaluation prioritize Top-K Hit Rate / Overlap with heavy ranking outputs alongside standalone AUC?

How do C++ serving engines leverage AVX-512 SIMD and INT8 quantization to maximize pre-ranking throughput over 1000 items in 5ms?