🏗️ System DesignID:
online-parallel-retrieval-gatewayOnline Parallel Retrieval Gateway
在线多路并行检索网关
🎯Core Definition
The Online Parallel Retrieval Gateway serves as the high-throughput coordination nexus for live RAG requests under a strict $<50\text{ms}$ SLA; upon receiving a user prompt, the gateway orchestrates a standardized pipeline: 1) Query Transformation (co-reference resolution, HyDE hypothetical embeddings, sub-query decomposition); 2) Asynchronous Parallel Fan-Out (concurrently dispatching async I/O queries across Elasticsearch BM25, Milvus/Qdrant dense vector databases, and GraphRAG knowledge graphs); 3) Rank Alignment via Reciprocal Rank Fusion (RRF) merging heterogeneous results into Top 50 candidates; 4) Cross-Encoder Neural Reranking (BGE-Reranker / Cohere) selecting Top 3-5 grounded chunks; 5) Circuit breaking and timeout fallbacks.
💡Use Cases
Multi-hop enterprise Q&A systems, intelligent customer support engines, and large-scale codebase semantic search.
⚡Key Problems Solved
Serial querying across disparate databases causes cumulative latency blowouts; the parallel gateway leverages non-blocking async fan-outs, robust RRF fusion, and circuit breaking to guarantee high availability and sub-50ms execution.
🎯5 High-Frequency Exam Points
1
Design a high-throughput async fan-out and fan-in retrieval gateway in Go/Rust with strict deadline contexts?
2
Explain why RRF mathematically achieves unbiased candidate fusion across incompatible score spaces (BM25 vs Cosine Similarity)?
3
How to optimize Cross-Encoder GPU memory and latency budgets using TensorRT / ONNX Runtime quantization for sub-20ms reranking?
4
Explain sub-query decomposition strategies that split comparative user questions into parallel search branches before merging?
5
Design circuit breaking and fallback policies that return partial BM25 results when vector database nodes time out past 30ms?
📖 In-depth Guide:📄 llm-rag-agent-system-design →
Updated 2026-08-14🎯
🚀 Start Card Practice ➔Test Your Knowledge: Practice Questions for "Online Parallel Retrieval Gateway"
Single choice pitfall questions with instant feedback and mistake tracking.