M9-017M9: Senior ML & AI System DesignFeature Store, Sharding & CI InfraMedium
⚡Flashcard Drill➔Mastery:

Design a 10-Billion-Scale Distributed Vector Search Architecture

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M9-018 →
📐 Mathematical Definition
1010×768×4B=30 TB→BQ/PQ 16x+NVMe Rescore∼2 TB RAM10^{10} \times 768 \times 4\text{B} = 30\text{ TB} \xrightarrow{\text{BQ/PQ } 16\text{x} + \text{NVMe Rescore}} \sim 2\text{ TB RAM}
⚡ One-Sentence Core Answer
Scale vector retrieval to 10B records using in-memory quantized sketches + NVMe FP16 rescoring (DiskANN/IVF-PQ), tenant shard routing, and background LSM compaction.

📌 Key Takeaways

  • •
    L1 Sizing Math: 10B x 768d FP32 hits 30 TB raw (+20 TB HNSW pointers)—compression & SSD tiers are required
  • •
    L2 Memory/SSD Hybrid: traverse Binary/SQ8 sketches in RAM and rescore Top-200 from NVMe FP16 chunks
  • •
    L2 Tenant Partitioning: route by tenant_id/country key to crush multi-shard scatter-gather fan-out
  • •
    L3 LSM Segments: buffer live writes in memory deltas and compact tombstones asynchronously

📐 Mathematical Derivations

3x–5x oversampling over compact in-memory quantized sketches paired with NVMe FP16 rescoring keeps >98% Recall@10 at 1/16th RAM footprint.

🏭 Production Trade-offs

Never mix vectors from incompatible embedding checkpoints inside one collection; roll out new models via dual-write alias blue-green collections.
⚠️ Common Interview Pitfalls
  • ✕
    Skipping LSM segment compaction until delete tombstones clog graph hops
  • ✕
    Ignoring ~1.5x HNSW graph link pointer overhead during RAM capacity sizing
🎯 Interviewer Follow-up Inquiries
  • ?
    Why does Vamana/DiskANN cut random NVMe page reads better than clustered IVF lists?
  • ?
    How do you split hot enterprise tenants out of shared multi-tenant shard clusters?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM9-016: Design a Zero-Skew Online/Offline Feature Store with Point-in-Time Travel📋 Back to BankNext →M9-018: Design a Layout-Aware Document ETL & Parent-Child RAG Indexing Pipeline