M9-017M9: Senior ML & AI System DesignFeature Store, Sharding & CI InfraMedium
⚡Flashcard Drill➔Mastery:
Design a 10-Billion-Scale Distributed Vector Search Architecture
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Scale vector retrieval to 10B records using in-memory quantized sketches + NVMe FP16 rescoring (DiskANN/IVF-PQ), tenant shard routing, and background LSM compaction.
📌 Key Takeaways
- •L1 Sizing Math: 10B x 768d FP32 hits 30 TB raw (+20 TB HNSW pointers)—compression & SSD tiers are required
- •L2 Memory/SSD Hybrid: traverse Binary/SQ8 sketches in RAM and rescore Top-200 from NVMe FP16 chunks
- •L2 Tenant Partitioning: route by tenant_id/country key to crush multi-shard scatter-gather fan-out
- •L3 LSM Segments: buffer live writes in memory deltas and compact tombstones asynchronously
📐 Mathematical Derivations
3x–5x oversampling over compact in-memory quantized sketches paired with NVMe FP16 rescoring keeps >98% Recall@10 at 1/16th RAM footprint.
🏭 Production Trade-offs
Never mix vectors from incompatible embedding checkpoints inside one collection; roll out new models via dual-write alias blue-green collections.
⚠️ Common Interview Pitfalls
- ✕Skipping LSM segment compaction until delete tombstones clog graph hops
- ✕Ignoring ~1.5x HNSW graph link pointer overhead during RAM capacity sizing
🎯 Interviewer Follow-up Inquiries
- ?Why does Vamana/DiskANN cut random NVMe page reads better than clustered IVF lists?
- ?How do you split hot enterprise tenants out of shared multi-tenant shard clusters?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚