M9-016M9: Senior ML & AI System DesignFeature Store, Sharding & CI InfraHard
⚡Flashcard Drill➔Mastery:
Design a Zero-Skew Online/Offline Feature Store with Point-in-Time Travel
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Guarantee zero training-serving skew via shared streaming feature transforms feeding Redis online stores and Iceberg offline tables joined strictly via AS-OF time travel.
📌 Key Takeaways
- •L1 Dual Pitfalls: offline/online transform code divergence and post-event same-day leakage
- •L2 Point-in-Time Join: align historical Iceberg logs strictly <= sample event timestamp
- •L2 Sub-5ms Serving: materialize sliding aggregates via Flink into Redis for single pipelined MGETs
- •L3 Request Logging: archive actual online inference vectors to eliminate feature drift on known features
📐 Mathematical Derivations
Pair online request-time feature logging (guaranteed zero skew) with offline Iceberg AS-OF event replay (rapid iteration on newly invented features).
🏭 Production Trade-offs
Decompose 30-day sliding windows into hourly/daily rollups to shrink RocksDB operator state by >10x cleanly.
⚠️ Common Interview Pitfalls
- ✕Equi-joining daily feature partitions leaks end-of-day updates backward onto morning events
- ✕Issuing sequential per-feature RPC calls rather than pipelined entity multi-gets
🎯 Interviewer Follow-up Inquiries
- ?How should late-arriving mobile events be handled via Flink allowed-lateness + side outputs?
- ?How do you tier heavy item embeddings separately from fast scalar counters in Redis?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚