M9-016M9: Senior ML & AI System DesignFeature Store, Sharding & CI InfraHard
⚡Flashcard Drill➔Mastery:

Design a Zero-Skew Online/Offline Feature Store with Point-in-Time Travel

🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
Next: M9-017 →
📐 Mathematical Definition
F(e,t)=AS-OF ⁣(max⁡{τ≤t:(e,τ)∈History})  ⟹  Skew=0F(e, t) = \text{AS-OF}\!\left(\max\{\tau \le t : (e, \tau) \in \text{History}\}\right) \implies \text{Skew}=0
⚡ One-Sentence Core Answer
Guarantee zero training-serving skew via shared streaming feature transforms feeding Redis online stores and Iceberg offline tables joined strictly via AS-OF time travel.

📌 Key Takeaways

  • •
    L1 Dual Pitfalls: offline/online transform code divergence and post-event same-day leakage
  • •
    L2 Point-in-Time Join: align historical Iceberg logs strictly <= sample event timestamp
  • •
    L2 Sub-5ms Serving: materialize sliding aggregates via Flink into Redis for single pipelined MGETs
  • •
    L3 Request Logging: archive actual online inference vectors to eliminate feature drift on known features

📐 Mathematical Derivations

Pair online request-time feature logging (guaranteed zero skew) with offline Iceberg AS-OF event replay (rapid iteration on newly invented features).

🏭 Production Trade-offs

Decompose 30-day sliding windows into hourly/daily rollups to shrink RocksDB operator state by >10x cleanly.
⚠️ Common Interview Pitfalls
  • ✕
    Equi-joining daily feature partitions leaks end-of-day updates backward onto morning events
  • ✕
    Issuing sequential per-feature RPC calls rather than pipelined entity multi-gets
🎯 Interviewer Follow-up Inquiries
  • ?
    How should late-arriving mobile events be handled via Flink allowed-lateness + side outputs?
  • ?
    How do you tier heavy item embeddings separately from fast scalar counters in Redis?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
📚

Associated Knowledge Base Guides & Mindmaps

← PreviousM9-015: Design a Full-Duplex Sub-400ms Voice-to-Voice Agent with Instant Barge-In📋 Back to BankNext →M9-017: Design a 10-Billion-Scale Distributed Vector Search Architecture