M7-053M7: Retrieval, Ranking & RecSysRecommender Systems FoundationsEasy
⚡Flashcard Drill➔Mastery:
Explain Matrix Factorization (MF) and Latent Factor Models in Collaborative Filtering
🎯 Active Recall Self-Check (Draft First, Verify After)
Login-free practice tracker: recall core answer, audit pitfalls, and log review status.
Step 1 · Draft your concise answer & assumptions
Step 2 · Check Pitfalls & Follow-up Inquiries
📐 Mathematical Definition
⚡ One-Sentence Core Answer
Matrix Factorization decomposes a sparse user-item interaction matrix into low-rank user and item latent embedding matrices, predicting ratings via vector dot products trained through Stochastic Gradient Descent (SGD) or Alternating Least Squares (ALS).
📌 Key Takeaways
- •Low-rank projection: Decomposes sparse matrix R (M x N) into user factors P (M x k) and item factors Q (N x k), where k << min(M, N).
- •Latent semantic space: Dot product p_u^T q_i captures affinity between a user's latent preference profile and an item's latent attributes.
- •Bias terms integration: Extends basic dot products with global average, user bias, and item bias to isolate baseline rating tendencies.
📐 Mathematical Derivations
Mathematical Formulation: Matrix Factorization Mechanics (Koren et al., Netflix Prize).
(1) The Matrix Decomposition Framework:
Let user-item rating matrix contain sparse observed ratings . MF approximates as the product of two low-rank matrices:
where represents user 's latent preference vector, and represents item 's latent feature vector.
(2) Biased Matrix Factorization Formulation:
Observed ratings reflect systematic rating baselines (some users are critical raters; blockbuster items receive uniformly higher ratings). The predicted rating incorporates global baseline , user bias , and item bias :
(3) Regularized Optimization Objective:
(4) Optimization Algorithms:
- Stochastic Gradient Descent (SGD):
For each observed rating , evaluate prediction error , and update parameters in the gradient direction:
- Alternating Least Squares (ALS):
Fix and solve for analytically in closed form (ridge regression); then fix and solve for . Highly parallelizable across distributed CPU clusters.
(1) The Matrix Decomposition Framework:
Let user-item rating matrix contain sparse observed ratings . MF approximates as the product of two low-rank matrices:
where represents user 's latent preference vector, and represents item 's latent feature vector.
(2) Biased Matrix Factorization Formulation:
Observed ratings reflect systematic rating baselines (some users are critical raters; blockbuster items receive uniformly higher ratings). The predicted rating incorporates global baseline , user bias , and item bias :
(3) Regularized Optimization Objective:
(4) Optimization Algorithms:
- Stochastic Gradient Descent (SGD):
For each observed rating , evaluate prediction error , and update parameters in the gradient direction:
- Alternating Least Squares (ALS):
Fix and solve for analytically in closed form (ridge regression); then fix and solve for . Highly parallelizable across distributed CPU clusters.
🏭 Production Trade-offs
In-Depth Analysis & Engineering Trade-offs: ① SGD vs. ALS operational trade-off—SGD is fast, lightweight, and handles incremental streaming updates easily; ALS solves quadratic subproblems analytically, making it exceptionally parallelizable on distributed frameworks (Apache Spark) and naturally suited for implicit feedback datasets where every unobserved pair is treated as a weighted zero. ② Linear inner-product limitations—MF computes purely linear interactions ; it cannot model complex non-linear feature crosses or utilize dynamic side features (user age, item category, time of day); modern systems supersede MF with deep models (Wide&Deep, DLRM). ③ Latent factor dimensionality $k$ tuning—small () prevents overfitting on sparse data; large () increases expressiveness but requires heavier regularization . ④ Implicit feedback adaptation (Hu, Koren, Volinsky)—in real-world settings, ratings are rarely explicit; adapting MF to implicit clicks requires associating binary preference with continuous confidence weights . ⑤ Serving via vector ANN—once and are solved, item retrieval is identical to vector search: query vector is , and top- items are retrieved via inner product search against using HNSW or ScaNN. ⑥ Interview takeaway—write out the biased MF formulation , detail the regularized objective, contrast SGD and ALS, and highlight how MF maps directly onto modern vector ANN serving.
⚠️ Common Interview Pitfalls
- ✕Attempting to run standard SVD decomposition on sparse matrices without masking unobserved entries, treating missing values as actual zero ratings.
- ✕Omitting user and item bias terms (b_u, b_i), forcing latent vectors to absorb baseline scale offsets and degrading recommendation accuracy.
- ✕Using SGD on implicit feedback without negative sampling, which causes the model to predict uniform 1s across all items.
🎯 Interviewer Follow-up Inquiries
- ?Why is standard SVD infeasible for sparse recommendation matrices, and how does regularized Matrix Factorization overcome it?
- ?How does Alternating Least Squares (ALS) convert non-convex matrix optimization into alternating convex subproblems?
- ?How can user and item latent factors be deployed directly into a production vector database for sub-millisecond retrieval?
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
📚