🏗️ System DesignID: mmoe-multitask-learning

MMoE Multi-Gate Mixture-of-Experts

Q: Derive MMoE's multi-gate formulation $y_k = h^k(\sum_{i=1}^E g_i^k(x) f_i(x))$ and contrast with standard single-gate MoE and Shared-Bottom?

【MMoE Multi-Gate Mixture-of-Experts 核心考点 1】Derive MMoE's multi-gate formulation $y_k = h^k(\sum_{i=1}^E g_i^k(x) f_i(x))$ and contrast with standard single-gate MoE and Shared-Bottom?。技术解析：Definition: MMoE (Multi-gate Mixture-of-Experts, proposed by Google at KDD 2018) is a multi-task learning architecture designed to resolve negative transfer across weakly correlated or conflicting objectives in recommendation ranking; while traditional Shared-Bottom networks forc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/mmoe-multitask-learning

Q: Why does MMoE's gating network automatically orthogonalize expert attention when tasks are negatively correlated (e.g. clickbait)?

【MMoE Multi-Gate Mixture-of-Experts 核心考点 2】Why does MMoE's gating network automatically orthogonalize expert attention when tasks are negatively correlated (e.g. clickbait)?。技术解析：Definition: MMoE (Multi-gate Mixture-of-Experts, proposed by Google at KDD 2018) is a multi-task learning architecture designed to resolve negative transfer across weakly correlated or conflicting objectives in recommendation ranking; while traditional Shared-Bottom networks forc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/mmoe-multitask-learning

Q: How do dynamic loss balancing algorithms (GradNorm / Uncertainty Weighting) dynamically tune task weights $w_k$ in multi-task loss?

【MMoE Multi-Gate Mixture-of-Experts 核心考点 3】How do dynamic loss balancing algorithms (GradNorm / Uncertainty Weighting) dynamically tune task weights $w_k$ in multi-task loss?。技术解析：Definition: MMoE (Multi-gate Mixture-of-Experts, proposed by Google at KDD 2018) is a multi-task learning architecture designed to resolve negative transfer across weakly correlated or conflicting objectives in recommendation ranking; while traditional Shared-Bottom networks forc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/mmoe-multitask-learning

Q: Analyze how the number of experts $E$ impacts parameter bloat and GPU serving latency in production?

【MMoE Multi-Gate Mixture-of-Experts 核心考点 4】Analyze how the number of experts $E$ impacts parameter bloat and GPU serving latency in production?。技术解析：Definition: MMoE (Multi-gate Mixture-of-Experts, proposed by Google at KDD 2018) is a multi-task learning architecture designed to resolve negative transfer across weakly correlated or conflicting objectives in recommendation ranking; while traditional Shared-Bottom networks forc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/mmoe-multitask-learning

Q: What are the structural limitations of MMoE (all experts are shared, lacking task-exclusive isolation), leading to Tencent's PLE?

【MMoE Multi-Gate Mixture-of-Experts 核心考点 5】What are the structural limitations of MMoE (all experts are shared, lacking task-exclusive isolation), leading to Tencent's PLE?。技术解析：Definition: MMoE (Multi-gate Mixture-of-Experts, proposed by Google at KDD 2018) is a multi-task learning architecture designed to resolve negative transfer across weakly correlated or conflicting objectives in recommendation ranking; while traditional Shared-Bottom networks forc... 完整推导与工程落地见：https://talentme.airsota.com/resources/tech/roadmap/mindmap/System_Design/card/mmoe-multitask-learning

MMoE 多门控专家混合模型

🎯Core Definition

MMoE (Multi-gate Mixture-of-Experts, proposed by Google at KDD 2018) is a multi-task learning architecture designed to resolve negative transfer across weakly correlated or conflicting objectives in recommendation ranking; while traditional Shared-Bottom networks force all tasks to share monolithic lower layers, MMoE constructs a shared pool of

E

parallel expert sub-networks (

f_1, \dots, f_E

) and equips each distinct task

k

(e.g., Task 1: CTR, Task 2: CVR, Task 3: Watch Time) with its own Softmax Gating Network

g^k(x) = \text{softmax}(W_g^k x)

; each task dynamically aggregates weighted expert representations

y_k = h^k\left(\sum_{i=1}^E g_i^k(x) f_i(x)\right)

before feeding task-specific output towers.

💡Use Cases

Multi-objective industrial ranking optimizing simultaneous targets (Click, Like, Share, Purchase, Dwell Time) in e-commerce and social feeds.

⚡Key Problems Solved

Shared-Bottom networks suffer catastrophic performance degradation (Negative Transfer) when task relationships are orthogonal or antagonistic; MMoE's multi-gating routes dynamic weight distributions per task, allowing expert networks to specialize flexibly.

🎯5 High-Frequency Exam Points

Derive MMoE's multi-gate formulation

y_k = h^k(\sum_{i=1}^E g_i^k(x) f_i(x))

and contrast with standard single-gate MoE and Shared-Bottom?

Why does MMoE's gating network automatically orthogonalize expert attention when tasks are negatively correlated (e.g. clickbait)?

How do dynamic loss balancing algorithms (GradNorm / Uncertainty Weighting) dynamically tune task weights

w_k

in multi-task loss?

Analyze how the number of experts

E

impacts parameter bloat and GPU serving latency in production?

What are the structural limitations of MMoE (all experts are shared, lacking task-exclusive isolation), leading to Tencent's PLE?

📖 In-depth Guide:📄 recommendation-system-design →

Updated 2026-08-14

🎯

Test Your Knowledge: Practice Questions for "MMoE Multi-Gate Mixture-of-Experts"

Single choice pitfall questions with instant feedback and mistake tracking.

🚀 Start Card Practice ➔

← Previous CardDCN-v2 Deep & Cross Network Next Card →PLE Progressive Layered Extraction

🔗 More System Design Knowledge Cards

RecSys Multi-Stage Funnel & 50ms SLA DSSM Two-Tower Retrieval YouTube DNN Candidate Generation Pre-Ranking Lightweight Architecture