P3MediumPart P · MLE Production Engineering & Calibration
ONNX / FP16 / INT8 Export Parity Audit (Cosine & Top-K)
Verify exported runtime outputs against FP32 reference models across max absolute error, cosine similarity, and rank-sensitive Top-K overlap.
⏱️ Time:
O(B * C log C)💾 Space:
O(B * C)🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def model_parity_audit(ref_logits: np.ndarray, exp_logits: np.ndarray, top_k: int = 5) -> dict:
ref =
exp = exp_logits.astype(np.float64)
max_abs = float(np.max(np.abs(ref - exp)))
num = np.sum(ref * exp, axis=-1)
den = np.linalg.norm(ref, axis=-1) * np.linalg.norm(exp, axis=-1)
cos_sim = float(np.mean(num / np.maximum(den, 1e-12)))
ref_top = np.argsort(-ref, axis=-1)[:, :top_k]
exp_top = np.argsort(-exp, axis=-1)[:, :top_k]
overlaps = [len(set(r.tolist()) & set(e.tolist())) / top_k for r, e in zip(ref_top, exp_top)]
return
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How can cosine similarity read 0.998 while production ads revenue drops?
Q2: Which operators most commonly drift or NaN under FP16 export?
💡
Core Mental Anchor / Mnemonic
Audit in FP64: pair cosine fidelity with rank-order Top-K overlap
📐 Mathematical Derivation & Core Formula
$$\text{CosSim}(u, v) = \frac{u \cdot v}{\|u\|_2 \|v\|_2},\quad \text{MSE} = \frac{1}{D}\|u - v\|_2^2,\quad \text{TopKOverlap} = \frac{|T_{\text{ref}} \cap T_{\text{export}}|}{K}$$
Absolute MSE alone misses directional shifts and rank flips near decision boundaries. Evaluating pairwise cosine similarity alongside Top-K rank overlap in FP64 exposes both scaling drift and destructive logit inversions before deployment.
🔄 Tensor Dimensions & Shape Flow
ref_logits, exp_logits: (B, V) -> parity scalar audit
🛡️ Numerical Stability & Pitfalls Checklist
- Promote both tensors to float64 inside the audit harness so metrics are exact
- Pair cosine similarity with rank-order Top-K agreement and calibration ECE checks
- Pin LayerNorm and Softmax accumulations to FP32 when FP16 drift trips CI
💻 Reference Code Implementation
import numpy as np
def model_parity_audit(ref_logits: np.ndarray, exp_logits: np.ndarray, top_k: int = 5) -> dict:
ref = ref_logits.astype(np.float64)
exp = exp_logits.astype(np.float64)
max_abs = float(np.max(np.abs(ref - exp)))
num = np.sum(ref * exp, axis=-1)
den = np.linalg.norm(ref, axis=-1) * np.linalg.norm(exp, axis=-1)
cos_sim = float(np.mean(num / np.maximum(den, 1e-12)))
ref_top = np.argsort(-ref, axis=-1)[:, :top_k]
exp_top = np.argsort(-exp, axis=-1)[:, :top_k]
overlaps = [len(set(r.tolist()) & set(e.tolist())) / top_k for r, e in zip(ref_top, exp_top)]
return {"max_abs_err": max_abs, "mean_cosine": cos_sim, "topk_agreement": float(np.mean(overlaps))}🧪 Runnable Assertions & Validation
Copy and run directly in Python / Jupyter:
rng = np.random.default_rng(42)
ref = rng.normal(0, 2.0, (16, 100))
ok = ref + rng.normal(0, 1e-3, (16, 100))
audit_ok = model_parity_audit(ref, ok, top_k=5)
assert audit_ok['mean_cosine'] > 0.9999 and audit_ok['topk_agreement'] >= 0.95
bad = -ref
audit_bad = model_parity_audit(ref, bad, top_k=5)
assert audit_bad['mean_cosine'] < 0
print('P3 PASS')🎯 Core Follow-up Inquiries & Reference Answers
Q1:How can cosine similarity read 0.998 while production ads revenue drops?
Constant scale or bias drift preserves cosine direction while corrupting calibrated pCTR magnitudes used in eCPM bidding formulas.
Q2:Which operators most commonly drift or NaN under FP16 export?
Variance reductions in LayerNorm, deep dot-product accumulations, and unscaled exp() inside attention.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.