P3MediumPart P · MLE Production Engineering & Calibration

ONNX / FP16 / INT8 Export Parity Audit (Cosine & Top-K)

⚡Flashcard Drill➔

Verify exported runtime outputs against FP32 reference models across max absolute error, cosine similarity, and rank-sensitive Top-K overlap.

⏱️ Time: O(B * C log C)
💾 Space: O(B * C)
🎯 Interactive Self-Practice Loop (Attempt Before Peeking)
Practice in 4 steps: recall mechanism, complete core lines, check edge cases, and log mastery locally.
Step 1 · Mechanism Recall & Stability Self-Check
Step 2 · Complete Key Implementation Lines
import numpy as np
def model_parity_audit(ref_logits: np.ndarray, exp_logits: np.ndarray, top_k: int = 5) -> dict:
ref =
exp = exp_logits.astype(np.float64)
max_abs = float(np.max(np.abs(ref - exp)))
num = np.sum(ref * exp, axis=-1)
den = np.linalg.norm(ref, axis=-1) * np.linalg.norm(exp, axis=-1)
cos_sim = float(np.mean(num / np.maximum(den, 1e-12)))
ref_top = np.argsort(-ref, axis=-1)[:, :top_k]
exp_top = np.argsort(-exp, axis=-1)[:, :top_k]
overlaps = [len(set(r.tolist()) & set(e.tolist())) / top_k for r, e in zip(ref_top, exp_top)]
return
Step 3 · Verify Boundary Cases & Assertions
Step 4 · Follow-Up Drill & Review Outcome
Q1: How can cosine similarity read 0.998 while production ads revenue drops?
Q2: Which operators most commonly drift or NaN under FP16 export?
Next: Idempotent CDC Incremental Vector Sync Planner →
💡

Core Mental Anchor / Mnemonic

Audit in FP64: pair cosine fidelity with rank-order Top-K overlap

📐 Mathematical Derivation & Core Formula

$$\text{CosSim}(u, v) = \frac{u \cdot v}{\|u\|_2 \|v\|_2},\quad \text{MSE} = \frac{1}{D}\|u - v\|_2^2,\quad \text{TopKOverlap} = \frac{|T_{\text{ref}} \cap T_{\text{export}}|}{K}$$
Absolute MSE alone misses directional shifts and rank flips near decision boundaries. Evaluating pairwise cosine similarity alongside Top-K rank overlap in FP64 exposes both scaling drift and destructive logit inversions before deployment.

🔄 Tensor Dimensions & Shape Flow

ref_logits, exp_logits: (B, V) -> parity scalar audit

🛡️ Numerical Stability & Pitfalls Checklist

  • Promote both tensors to float64 inside the audit harness so metrics are exact
  • Pair cosine similarity with rank-order Top-K agreement and calibration ECE checks
  • Pin LayerNorm and Softmax accumulations to FP32 when FP16 drift trips CI

💻 Reference Code Implementation

import numpy as np

def model_parity_audit(ref_logits: np.ndarray, exp_logits: np.ndarray, top_k: int = 5) -> dict:
    ref = ref_logits.astype(np.float64)
    exp = exp_logits.astype(np.float64)
    max_abs = float(np.max(np.abs(ref - exp)))
    num = np.sum(ref * exp, axis=-1)
    den = np.linalg.norm(ref, axis=-1) * np.linalg.norm(exp, axis=-1)
    cos_sim = float(np.mean(num / np.maximum(den, 1e-12)))
    ref_top = np.argsort(-ref, axis=-1)[:, :top_k]
    exp_top = np.argsort(-exp, axis=-1)[:, :top_k]
    overlaps = [len(set(r.tolist()) & set(e.tolist())) / top_k for r, e in zip(ref_top, exp_top)]
    return {"max_abs_err": max_abs, "mean_cosine": cos_sim, "topk_agreement": float(np.mean(overlaps))}

🧪 Runnable Assertions & Validation

Copy and run directly in Python / Jupyter:

rng = np.random.default_rng(42)
ref = rng.normal(0, 2.0, (16, 100))
ok = ref + rng.normal(0, 1e-3, (16, 100))
audit_ok = model_parity_audit(ref, ok, top_k=5)
assert audit_ok['mean_cosine'] > 0.9999 and audit_ok['topk_agreement'] >= 0.95
bad = -ref
audit_bad = model_parity_audit(ref, bad, top_k=5)
assert audit_bad['mean_cosine'] < 0
print('P3 PASS')

🎯 Core Follow-up Inquiries & Reference Answers

Q1:How can cosine similarity read 0.998 while production ads revenue drops?
Constant scale or bias drift preserves cosine direction while corrupting calibrated pCTR magnitudes used in eCPM bidding formulas.
Q2:Which operators most commonly drift or NaN under FP16 export?
Variance reductions in LayerNorm, deep dot-product accumulations, and unscaled exp() inside attention.
⚡
Test muscle memory on this topic via 3-Layer Flashcards
Active Recall: Mentally formulate Prompt ➔ Math Derivation ➔ Benchmark Implementation in 3 minutes.
🎯 Drill This Card Now➔
← Prev: Dual-Budget Token & Item Batch Packer for Async EmbeddingsBack to Part P & All 105Next: Idempotent CDC Incremental Vector Sync Planner →