OOF Target Encoding & Anti-Leakage

Out-of-Fold (OOF) Target Encoding is the gold-standard dense representation method for high-cardinality categorical features (ZIP codes, merchant IDs, device models with $10^4+$ categories) engineered with strict anti-leakage invariants; standard Target Encoding replaces categorical levels with smoothed conditional expectations: $\hat{x}_i = \lambda(n) \bar{y}_c + (1 - \lambda(n)) \bar{y}_{\text{global}}$ (where $\lambda(n)$ is a Bayesian shrinkage weighting factor based on class count $n$); to prevent catastrophic target leakage, OOF splits training data into $K$ folds: target encodings for Fold $k$ are computed exclusively from the remaining $K-1$ folds; test sets are mapped using global smoothed means from the full training set.

High-cardinality feature compression in tabular ML, ad ranking CTR merchant features, and Kaggle competitive modeling.

One-Hot encoding blows up dimensionality and memory; naive target encoding memorizes target labels directly into features causing massive over-fitting; OOF guarantees leak-free, high-signal continuous features.

Test Your Knowledge: Practice Questions for "OOF Target Encoding & Anti-Leakage"

Single choice pitfall questions with instant feedback and mistake tracking.