OOF Target Encoding & Anti-Leakage

Out-of-Fold (OOF) Target Encoding is the gold-standard dense representation method for high-cardinality categorical features (ZIP codes, merchant IDs, device models with $10^4+$ categories) engineered with strict anti-leakage invariants; standard Target Encoding replaces categorical levels with smoothed conditional expectations: $\hat{x}_i = \lambda(n) \bar{y}_c + (1 - \lambda(n)) \bar{y}_{\text{global}}$ (where $\lambda(n)$ is a Bayesian shrinkage weighting factor based on class count $n$); to prevent catastrophic target leakage, OOF splits training data into $K$ folds: target encodings for Fold $k$ are computed exclusively from the remaining $K-1$ folds; test sets are mapped using global smoothed means from the full training set.

High-cardinality feature compression in tabular ML, ad ranking CTR merchant features, and Kaggle competitive modeling.

One-Hot encoding blows up dimensionality and memory; naive target encoding memorizes target labels directly into features causing massive over-fitting; OOF guarantees leak-free, high-signal continuous features.

Write the complete Pure Numpy/Pandas implementation of 5-fold OOF Target Encoding with Bayesian smoothing?

Mathematically explain why naive target encoding causes target leakage by contaminating feature values with the row's own label?

Explain how CatBoost's Ordered Target Encoding calculates cumulative online historical statistics to eliminate leakage naturally?

How does adding Gaussian noise/jittering to target-encoded features regularize trees against memorizing rare categories?

How should target-encoded transformers handle unseen categorical levels during online inference using global prior baselines?