Catastrophic Forgetting & Rehearsal

Mitigating Catastrophic Forgetting via Experience Rehearsal & Parameter Freezing ensures that foundation models do not degrade in broad general intelligence, coding logic, or multi-lingual proficiency while fine-tuning on narrow domain datasets; core engineering strategies span: 1) Experience Rehearsal & Pre-training Data Mixing: co-mingling a 5%-20% ratio of high-entropy general instruction data (ShareGPT, GSM8k, code tokens) with domain samples to keep general neural representations active; 2) Parameter Isolation: utilizing PEFT/LoRA to freeze $>99\%$ of original backbone weights, rendering foundational knowledge physically immutable; 3) Spherical Linear Interpolation (SLERP / Weight Merging): blending fine-tuned checkpoints with base models via $\theta_{\text{merged}} = (1-\lambda) \theta_{\text{base}} + \lambda \theta_{\text{sft}}$ to smoothen specialized parameters.

Domain-specific continuous fine-tuning, multi-task instruction scaling, and guarding against model degradation.

Naive fine-tuning on domain data triggers catastrophic regression across standard benchmarks (MMLU/GSM8k collapsing by 50%+); rehearsal and parameter isolation maintain robust generalization.

Explain why Full Fine-Tuning is significantly more susceptible to catastrophic forgetting compared to LoRA parameter isolation?

Analyze the Pareto trade-off of general-to-domain data mixing ratios (e.g. 10% general vs 90% domain) in preserving benchmark scores?

Why does Spherical Linear Interpolation (SLERP) outperform naive linear averaging when merging fine-tuned checkpoints?

How does KL-penalty loss regularization mathematically restrict policy drift away from the base model distribution during SFT?

Design an automated regression evaluation gate running MMLU, GSM8k, and HumanEval benchmarks before deploying fine-tuned weights?