⚡ LLMsID:
prm-rlvrPRM & RLVR
PRM 与 RLVR
🎯Core Definition
PRM (Process Reward Model) scores each intermediate reasoning step, refining credit assignment; RLVR (RL from Verifiable Rewards) replaces human/AI reward models with rule/program-verifiable rewards — e.g. exact-answer matching for math, unit tests () for code — giving zero-cost, unbiased, unhackable reward signals. DeepSeek-R1 famously used RLVR (with GRPO over group samples, no critic) to elicit reasoning on math and code.
💡Use Cases
tasks with ground-truth verification (math, code, logic); PRMs for training (step-level signals) and inference (MCTS search, step reranking); RLVR is the core training paradigm behind o1/R1-style reasoning models, linking into the RL knowledge chain.
⚡Key Problems Solved
outcome rewards are sparse and cannot localize the wrong step (credit assignment); human/AI reward models are costly and hackable. PRMs give fine-grained process signals and RLVR gives automatic verifiable rewards — together they scale RL for reasoning models.
🎯5 High-Frequency Exam Points
1
PRM vs ORM (outcome reward model): differences, PRM's advantages, and its main challenges?
2
Why does RLVR eliminate reward hacking? Examples of verifiable rewards?
3
How does DeepSeek-R1 combine RLVR and GRPO for reasoning training?
4
How are PRMs used at test time (MCTS, best-of-N reranking)?
5
Limitations of PRM/RLVR? What verifiability do the tasks require?
📖 In-depth Guide:📄 reasoning-and-cot →
Updated 2026-08-12🎯
🚀 Start Card Practice ➔Test Your Knowledge: Practice Questions for "PRM & RLVR"
Single choice pitfall questions with instant feedback and mistake tracking.