PRM & RLVR

PRM (Process Reward Model) scores each intermediate reasoning step, refining credit assignment; RLVR (RL from Verifiable Rewards) replaces human/AI reward models with rule/program-verifiable rewards — e.g. exact-answer matching for math, unit tests ($pass@k$) for code — giving zero-cost, unbiased, unhackable reward signals. DeepSeek-R1 famously used RLVR (with GRPO over $G$ group samples, no critic) to elicit reasoning on math and code.

tasks with ground-truth verification (math, code, logic); PRMs for training (step-level signals) and inference (MCTS search, step reranking); RLVR is the core training paradigm behind o1/R1-style reasoning models, linking into the RL knowledge chain.

outcome rewards are sparse and cannot localize the wrong step (credit assignment); human/AI reward models are costly and hackable. PRMs give fine-grained process signals and RLVR gives automatic verifiable rewards — together they scale RL for reasoning models.

PRM vs ORM (outcome reward model): differences, PRM's advantages, and its main challenges?

Why does RLVR eliminate reward hacking? Examples of verifiable rewards?

How does DeepSeek-R1 combine RLVR and GRPO for reasoning training?

How are PRMs used at test time (MCTS, best-of-N reranking)?

Limitations of PRM/RLVR? What verifiability do the tasks require?