RL Training/Serving (cross-module)

RL training/serving is the sampling-and-verification infrastructure for RLVR (RL with Verifiable Rewards): training samples $N$ rollout trajectories per prompt and scores/filters them with a verifier; serving hosts test-time scaling through extended generation.

training and test-time scaling for tasks with verifiable rewards (math/code), e.g. o1-style reasoning models; rollout clusters, verifiers, and test-time compute are common RL-infra interview topics.

when human reward labeling does not scale, RLVR substitutes verifiable rewards plus massive parallel sampling, with verifiers automatically filtering low-quality trajectories; at inference, longer thinking / repeated sampling trades compute for accuracy. See the RL module: algorithm and training details in the RL module (guide agentic-rl-and-reasoning-search).

RLVR vs RLHF (verifiable rewards vs reward models) — how to design the sampling-verification pipeline?

Large-scale rollout infra: how to schedule training/serving co-location, sampling throughput, and verifier compute?

How are verifiers built to score/filter trajectories (answer checking, unit-test execution)?

Test-time scaling: how do longer generation or repeated sampling improve accuracy, and how is cost controlled?

How is rollout data replayed into policy updates (GRPO/PPO), and how does sampling parallelism affect reward distribution?