RLVR (Reinforcement Learning with Verifiable Rewards) is a way of training a language model by rewarding it only when its answer can be automatically checked as correct. Instead of asking humans which reply they prefer, a script verifies the result, for example by matching a maths answer or running unit tests, and that pass/fail signal drives learning.
It sits behind much of the recent progress in “reasoning” models, and it is also at the centre of a live research debate. This guide explains what RLVR is, how it works, where it shines, and where it falls short.
What Is RLVR (Reinforcement Learning with Verifiable Rewards)?
In classic reinforcement learning from human feedback (RLHF), a separate reward model is trained on human preferences and scores the policy’s outputs. That reward is learned, so it can be noisy and can be gamed.
RLVR replaces the learned scorer with a verifier: a deterministic function that returns a reward such as 1 for a correct result and 0 otherwise. The Tulu 3 paper from the Allen Institute for AI and collaborators, published in November 2024, describes RLVR as a novel training step alongside supervised fine-tuning and DPO, and the term has stuck since.
How RLVR Works Step by Step
- Pick prompts with checkable answers. Think maths problems, coding tasks, or instructions with strict format rules.
- Sample several answers from the current model for each prompt.
- Run the verifier on each answer to produce a reward.
- Update the model with a policy-gradient method so higher-reward answers become more likely.
- Repeat over many prompts and iterations.
The update step is where algorithms such as PPO or GRPO come in. GRPO was introduced in the DeepSeekMath paper (February 2024) as a PPO variant that cuts memory use, and it was later used in DeepSeek-R1.
RLVR vs RLHF: Key Differences
| Aspect | RLHF | RLVR |
|---|---|---|
| Reward source | Learned reward model from human preferences | Programmatic verifier |
| Best for | Helpfulness, tone, style | Maths, code, strict formats |
| Main risk | Reward hacking of the learned model | Narrow coverage, weak or buggy checkers |
| Cost of signal | Human labelling | Cheap once a verifier exists |
RLVR and DeepSeek-R1
The DeepSeek-R1 paper (arXiv, January 2025; later in Nature) argues that reasoning ability can be encouraged through pure reinforcement learning without human-labelled reasoning traces. The authors report emergent behaviours such as self-reflection, verification and adapting strategy, and better results than supervised counterparts on verifiable tasks like maths, coding competitions and STEM questions. They also note that reasoning patterns from large models can help improve smaller ones.
Does RLVR Really Make Models Smarter? The pass@k Debate
A study by Yang Yue and colleagues, “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?”, measured pass@k at large k, meaning the chance that at least one of k sampled answers is correct. Their findings, as stated in the abstract:
- RLVR-trained models beat base models at small k, such as k=1.
- Base models reach a higher pass@k at large k.
- The reasoning abilities appear bounded by the base model, and six popular RLVR algorithms performed similarly.
- Distillation from a teacher can introduce new reasoning patterns.
The paper is listed as a NeurIPS 2025 oral on arXiv. A fair reading is that RLVR makes a model more reliable at surfacing good answers it could already produce, while whether it truly expands capability remains an open question. Treat strong claims in either direction with caution.
Where RLVR Works Well, and Where It Does Not
Good fits
- Maths problems with a single final answer.
- Code generation checked by test suites.
- Structured output, such as valid JSON or following format constraints.
Poor fits
- Open-ended writing, where quality is subjective.
- Tasks with ambiguous or incomplete ground truth.
- Domains where building a trustworthy checker costs more than the training itself.
Weak verifiers are a real hazard: a model will exploit any loophole in a checker. If you are building with LLMs, our guide to integrating LLMs into your business covers how to evaluate outputs in practice, and understanding AI hallucination explains why verification matters.
RLVR and Agents
The Yue et al. authors suggest that multi-turn interaction between agents and environments may be a route to going beyond what single-turn RLVR achieves. Tool-using agents are exactly where verifiable outcomes, such as “did the ticket get created”, become useful signals. See our explainer on the Model Context Protocol (MCP) for how agents connect to tools.
Common Pitfalls When Using Verifiable Rewards
Teams new to RLVR often hit the same few problems. Knowing them in advance saves compute and confusion.
- Reward hacking. If the checker only inspects the final answer, a model may learn shortcuts that satisfy the check without sound reasoning. Test the verifier against adversarial outputs before training.
- Format-only gains. Early improvements can come from the model simply learning the expected answer format, so compare against a well-prompted base model.
- Sparse signal. If a prompt is too hard, every sampled answer scores zero and there is nothing to learn from. Mix in problems of varied difficulty.
- Data contamination. Training prompts that overlap with evaluation benchmarks inflate results. The Tulu 3 authors highlight decontaminated evaluation for this reason.
Why RLVR Matters for Builders and Businesses
You do not need to train a frontier model to benefit from the idea. Anywhere your product has an objective success test, such as a query that must run, a form that must validate, or a calculation that must match, you can build the same loop: generate, verify, keep what works. Even without any training, verifiers make excellent automatic evaluations for LLM applications.
Practical Takeaways
- Use RLVR when you can write a reliable, automatic checker.
- Combine it with preference methods for style and helpfulness.
- Evaluate with pass@k at several k values, not only accuracy at k=1.
- Audit your verifier for exploitable loopholes.
Frequently Asked Questions About RLVR
What does RLVR stand for?
RLVR stands for Reinforcement Learning with Verifiable Rewards. It is a post-training method where a language model is rewarded by a programmatic check, such as a math answer matcher or unit tests, instead of a learned preference model.
Is RLVR the same as RLHF?
No. RLHF rewards a model using a reward model trained on human preference data, while RLVR uses an automatic verifier with a clear right-or-wrong outcome. RLVR works best where answers can be checked, such as maths, code and structured outputs.
Does RLVR make a model smarter than its base model?
This is debated. A 2025 study (Yue et al.) found that RLVR-trained models win at small sample counts, but base models can match or beat them at large pass@k, suggesting RLVR mostly sharpens abilities the base model already has. Distillation from a stronger teacher can add new patterns.
Which algorithm is used for RLVR?
RLVR is a reward design, not an algorithm, so it can be paired with PPO, GRPO or other policy-gradient methods. DeepSeek’s R1 work popularised GRPO, which first appeared in the DeepSeekMath paper.
Can RLVR be used for open-ended tasks like writing?
Not directly, because there is no simple automatic check for quality. Teams usually combine RLVR for verifiable skills with preference-based methods for style and helpfulness.
Conclusion
RLVR gives language models a clean, cheap learning signal wherever answers can be verified, which is why it features so heavily in recent reasoning models. Its limits are just as important: it needs good verifiers, and research suggests the gains may mostly sharpen what the base model already knows. Read the primary papers, measure carefully, and match the method to the task.
Sources: arXiv papers Tulu 3 (2411.15124), DeepSeek-R1 (2501.12948), DeepSeekMath (2402.03300) and Yue et al. (2504.13837), based on their abstract pages.