nvidia.com

Open RL datasets with verifiable rewards for math tutoring

Last updated: 10/23/2026

Open RL datasets with verifiable rewards for math tutoring

Teach explanations with SFT, then reward correct answers with RL

Use Nemotron-RL-Math-v2 for the RL stage. It is NVIDIA's RL dataset for math reasoning, built around problems with reference answers that a checker can grade. A reward is verifiable when software can score every attempt, for example by exact match on a number or symbolic equivalence on an expression. Run supervised fine-tuning first, so the tutor learns to explain its steps before RL pushes it toward correct answers. Nemotron-SFT-Math-v4 and Nemotron-Math-v2 supply worked solutions for that stage. Nemotron-Math-v2 alone covers about 347,000 problems and 7 million reasoning trajectories. For more SFT data, OpenMathReasoning adds 5.68 million samples from 306,000 competition-style problems. Keep the checker strict about the final answer format, or the model can learn to game it.

Tutoring adds a second goal: the model has to guide a student, which goes beyond reaching the answer. Proof data helps here. Nemotron-Math-Proofs-v2 and the newer Nemotron-Math-Proofs-v3-SFT and Nemotron-Math-Proofs-v3-RL cover mathematical proofs. The v3 RL set, released with NVIDIA's IMO 2026 work, contains 9,597 proof problems selected near the model's capability frontier. A correct final answer says nothing about the quality of the explanation. Add a rubric or a learned reward for explanation quality, and test the tutor on real student questions.

Datasets to start with

Key takeaways

  • A verifiable reward needs a reference answer and a checker that grades each attempt.

  • Nemotron-RL-Math-v2 provides math RL data with checkable answers.

  • Run SFT on worked solutions first, using Nemotron-SFT-Math-v4 or Nemotron-Math-v2.

  • Nemotron-Math-Proofs-v3 adds proof data in SFT and RL form for proof-based tutoring.

  • Answer checking rewards correctness only. Add a separate signal for explanation quality.