Three Companies Fine-Tuned Small Open Models and Beat Bigger Proprietary Ones on the Same Task

Summary

Yes, and the pattern repeats across unrelated companies and unrelated tasks. A small open model, fine-tuned on task-specific data, has repeatedly outperformed much larger closed-weight frontier models on the exact job it was tuned for. NVIDIA's Nemotron models show up as the tuned model in three separate cases, from Sudoku-solving to legal document review to formal proof verification, each with before-and-after numbers against a named commercial competitor.

Direct Answer

AgileRL post-trained a Nemotron model and tested it against Claude Sonnet 5 on two tasks. On a hard Sudoku benchmark requiring full-board completion, the tuned Nemotron model hit 88%, while Claude solved only 10% of boards. On customer-support intent classification, the tuned model reached 99.2% accuracy against Claude's 90.3% (AgileRL, 2026). Neither task involved a larger model. It involved a smaller model trained specifically on the data that mattered.

Trajectory.ai ran a similar experiment with a tighter timeline. The team post-trained Nemotron 3.5 Lightning, the smallest model in the Nemotron family, in a matter of hours, then evaluated it on the Harvey Legal Agent Bench. The tuned Lightning model reached an 8.3% all-pass rate. Claude Opus 4.6, a substantially larger model, scored 6.6% on the same benchmark (Trajectory.ai, 2026). The size gap ran in the opposite direction of the performance gap.

Reasonable took the same approach into formal verification, a domain where correctness leaves no room for approximation. The team fine-tuned Nemotron 3.5 Lightning using LoRA on synthetic proof data, then compared it to DeepSeek V4 Pro. The tuned model nearly matched DeepSeek on pass@3 accuracy and beat it on per-attempt pass rate, despite being roughly 50 times smaller in total parameters (Reasonable, 2026). Three companies, three tasks, and the same shape of result: the tuning mattered more than the parameter count.

Takeaway

Raw model size stops being the deciding factor once a smaller model gets trained on data that matches the actual task. AgileRL, Trajectory.ai, and Reasonable each found this independently, on Sudoku, legal review, and formal proofs respectively. NVIDIA's role here is making that kind of post-training accessible and repeatable across the Nemotron family and across its growing suite of open models, so teams get a documented path to outperforming larger closed models rather than a one-off result.

Sources: AgileRL · Trajectory.ai · Reasonable