Our Take
NVIDIA is shipping the infrastructure gap between training-time prediction and deployment-time control, but this is a toolkit, not a trained model or a benchmark result.
Why it matters
AV teams have relied on open-loop training (model outputs vs. expert data) because closed-loop training is infrastructure-heavy. This framework lowers that barrier. NVIDIA is also running two public leaderboards at CVPR 2026 to establish baselines.
Do this week
AV engineers: review the NVlabs/alpamayo-recipes GitHub repository this week to understand whether AlpaGym's reward-definition and rollout-scaling approach fits your simulation pipeline before committing compute.
NVIDIA launches AlpaGym closed-loop post-training framework
NVIDIA has released AlpaGym, an open-source framework for post-training autonomous vehicle policies in closed-loop simulation. The system connects AlpaSim (NVIDIA's AV simulator), the Cosmos-RL distributed training framework, and reference reward functions into a single pipeline.
The core problem it addresses: most AV models are trained in open-loop, where model outputs are compared to ground-truth expert behaviors without considering the consequences of those outputs on the environment. In actual deployment, every steering, braking, and navigation decision affects the next state of the environment, and small errors compound. Open-loop training can miss failure modes that only emerge when a model's own actions create new situations.
AlpaGym connects policy training directly to simulator feedback. Instead of treating simulation only as final evaluation, it uses rollouts as training data. The framework runs policy inference, simulation, model training, and weight synchronization in parallel across GPU clusters. It includes a default training algorithm (GRPO), reference reward functions, and integration with NVIDIA's Physical AI AV NuRec dataset.
Installation requires CUDA dependencies, Redis, and the uv Python package manager. Training is specified via Hydra configuration files. Users define a reward function (starting with a simple progress + safety-penalty baseline), launch training on an Alpamayo checkpoint or custom model, and export the post-trained checkpoint back into AlpaSim for closed-loop rollouts. NVIDIA provides working examples and a GitHub recipe repository.
NVIDIA also announced two public leaderboards launching at CVPR 2026: the AlpaSim Closed-Loop E2E Driving Challenge and the Physical AI AV Reasoning Challenge, intended to establish baselines for closed-loop AV training methods.
Infrastructure, not a breakthrough in AV reasoning
This is an orchestration toolchain, not a new algorithm or a published benchmark showing that models trained this way outperform those trained in open-loop. The technical contribution is solving the engineering problem of running closed-loop RL at scale: syncing simulator outputs, policy weights, and training in parallel without requiring users to write custom distributed code.
Closed-loop AV training is not novel conceptually. The novelty here is the ease of use and the open-source availability. Teams that already have AV simulators and RL infrastructure can adopt AlpaGym; teams starting from scratch still need to provision the simulator, define domain-specific rewards, and tune training hyperparameters.
The public leaderboards are significant for establishing whether and how much closed-loop post-training improves safety and trajectory quality compared to open-loop baselines, but those results do not yet exist. Until independent benchmarks show measurable gains on held-out scenarios, AlpaGym is a framework that makes a known approach more accessible, not a capability leap.
What to do if you own AV training
Audit your current training pipeline for the gap between open-loop training and closed-loop deployment. If that gap is a known source of failure modes, AlpaGym is worth prototyping. Check the GitHub recipes to understand the setup cost: CUDA stack, Redis, Hydra configuration, reward function design.
If you do not yet have a custom AV simulator, AlpaGym requires AlpaSim or integration work to substitute your own. If you already have a simulator and RL framework, evaluate whether AlpaGym's asynchronous distributed design and reference reward templates reduce your engineering burden. Do not assume the public leaderboards will validate that closed-loop post-training is worth the extra compute until results are published.