Summary: On November 5, 2026, NVIDIA published a set of worked examples showing developers on its own teams directing frontier AI agents — primarily GPT-6 Astra, with Claude and Opus also named — to build simulation applications on NVIDIA Omniverse libraries. The demos cover everything from converting CAD files into simulation-ready robot models to stress-testing a warehouse scene at scale. For anyone building physical AI products, these are concrete reference workflows, not just a feature announcement.
What the agents actually do
The pattern across every project is the same. A developer gives a natural-language instruction, the agent does the assembly work — building OpenUSD models, configuring joints and collision geometry, writing application code — and the developer reviews the result and redirects. Omniverse libraries supply the GPU-accelerated physics, rendering and sensor simulation underneath; named libraries include ovphysx for physics, ovstage for scene updates, ovrtx for RTX rendering, ovui for interface and ovstream for streaming to a browser.
What makes this more than a demo reel is the input variety. Nick Scarcella, a senior technical artist on NVIDIA’s BondLab team, guided Astra to recreate an OP-1 portable synthesizer from product images, manuals and technical direction, then exported it into OpenUSD and tested it in Houdini and Blender. Jens Jebens, a senior product manager for OpenUSD, had Astra model a car suspension in PTC Onshape, configure it in Isaac Sim, and — after using simulation feedback to revise a bolt-reaching wrench — reported a successful component removal in simulation. David Wehr, from the simulation technology team, used Astra to interpret panoramic site photos and align generated building geometry with point-cloud measurements, producing an editable warehouse model with the shelving and machinery stripped out so layouts can be rearranged before any physical change.
The numbers worth noting
A few projects carry hard numbers that tell you what these pipelines can sustain.
Bhavik Sanganee, a principal engineer at NVIDIA, built a warehouse simulation with Astra and pushed its complexity up. Performance first dropped below target when the scene doubled in size; running physics, image rendering and video preparation concurrently restored it. At 80 conveyors, 40 arms and 800 boxes, the simulation held 30 frames per second at full HD over a two-minute test, with the arms completing 640 successful box placements. Doubling the scene again hurt performance, but testing traced the bottleneck to the physics and robot-control pipeline — a specific thing to fix, which is arguably the more useful outcome.
Paul Kind, from the Omniverse documentation team, assembled a kitchen scene by pointing an agent at the SimReady Central Library, announced at GTC Berlin, which offers more than 2,000 simulation-ready USD assets from equipment manufacturers and asset providers. His scene used 180 placements from 63 unique library assets plus 18 custom geometry pieces, reviewed through Isaac Sim renders.
Other examples show the range: Pradeep Kaushik built a gas-turbine what-if application on top of Cantera and SciPy with six test procedures including a stuck fuel valve; Oyindamola Omotuyi built a cable-routing simulation in Isaac Lab with Newton physics and a reinforcement learning controller for USB-C insertion; Frank DeLise turned a SimReady warehouse and humanoid into an interactive first- and third-person simulator.
Where the results are explicitly preliminary
The write-ups are honest about limits, and that’s useful signal. The YuMi robot asset conversion — CAD to OpenUSD with configured joints and collision geometry, validated against SimReady Foundation requirements, with four completed pick-and-place cycles — used estimated physical properties that were not calibrated against the real robot. Omotuyi’s cable controller relied on information supplied directly by the simulation rather than what a physical robot could get from cameras and sensors; testing a friction-held grasp is listed as future work. These simulations demonstrate behavior under simulated conditions, and the sources don’t establish that they match real-world physical behavior.
There’s also a completeness caveat: the published page as collected cuts off mid-sentence in the autonomous-driving section, where Doyub Kim directs Astra to build Zero to Alpamayo, a reusable simulation environment based on San Francisco’s Market Street. The details of that final project aren’t available in the excerpt, so I won’t draw conclusions from it or from where the text stops.
The scale story has a real cost dimension too. Concurrency rescued performance once, but the second doubling broke it again — the throughput you get isn’t a fixed property of the libraries; it depends on pipeline architecture, and diagnosing that is still a developer’s job even with an agent in the loop.
What a builder should do next
If you’re working in robotics, warehouse automation or industrial digital twins, the immediate move is to check whether your existing assets can enter this pipeline: CAD files, site scans and product documentation are the inputs these demos consumed. NVIDIA links a five-step SimReady tutorial for the CAD-to-asset conversion and an Onshape importer guide for the CAD-to-simulation path via SimReady Publisher — Sensori, a robotics startup, uses that path to carry its Yuri robot into Isaac Sim with geometry, joints and configuration intact.
Second, treat the SimReady Central Library as a starting inventory rather than building scenes from scratch. The simulation properties are embedded in the assets, which is what makes conversational scene assembly practical — Kind’s 180-placement kitchen came from conversation plus render review.
Third, plan for calibration. Every demo that produced a controller or a physical asset stopped short of validating against real hardware. If your roadmap depends on sim-to-real transfer, budget for that step now; the sources don’t quantify how much time or cost these agent workflows save versus traditional development, so measure it on your own first project. The bottleneck-hunting pattern from Sanganee’s warehouse test — increase load, find what breaks, fix that specific pipeline — is a repeatable method you can apply regardless of which agent you use. This fits a broader shift I covered earlier in Mistral and NVIDIA targeting engineering’s simulation bottleneck: simulation is becoming the place where AI agents earn their keep in physical industries, one verified workflow at a time.