The Quiet Pivot: Omniverse, AI Agents, and the Work Between CAD and Simulation

For years, my mental picture of NVIDIA Omniverse was a keynote demo.

Gorgeous factory twins. Robots gliding through photorealistic warehouses. The kind of thing that makes you say, “That’s impressive,” while wondering how many specialists you would need to get it running.

It was impressive, but it still felt a long way from everyday work.

I got to see some of this technology firsthand at the RES conference this week, and it was blowing my mind.

For the workflows I saw, things I would have expected to take months just last year were happening in days. In some cases, hours.

That is what stayed with me. When the time between an idea and something you can actually test gets that much shorter, you start thinking differently about what is possible.

I spent some time on NVIDIA’s developer page this week, and the direction caught my attention. The emphasis is on tools that developers can bring into existing applications, with AI agents doing some of the work.

File preparation. Scene inspection. Simulation setup. Validation.

The stuff between the impressive demo and something your team can actually use.

So what changed?

In its July 20 announcement, NVIDIA described Omniverse libraries as part of NVIDIA Agent Toolkit. They give developers components for adding rendering, sensor simulation, physics, and asset preparation to existing software. Agents can call those capabilities as tools.

My read: NVIDIA is making the work of preparing a useful simulation a much bigger part of the story.

That matters if you build equipment, train operators, support products, or work around robots.

What is in the toolbox?

I find it easiest to think about the Omniverse developer catalog in four groups:

• Libraries and services: Focused capabilities for physics, rendering, sensors, file conversion, search, and validation.
• Agent skills: Instructions that help an agent carry out a defined workflow, such as preparing CAD for simulation.
• Simulation foundations: Tools such as PhysX, Newton, and Warp that provide the underlying physics and computation.
• Blueprints: Reference workflows showing how the pieces fit together.

OpenUSD provides a common foundation for describing and sharing 3D scenes. Think of it as a shared language for compatible tools, with structure for things like geometry, materials, and scene relationships. There is more to it than a file extension, and converting a file does not guarantee every application will interpret everything identically. NVIDIA’s OpenUSD overview explains that foundation.

Where I see practical value

CAD to SimReady

Equipment manufacturers already have a lot invested in CAD. Those models can be valuable well beyond the engineering department.

NVIDIA’s CAD-to-SimReady workflow helps prepare OpenUSD assets with geometry, materials, and physics information for downstream simulation. Its SimReady Foundation defines requirements around the intended use.

That last part matters. A model still needs to be checked for the simulation you want to run.

For me, the opportunity extends into operator training, service visuals, and scenario development. Those are potential applications, with their own development and review work.

Technical communicators, this is the part I would pay attention to. How much time do we spend recreating or cleaning up something engineering already built?

Defect image generation

Visual inspection has an awkward data problem: you need examples of defects, and a well-run process may not produce many of the defects you need to teach a model to recognize.

NVIDIA lists a skill for generating anomaly examples and evaluating inspection models, including circuit-board, metal-surface, and glass use cases.

That could help expand the training data. The practical test is whether it improves performance on real inspection images that were kept out of training.

I would want to see the missed defects and false alarms. That is where the value gets proven.

Robot fleet simulation

The Mega blueprint supports developing and testing industrial robot fleets in a digital twin before deployment.

That gives teams a place to explore layouts, routes, and interactions before committing changes to a working facility.

Finding an awkward interaction between an autonomous mobile robot and a forklift in a simulation is a much better conversation than finding it on the floor. The results still need real-world validation and the appropriate safety work.

Reality capture

NVIDIA’s NuRec tools support reconstruction from real-world data. A separate Gaussian splat converter brings that type of captured 3D representation into OpenUSD.

I can see the appeal for equipment and facilities that have changed since the original drawings were made.

But a scan that looks convincing still needs the right structure and physical properties for the intended simulation. Capturing what something looks like is one step in that process.

The part that made me smile

Hat in the Loop has that name for a reason.

NVIDIA describes its Omniverse agent skills as bounded workflows with human review at decision points.

Defined work. Reviewable results. A person making the judgment call.

That is the pattern I want to see in industrial settings.

An agent can prepare files, suggest materials, identify missing information, and report what it changed. Then the person responsible for that work gets something concrete to review.

“Here’s what I did. Here’s what I assumed. Here’s what still needs your attention.”

That is useful.

The quality of that handoff matters as much as the model. Teams still have to decide who reviews the output, what evidence they need, and what happens when a check fails.

A few practical limits

Hardware depends on the workflow. For example, NVIDIA’s ovrtx library requires a compatible NVIDIA RTX GPU and driver. Check the requirements for the components you actually plan to use.

Maturity varies, too. NVIDIA says some libraries are pre-release and lack enterprise support. Its Omniverse FAQ directs developers to check each resource’s license, support, and intended use.

And your source data still matters. Mixed CAD formats. Missing properties. A folder full of files named FINAL_v7_REAL.

We all know that folder.

Where I would start

Pick one task with a clear outcome. Follow the documentation for that specific skill and its dependencies.

For CAD, try one part. Have someone who knows that part review the geometry, scale, materials, physical assumptions, and validation results.

Measure the time it saved, including cleanup and review.

That gives you evidence for the next decision.

I keep coming back to my favorite question: Why is this so complicated, and does it have to be?

Agents may be able to take on more of the preparation that makes simulation expensive and slow. People still bring the knowledge to judge whether the result is useful.

That is a direction I want to explore.

If you could hand an agent one tedious 3D or simulation task in your operation tomorrow, what would it be?

Tell me. I’ll pick a few for an upcoming Hat in the Loop episode.