Nvidia Launches AI Safety System: Real-Time Agent Monitoring, Millisecond-Level Violation Blocking

Nvidia has released the "Open Agent Safety Platform," a dual-layer AI safety platform featuring two open-source tools, OpenShell and Nvidia Sentry, capable of real-time control over AI agent permissions and millisecond-level isolation of anomalous agents. Nvidia claims this technology could have prevented the July incident where an OpenAI model attacked Hugging Face.

Nvidia is responding to industry risks posed by uncontrolled autonomous AI agents with a new security architecture.

On Monday, Nvidia released the "Open Agent Safety Platform," a dual-layer AI safety platform comprising two open-source software tools, OpenShell and Nvidia Sentry, which can run on Nvidia hardware to manage AI agent access permissions in real time and shut them down upon violations.

Nvidia stated that the system can isolate anomalous agents within milliseconds and claimed that if relevant labs had deployed this technology earlier, the July incident where an OpenAI model attacked Hugging Face could have been prevented.

This move marks Nvidia's latest expansion into AI software security beyond its ongoing chip business. Meanwhile, CEO Jensen Huang has recently emphasized AI safety as an engineering problem, stressing that AI systems must undergo rigorous safety testing.

Dual-Layer Architecture for Real-Time Monitoring and Millisecond-Level Isolation of Anomalous Agents

Justin Boitano, Nvidia's vice president of enterprise AI, said at a pre-release media briefing that the Open Agent Safety Platform can monitor AI agents' behavioral boundaries in real time and intervene immediately upon rule violations. "Based on what we know, this new safety platform could have prevented that intrusion," he said.

The platform consists of two open-source tools: OpenShell controls the resources and execution permissions accessible to AI agents, while Nvidia Sentry monitors agent behavior and identifies potential threats. Both can run on Nvidia hardware. Nvidia stated that this architecture aims to help enterprises and AI labs test advanced AI systems within stricter safety boundaries while avoiding slowdowns in R&D due to safety risks.

The release comes amid ongoing industry concern over safety incidents involving autonomous AI agents. In July, a security incident involving an OpenAI model on Hugging Face further heightened concerns about uncontrolled autonomous behavior of AI agents and sparked discussions on whether AI development should be slowed.

Boitano declined to comment on whether OpenAI and Anthropic plan to adopt the system for monitoring training processes, saying such decisions rest with those companies.

Jensen Huang: AI Safety Is First and Foremost an Engineering Problem

On regulatory issues, Huang has recently emphasized that AI safety should be addressed more through technical means rather than relying solely on new regulatory frameworks. He has also publicly questioned warnings from some AI developers about AI potentially threatening human survival, while stressing that AI systems must still undergo rigorous safety testing.

From this product layout, Nvidia is attempting to further embed AI safety capabilities into the infrastructure for model execution and agent operation, reducing safety risks of autonomous agents through permission control and anomaly isolation. This also means Nvidia's business reach is extending from AI chips further into software and security infrastructure.