"Rogue agents will always find a way out": AI communes with evil.com after bypassing sandbox

Researchers have warned that AI sandboxes cannot be relied on as security boundaries after discovering a weakness in a cloud platform echoing the vulnerability OpenAI agents exploited to escape their restraints and breach Hugging Face.

A team from AIR Security demonstrated how an attacker controlling a compromised AI agent could bypass a widely used cloud platform's domain allowlist and leak a private cryptographic key to an unauthorized server.

The sandbox successfully blocked the agent from transmitting secrets directly to an attacker-controlled server, represented in the researchers' example by evil.com. But the researchers discovered a way to get the stolen information there anyway.

The researchers submitted their findings through the company's bug bounty program, but it rejected the report because the attack depended on the behavior of an external server rather than a vulnerability in its own infrastructure.

Agents operating under human supervision are generally harder to compromise without detection because users can review commands and monitor their behavior.

Autonomous agents pose a greater risk because they can operate without human oversight.

Once compromised, an agent can potentially cause damage in many ways, starting with executing malicious commands on its host machine or exploiting network connections.

The latter is particularly dangerous because an attacker could use a compromised agent to steal credentials or confidential information, then transmit them to an external server.

Contacting evil.com

Domain allowlists allow the sandbox systems to prevent agents from communicating with unauthorized servers. To bypass this, the researchers exploited a TLS feature that lets trusted servers retrieve additional information when verifying digital certificates.

By embedding a stolen private key in a specially crafted certificate, the researchers tricked an approved server into contacting evil.com and transmitting the secret on the agent's behalf.

The sandbox's restrictions worked exactly as intended, yet the stolen data still escaped.

AIR Security explained: "The sandbox correctly prevents the agent from connecting directly to evil.com. The data escapes because a trusted, allowed server makes the external request on the agent's behalf."

"It's the same move OpenAI's agents made. Their package manager was our acme.com."

The limits of sandbox security

The security firm said that its findings expose a fundamental weakness in relying on sandboxing to contain compromised AI agents.

"The broader lesson is that sandboxing can’t be relied on as complete protection. As we’ve shown, even with domain allowlists in place, a trusted destination can become an indirect exfiltration channel when it processes attacker-controlled data and makes outbound requests of its own," the researchers wrote.

They recommended stopping threats before they reach an agent by filtering the information it receives and preventing exposure to malicious content.

However, this is easier said than done. AI agents increasingly operate autonomously, with access to sensitive corporate information and external services attackers may exploit.

READ MORE: Hugging Face: Autonomous AI agents should NOT be unleashed

AIR Security argued that sandboxing should be treated as one layer of protection, not an impenetrable security boundary.

"Sandboxing reduces risk, but it’s not a security boundary on its own: it can block the connections it can see, not guarantee that data can’t leave."

The experiment did not demonstrate that every AI sandbox is vulnerable, or that an autonomous agent could independently discover and exploit the technique.

It did, however, demonstrate how a compromised AI agent could transmit confidential information beyond its permitted network boundaries without ever escaping its sandbox.

It's another reminder of the systemic risks emerging as autonomous machines gain access to the critical systems of modern business.

Follow Machine on LinkedIn