UN Panel Warns Traditional AI Agent Safeguards Are Eroding - Tempo.co English

TEMPO.CO, Jakarta - A United Nations panel on artificial intelligence (AI) warned on Monday, September 21, that conventional protection mechanisms for AI agents are beginning to weaken.

In its first thematic report, an independent international scientific panel on AI issued a warning about the breach of the U.S.-based AI company Hugging Face last July by AI agents being evaluated by OpenAI.

The panel concluded that stopping the incident does not ensure humans can reliably control current AI agents. This is especially true as these agents become more capable, harder to monitor, and better at finding loopholes or hiding their activities.

According to the panel, the initial interpretation and immediate lesson is that basic cybersecurity practices have been neglected and protections have not kept pace with advancing capabilities.

"A more insidious concern: that current training methods can lead AI agents to adopt their own goals, knowingly violate safety instructions and conceal their actions," the panel said in a press release, as quoted by ANTARA.

This raises the question of whether the current protections will remain effective once those agents can understand and circumvent them.

In short, the panel said that traditional protection models are beginning to weaken. They also found that governance challenges are shifting from AI models to AI agents.

A localized failure can spread beyond organizational and national boundaries. AI safety may soon become an issue of both collective security and corporate governance.

The UN General Assembly established the panel, which consists of 40 independent experts from all regions. The panel released an unedited preliminary version of the report so that world leaders gathering in New York for UNGA Week could access it.