Journalism begins where hype ends

,,

I visualise a time when we will be to robots what dogs are to humans, and I’m rooting for the machines."

—Claude Shannon

Rogue AI Incidents Spur EU to Seek AI Model Safety Data Under EU AI Act

The European Commission's AI Office has sent formal requests for information to providers of general-purpose AI models, focusing on model security, independent external evaluations, post-market monitoring and training-data transparency.
European Commission Executive Vice-President Henna Virkkunen pictured alongside the EU Artificial Intelligence Act logo.
August 30, 2026 11:29 PM IST | Written by Vaibhav Jha

Almost a month after the European Union Artificial Intelligence (EU AI) Act came into effect, the European Commission took the first step in its enforcement by asking global AI companies to submit information regarding AI models’ safeguards, independent evaluation and monitoring.

The European Commission’s AI office, sent a formal request to AI companies in the world, reacting to recent incidents of frontier AI models carrying autonomous operations to breach third-party organizations by exploiting their vulnerabilities. The case of OpenAI’s AI agents breaching Hugging Face platform has drawn considerable limelight drawing concerns over increasing capabilities of AI models sans human control.

In a post on LinkedIn, Henna Virkkunen, Executive Vice-President for Tech Sovereignty, Security and Democracy, said, “AI models are becoming increasingly capable and gave rise to a number of incidents during the summer. The AI Act gives us the power to require AI providers to strengthen their risk-mitigation measures and help ensure that the models and systems they place on the European market are safe.”

Key provisions of the AI Act covering general-purpose AI models, transparency requirements and related enforcement powers became applicable on August 2, 2026. The EU AI Act focuses on safety, privacy and transparency aspects of AI models and tools deployed in European markets by AI companies. The Act adopts a risk-based approach and is part of a broader package of policy measures that also includes the AI Continent Action Plan, the AI Innovation Package and AI Factories.

Virkkunen also informed that the AI Office has also sought information related to training of LLMs and a reminder to those “who have not participated in informal compliance dialogues with the AI Office. This publication requirement is intended to increase transparency and help copyright holders and other parties with legitimate interests exercise their rights.

Under the AI Act, providers must design AI systems to inform users when they are directly interacting with AI and add machine-readable marks to enable the detection of AI-generated or manipulated content. Deployers must inform users when they are exposed to deepfakes, AI-generated content on matters of public interest without human review or editorial control, and emotion recognition or biometric categorization systems.

The Commission is also introducing a range of tools to help organizations understand their AI Act obligations, including the Code of Practice on Transparency of AI-generated Content and the AI Act Service Desk.

Also Read: EU Prepares for Aug 2 with AI Act Transparency Guidelines; Over 100 Experts Share Frontier AI Findings

Author

  • Vaibhav Jha, editor and co-founder at AI FrontPage

    Vaibhav Jha is an Editor and Co-founder of AI FrontPage. In his decade long career in journalism, Vaibhav has reported for publications including The Indian Express, Hindustan Times, and The New York Times, covering the intersection of technology, policy, and society. Outside work, he’s usually trying to persuade people to watch Anurag Kashyap films.

    LinkedIn