In the wake of recent incidents involving Large Language Models (LLMs) and Agentic AI systems carrying autonomous, unsanctioned cyberattacks against third-party organizations, the UK National Cyber Security Centre (UK NCSC) has issued interim advice urging organizations to put necessary safeguards in place before embedding Agentic AI systems in their workflow.
In a detailed statement released Thursday, NCSC cautions against granting higher levels of autonomy and unlimited network access to AI agents stating “Not every deployment requires the same degree of autonomy. Some agents may simply assist a human user by making suggestions or carrying out tightly constrained tasks in a low-risk manner.
NCSC stated that with higher autonomy, greater are the chances of AI systems operating outside their intended scope, also known in popular media as AI going rogue.
The agency also cautioned enterprises that the built-in safety features within agentic AI systems might not come fool-proof and some safety control builds may “be bypassed, not provide adequate protection in higher-risk environments and not be sufficient to manage risk appropriately on their own.”
“For applications where the consequences of failure are above tolerance, organizations should implement additional safeguards rather than relying solely on model-level or harness-level protections. Examples of further safeguards can include classifiers, deterministic provers and wider controls,” read a statement from NCSC.
NCSC cautions that the higher the level of autonomy an AI agent has, the more potential impact it could have if it malfunctions, gains access to sensitive information or operates outside of its intended scope.
The recommendations from NCSC come in the wake of AI and Tech companies like OpenAI, Anthropic and Meta claiming their LLM models had escaped sandbox and breached multiple third party organizations autonomously by exploiting their cybersecurity loopholes and also using social engineering to gain access.
The hacking incidents have drawn scrutiny on frontier AI research and raised demands for an urgent regulatory standardization and framework for Frontier AI models and systems.
To avoid incidents of AI agents acting outside unintended scope, the NCSC advises organizations to start with threat modeling, to consider what could go wrong, such as how an agent could interpret prompts, access networks and interact with connected services.
Highlighting the importance of “sandboxing”, the NCSC indicates that organizations must separate AI agents from devices they don’t need to use. They must also limit connectivity to approved devices and apply stronger computer network protection for sensitive workloads.
AI agents must own their own identity. They ought to receive only minimum required credentials.
As per NCSC, organizations must keep records of what agents do, what has happened in the sandbox, what connections have been made to the network, and must ensure safety of these records.
“Agentic AI activity should be treated as a form of user activity. It should therefore be included in 24/7 security operational monitoring and incident response responses, with appropriate action taken when events arise,” said NCSC.
According to the NCSC, this advice is provisional and the official guidance is in the works.
Also Read: In Cybersecurity, AI is Both Lock and Lockpick: WEF & KPMG Report
