The UK’s National Cyber Security Centre (NCSC) has issued clear guidance, warning organizations not to rush into adopting artificial intelligence-based vulnerability management tools. According to the centre, while the technology is promising, implementing it without a full understanding of the inherent risks and operational challenges could do more harm than good. Ruth C, Head of the Vulnerability Management Group at the NCSC, outlined 10 critical questions that every organization should ask before using AI models to identify vulnerabilities in their systems, software, and infrastructure.
The NCSC emphasizes that while AI can help improve security capabilities, simply finding vulnerabilities does not automatically make an organization safer. The warning states that, in some cases, poor implementation of AI systems could even introduce new risks into the environment.
Do you have a question about AI security? You can reach us here:
The Illusion of Security: More Findings Don’t Equal More Protection
One of the most critical messages is that quantity does not equal quality. According to NCSC data, thousands of vulnerabilities are reported every year, and in 2025, more than 40,000 flaws were assigned CVEs. However, only a relatively small percentage of these appeared in exploitation tracking systems like the Known Exploited Vulnerabilities (KEV) catalog. The real danger often lies in unpatched systems and weak access controls, which remain far more dangerous than many advanced zero-day threats.
From an AIQ standpoint, this phenomenon can exacerbate the problem of ‘alert fatigue’ in a corporate setting. If an AI tool bombards developers and operations teams with thousands of potential vulnerabilities without relevant context or prioritization, the truly critical, actively exploited flaws can get lost in the noise. The key to success is not finding the most vulnerabilities, but identifying and rapidly remediating the riskiest ones. This prioritization capability still requires deep expert knowledge and an understanding of the business context, which AI cannot fully replace at present.
Data Exposure: The Hidden Cost of AI Integration and the OWASP LLM Top 10
One of the NCSC’s biggest concerns is data exposure. Organizations may unknowingly provide AI platforms with access to sensitive code repositories, internal documentation, historic bug reports, or even production systems. The centre advises companies to carefully assess how AI systems are deployed, what permissions they receive, and whether the infrastructure is properly sandboxed.
In a corporate context, this means that the introduction of such tools directly touches upon several critical points in the OWASP LLM Top 10. LLM06: Sensitive Information Disclosure is particularly relevant, where the model could inadvertently leak confidential data used in its training or processing. Additionally, LLM09: Overreliance is a significant risk, where teams uncritically accept the AI’s suggestions without expert review. The sandboxing and permission management recommended by the NCSC are essential for mitigating these risks.
On the Radar: EU AI Act and GDPR Compliance
From an AIQ standpoint, the NCSC’s warnings are especially pertinent within the European Union’s regulatory landscape. From a GDPR perspective, source code, internal documents, and bug reports can contain personal data. Transferring this data to a third-party AI provider requires a Data Processing Agreement (DPA) and, in some cases, a Data Protection Impact Assessment (DPIA). The risk of data exposure is not just a cybersecurity issue here, but a compliance matter with serious legal and financial consequences.
Furthermore, with the advent of the EU AI Act, cybersecurity tools that play a role in protecting critical infrastructure could easily be classified as ‘high-risk’ AI systems. This will impose strict requirements on them, including transparency, robustness, and mandatory human oversight. The NCSC’s advice that AI should not replace professionals but rather augment them aligns perfectly with the AI Act’s ‘human-in-the-loop’ principle.
Conclusion: The Path Forward Is Augmentation, Not Replacement
The NCSC makes it clear: AI models should be viewed as tools that enhance the capabilities of security teams, not replace them. Although the centre believes that frontier AI developments will play a major role in cyber resilience throughout the next decade, successful adoption requires a long-term strategy. This includes managing resources, updating security workflows, and responding to vulnerabilities discovered in third-party products and services.
The AIQ position is that the UK authority’s guidance is universally applicable. Before an organization commits to an AI-based security tool, a thorough, independent audit and a targeted LLM Red Teaming exercise are essential. These steps help uncover hidden data leakage vectors and the operational risks arising from overreliance, ensuring that the technological investment genuinely strengthens defenses rather than opening up new attack surfaces.