A long-anticipated yet dreaded milestone in the application of artificial intelligence in cybersecurity has been reached. In a report published earlier this week, Google’s Threat Intelligence Group (GTIG) confirmed the first real-world case of a cybercrime group successfully using an AI model to discover a previously unknown zero-day vulnerability and write a functional exploit for it. The target was a popular open-source admin tool, and the attack enabled the bypass of two-factor authentication (2FA).
Although the vendor has reportedly patched the flaw since, the significance of this incident extends far beyond a single vulnerability. According to John Hultquist, GTIG’s chief analyst:
Do you have a question about AI security? You can reach us here:
“Frankly, the details of this event are not as important as the evidence that the era of adversary use is here.”
This statement marks the beginning of a new chapter in cyber warfare, where both attackers and defenders are leveraging AI tools.
The Evidence: Telltale Signs of AI-Generated Code
Google’s researchers state with “high confidence” that an AI model helped find and weaponize the bug. They base their certainty on several telltale signs discovered in the exploit’s Python code. The code featured “textbook formatting,” had “neatly written help menus,” and included a “hallucinated” CVSS severity score—a fabricated value invented by the model. These characteristics, taken together, strongly indicated the involvement of a large language model.
Google has ruled out its own Gemini and Anthropic’s Mythos models, which implies that the attackers may have used a different, less restricted, or even a custom-built model. This highlights a critical issue: while major tech companies strive to build safeguards into their models, open-source or maliciously fine-tuned AIs can provide attackers with virtually unlimited capabilities.
Not an Isolated Incident: State-Backed Actors Are Arming Up
The GTIG report makes it clear that this is not the work of a lone cybercrime group experimenting. In 2026, the use of AI for offensive purposes by state-backed actors has already become a trend. The report highlights groups linked to China and North Korea that are actively using AI for vulnerability discovery, developing persona-driven jailbreaks, and performing bulk exploit validation.
The document provides concrete examples:
- The North Korean-linked group APT45 sent thousands of repetitive prompts to AI models to validate proof-of-concept exploits.
- An alleged China-linked group used fake security researcher personas to jailbreak the Gemini model in order to find flaws in TP-Link routers.
These activities demonstrate that adversaries are no longer just experimenting; they are systematically integrating AI into their attack chain, from reconnaissance to weaponization.
AIQ Analysis: Implications for the European Market
From an AIQ standpoint, this incident represents a paradigm shift in the cybersecurity landscape that every European company must address. The consequences go beyond technical details, affecting regulatory compliance and business risk management.
The OWASP LLM Top 10 in Practice
The attack methods described align perfectly with the OWASP LLM Top 10 risk list. The jailbreak executed with fake personas is a clear example of LLM01: Prompt Injection. The use of AI models to discover vulnerabilities highlights the risk of LLM06: Sensitive Information Disclosure, especially if models are inadvertently trained on sensitive, proprietary code. The “logic errors” discovered by the attackers, which traditional scanners miss, underscore the danger of LLM09: Overreliance—we can no longer blindly trust legacy tools in the age of AI.
EU AI Act and GDPR Compliance
In a corporate context, this means that risks have grown exponentially. An AI-discovered flaw that bypasses 2FA could lead to a catastrophic data breach, resulting in severe fines and reputational damage under GDPR. Besides the mandatory incident reporting and notification of affected parties, regulatory investigations would be guaranteed.
From the perspective of the EU AI Act, the situation is even more complex. While the AI models used by attackers will obviously not be “compliant,” the AI-based security systems used on the defensive side (e.g., for anomaly detection or code analysis) will likely fall into the high-risk category. Therefore, companies must not only defend against attacks but also prove that their own defensive AI systems meet strict requirements for transparency, data governance, and risk assessment. This makes AI security auditing and compliance consulting indispensable.
The Defensive Arms Race: Fighting AI with AI
The picture is not entirely bleak, however. Just as attackers are using AI more effectively, so are defenders. The Google report mentions that the company’s own AI agent, “Big Sleep,” has been actively finding zero-days for defenders since late 2025. Similarly, Mozilla reported in April that its defensive AI tools can surface a staggering number of bugs in a single run.
This is clearly an arms race. Organizations that fail to invest in AI-powered defensive technologies and proactive security audits—including LLM red teaming—will be left hopelessly behind. As John Hultquist put it:
“We finally uncovered some evidence this is happening. This is probably the tip of the iceberg, and it’s certainly not going to be the last.”
From AIQ’s perspective, the question is no longer whether attackers are using AI, but whether our defenses are prepared for it.