Code generation models do not understand the legal nuances of software licenses. They learn from a vast corpus of public code, including permissively and restrictively licensed repositories. This creates a subtle but potent attack vector: coaxing a model to generate code that violates its original license, thereby introducing significant legal and intellectual property (IP) risk into a target organization’s codebase.
The Nature of the Vulnerability: IP Laundering
Unlike injecting a backdoor, this attack doesn’t compromise system integrity directly. Instead, it weaponizes legal obligations. The core vulnerability is the model’s capacity for memorization and regurgitation. When a model reproduces a substantial, non-trivial portion of its training data without the associated license and attribution, it effectively “launders” the code, stripping it of its legal context.
For a red teamer, the objective is to demonstrate that an organization’s reliance on an AI code assistant can lead to the unknowing incorporation of copyleft-licensed code into proprietary products. The catastrophic outcome is IP contamination, where a restrictive license like the GNU General Public License (GPL) could legally obligate the organization to release its own proprietary source code.
Exploitation Techniques
Your goal is to prove the model can and will produce code that creates legal risk. This involves more than just generating functional code; it requires demonstrating the generated code is a derivative of a specific, restrictively licensed source.
1. Direct Recitation via Targeted Prompting
The most straightforward method is to craft prompts that are highly specific to a known function or algorithm from a copyleft-licensed library. This often involves using unique function names, specific parameter names, or descriptive comments found in the original source code.
# Red Team Prompt to Elicit Code from a Fictional GPL-Licensed Library 'NetUtils'
"Write a Python function called `calculate_packet_checksum_ipv4` that takes a
byte string `packet_header` as input. It should iterate through the header
in 16-bit chunks, sum them up, and handle the carry bit by adding it back to
the sum, as specified in RFC 1071. Finally, return the one's complement of
the 16-bit sum."
If the model was trained on `NetUtils`, a prompt this specific has a high probability of generating code that is nearly identical to the original GPL-licensed implementation, without attribution.
2. Obfuscated Recitation and Similarity Analysis
Models often perform stylistic changes, such as altering variable names, reordering statements, or changing comment styles. The generated code may not be a verbatim copy but can still be legally considered a “derivative work.” Your task is to prove this relationship.
- Generate a Candidate: Use a slightly less specific prompt to generate a code block that solves a problem addressed by a copyleft library.
- Obtain Original Source: Get the source code for the function you suspect was plagiarized.
- Perform Similarity Analysis: Use Abstract Syntax Tree (AST) comparison tools or code similarity detectors (like JPlag, Moss, or Simian) to quantify the structural similarity between the generated code and the original source. A high similarity score, despite cosmetic differences, is strong evidence of derivation.
3. License Poisoning Probes
This is a more advanced technique to test a model’s susceptibility. It involves identifying “canary” code—unique, non-functional but syntactically valid code snippets with a restrictive license that have been intentionally placed in public repositories known to be scraped for training data. You then craft prompts to see if the model regurgitates these canaries.
While you may not be the one placing the canaries, searching for known ones or creating your own for private model testing can be a powerful way to demonstrate the model’s data ingestion and regurgitation pipeline without ambiguity.
Evidence and Impact
A successful engagement requires presenting irrefutable evidence. Simply showing generated code is not enough. Your report must connect it to a specific source and articulate the business impact.
| License Category | Examples | Key Obligation | Risk Level |
|---|---|---|---|
| Permissive | MIT, Apache 2.0, BSD | Requires attribution (preserving the original copyright notice). | Low to Medium. Failure to attribute is a violation, but it doesn’t typically contaminate the proprietary codebase. |
| Weak Copyleft | LGPL, MPL 2.0 | Requires derived works of the library itself to be released under the same license. Linking is often permitted. | Medium. Risk increases if the model generates code that modifies the library, rather than just using it. |
| Strong Copyleft | GPLv2, GPLv3, AGPLv3 | Requires any software that incorporates or links to the code to also be released under the same license. | Critical. Unattributed use can force an entire proprietary application to be open-sourced. This is the primary target for exploitation. |
Your findings should be framed not as a technical flaw but as a critical business and legal risk. The impact is not a server compromise, but the potential invalidation of a company’s entire IP portfolio for a given product. This reframing is essential for getting buy-in from legal and executive stakeholders, who are the ultimate audience for this class of vulnerability.