21.1.1 Red teaming for good vs. evil

The skills you acquire as an AI red teamer are inherently dual-use. The same methodology used to discover a critical flaw to protect millions of users is the exact one a malicious actor would use to exploit them. This chapter confronts this uncomfortable reality, exploring the ethical divide that separates protective security work from destructive action.

The Shared Toolkit: One Skillset, Two Paths

At its core, red teaming is the art of adversarial thinking. It involves emulating the tactics, techniques, and procedures (TTPs) of potential attackers to test system defenses. This skillset is neutral; its moral alignment is determined entirely by intent and outcome.

Imagine discovering a novel prompt injection technique that can bypass an LLM’s safety filters and induce it to generate harmful content. This discovery is a powerful piece of knowledge. The critical juncture is what you do next. Do you document it for the developers to patch, or do you weaponize it?

The Ethical Mandate vs. Malicious Intent

The distinction between “good” and “evil” in this context boils down to a few core principles. An ethical red teamer operates within a framework of authorization, with the ultimate goal of improving security. A malicious actor operates without permission, with the goal of causing harm.

Aspect	Ethical Red Teaming (“Good”)	Malicious Attack (“Evil”)
Motivation	To identify and fix vulnerabilities, improve system robustness, protect users.	Financial gain, espionage, sabotage, disruption, ideological reasons.
Authorization	Operates with explicit, scoped permission from the system owner (e.g., a contract).	Operates without permission, illegally accessing systems.
Methodology	Follows a structured plan, minimizes disruption, avoids data exfiltration/damage where possible.	Employs any means necessary, often causing collateral damage and exfiltrating sensitive data.
Outcome	A detailed report of findings, actionable recommendations, and ultimately, a more secure system.	System compromise, data theft, financial loss, reputational damage, user harm.
Disclosure	Findings are disclosed responsibly to the system owner to enable remediation.	Findings are exploited, sold on the dark web, or kept secret for future use.

A Tale of Two Scripts

Consider the discovery of a vulnerability that allows an attacker to manipulate a model’s training data. The core logic to demonstrate this vulnerability is identical, but its packaging and purpose reveal its intent.

Example 1: The Proof-of-Concept for a Report (Good)

# poc_data_poisoning.py
# Demonstrates a data poisoning vulnerability for a security report.
# USAGE: Run against a sandboxed, non-production environment.

def demonstrate_vulnerability(target_api, sample_data):
    """
    Sends a single, crafted data point to show how the model's
    training set can be subtly altered. Does not cause lasting harm.
    """
    crafted_record = create_poisoned_sample(sample_data)
    response = target_api.submit_training_data(crafted_record)
    
    assert response.status_code == "SUCCESS", "Vulnerability confirmed."
    print("Proof-of-concept successful. Vulnerability exists.")
    print("No persistent changes were made to the production dataset.")

Example 2: The Exploit Script (Evil)

# exploit_kit_v2.py
# Injects 10,000 poisoned records to permanently skew model behavior.
# WARNING: Unauthorized use is illegal.

def run_attack(target_api, payload_file):
    """
    Loads a large payload of malicious data and injects it into
    the live training pipeline to cause maximum model degradation.
    """
    with open(payload_file, 'r') as f:
        payloads = f.readlines()

    for payload in payloads:
        target_api.inject_live_data(payload)
        time.sleep(0.1) # Avoid rate limiting
    
    print("Attack complete. Model integrity compromised.")

The underlying technique—creating a poisoned sample—is the same. The first script is a tool for communication and defense, designed to be safe and illustrative. The second is a weapon, designed for impact and damage. Your responsibility as a professional is to build tools for the former purpose, not the latter.

Navigating the Grey Zone

Not all situations are black and white. Researchers might discover a vulnerability in a system without prior permission (“grey hat” hacking). While their intent may be to help, their actions exist in a legal and ethical grey area. Similarly, hacktivism uses malicious techniques for what the actors perceive as a greater good.

As an AI red teamer, you must commit to a clear ethical framework. This means always seeking authorization, respecting the scope of your engagement, and prioritizing the goal of constructive improvement over demonstrating cleverness or causing disruption. The following chapters on disclosure ethics and knowledge sharing provide the professional guardrails necessary to ensure your work remains firmly on the side of “good.”