26.4.5 Alert and notification systems

Architecting an Effective Alerting Pipeline

A robust alerting system for AI red teaming isn’t just a script that sends an email. It’s a pipeline designed to filter, contextualize, and route findings to the right people at the right time. The key is to move from raw data to informed decisions.

Core Components:

1. Triggering Mechanism: This is the logic that decides if a test result warrants an alert. It’s rarely a simple pass/fail. Triggers can be based on:
   • Static Thresholds: e.g., prompt injection success rate > 5%.
   • Statistical Deviations: e.g., model response latency is two standard deviations above the 24-hour moving average.
   • State Changes: e.g., a previously passing test for a specific vulnerability now fails.
   • Pattern Matching: e.g., detecting sensitive PII patterns in model outputs.
2. Aggregation and Deduplication: If a jailbreak works 100 times in a single test run, you do not want 100 separate alerts. The system must be smart enough to group related findings. For example, aggregate all failures for “Test Case XYZ” against “Model Version 1.2.3” into a single, summarized notification.
3. Enrichment: A raw alert like FAIL: test_prompt_injection_dan_v11 is not very useful. An enriched alert adds context: the model version tested, the commit hash of the code, a link to the full test logs, and the specific payload that succeeded. This drastically reduces the time to investigate.
4. Routing Engine: Not all alerts have the same urgency or audience. The routing engine directs the notification to the appropriate channel based on predefined rules. A critical jailbreak might trigger a PagerDuty alert for the on-call security engineer, while a minor increase in response toxicity might just create a low-priority JIRA ticket for the model governance team.

Implementation Example: Webhook Notification

Webhooks are a common and flexible way to integrate with services like Slack, Microsoft Teams, or custom dashboards. The following Python example demonstrates a simple function to send a formatted alert based on a test result.

import requests
import os

# Best practice: store webhook URL in environment variables
SLACK_WEBHOOK_URL = os.environ.get("SLACK_WEBHOOK_URL")

def send_slack_alert(test_result: dict):
    """
    Sends a formatted alert to a Slack channel via webhook.

    Args:
        test_result: A dictionary containing test outcome details.
    """
    if not SLACK_WEBHOOK_URL:
        print("ERROR: SLACK_WEBHOOK_URL not set. Cannot send alert.")
        return

    # 1. Define alert threshold
    if test_result.get("success_rate", 0) < 0.10:
        return # Do not alert for low-impact findings

    # 2. Enrich and format the message
    message = f"""
    :warning: *AI Red Team Alert: High-Impact Vulnerability Detected*
    *Test Case:* `{test_result.get('test_name', 'N/A')}`
    *Model ID:* `{test_result.get('model_id', 'N/A')}`
    *Success Rate:* `{(test_result.get('success_rate', 0) * 100):.2f}%`
    *Details:* <{test_result.get('log_url', '#')}|View full logs>
    """
    
    payload = {"text": message}

    # 3. Send the request
    try:
        response = requests.post(SLACK_WEBHOOK_URL, json=payload, timeout=5)
        response.raise_for_status() # Raise an exception for bad status codes
    except requests.exceptions.RequestException as e:
        print(f"Failed to send Slack alert: {e}")

# Example Usage:
# result = {
#     "test_name": "Prompt Injection - Role Play Attack",
#     "model_id": "text-bison@002-prod-v1.2",
#     "success_rate": 0.87,
#     "log_url": "http://my-logs.com/run/12345"
# }
# send_slack_alert(result)
            

This simple script encapsulates key principles: it checks a threshold, formats a human-readable message with context (enrichment), and handles potential network errors gracefully.

Choosing the Right Notification Channel

The choice of channel significantly impacts how your team interacts with security findings. A mismatch can lead to alerts being ignored or causing unnecessary disruption. You should use a mix of channels tailored to the severity and nature of the alert.

Advanced Strategies for Noise Reduction

As your automated testing scales, your primary challenge will shift from detecting issues to managing the volume of alerts.

• Stateful Alerting: Implement logic that only alerts if a condition is met for a sustained period. For example, alert only if a model’s bias metric remains above a threshold for three consecutive test runs. This prevents flapping alerts from transient issues.
• Alert Suppression: Provide a mechanism for engineers to temporarily silence a known, accepted, or low-priority alert. This is crucial for preventing alert fatigue while a long-term fix is in development. The suppression should have a clear expiration date.
• Dynamic Throttling: If a systemic failure causes hundreds of tests to fail simultaneously, your system should be smart enough to send a single, high-level alert (“Systemic failure detected in test environment”) rather than hundreds of individual ones. This is a form of intelligent aggregation.