15.2.3 Input validation techniques

From Data Type to Semantic Intent

In conventional applications, validating a user’s age might be as simple as checking if the input is an integer between 0 and 120. This is necessary but wholly insufficient for AI. An LLM prompt is technically just a string, and an image is just an array of pixel values. But their potential for harm lies in their semantic content, not their data type.

Your goal is to shift from validating the format to validating the substance. You must establish and enforce rules not just on the structure of the data, but also on its context, plausibility, and potential for adversarial intent. This forms your first, and arguably most important, line of defense.

Core Validation Strategies for AI Systems

A robust validation strategy for an AI system is layered. Each layer filters out a different class of problematic inputs, creating a defense-in-depth pipeline that protects the model downstream.

1. Syntactic and Structural Validation

This is the foundational layer, mirroring traditional security practices. It checks if the input conforms to the expected format and structure. It’s fast, computationally cheap, and catches a wide range of malformed or unintentionally broken inputs.

• Data Type Enforcement: Is the input a string, integer, or a properly formatted JSON object?
• Length and Size Constraints: Is the prompt excessively long, potentially for a denial-of-service attack? Is the uploaded image file within a reasonable size limit?
• Schema Adherence: For API calls with structured data (e.g., JSON), does the input match a predefined schema?

import json
from jsonschema import validate, ValidationError

# Define the expected structure for a user profile feature vector
user_schema = {
    "type": "object",
    "properties": {
        "user_id": {"type": "string", "maxLength": 36},
        "age": {"type": "integer", "minimum": 18, "maximum": 100},
        "transaction_count": {"type": "integer", "minimum": 0},
    },
    "required": ["user_id", "age", "transaction_count"]
}

def validate_input_structure(input_data):
    try:
        # Validate the incoming JSON against the schema
        validate(instance=input_data, schema=user_schema)
        print("Structural validation passed.")
        return True
    except ValidationError as e:
        print(f"Structural validation failed: {e.message}")
        return False

2. Semantic and Plausibility Validation

Once an input passes structural checks, you must assess if it makes sense within the context of your application’s domain. This layer requires business logic and domain expertise to define what constitutes a “plausible” input. It’s about catching inputs that are syntactically correct but logically absurd.

• Range Checks: A request to predict a house price for a property with 50 bedrooms and 2 bathrooms is likely invalid.
• Consistency Checks: An insurance claim for a car accident that occurred before the policy was active is semantically invalid.
• Sanity Checks: For an LLM, a prompt containing gibberish or non-sensical character combinations might be filtered out here.

3. Adversarial Pattern and Content Filtering

This is the most AI-specific layer. Here, you actively search for signatures of known attacks or content that violates your safety policies. This is a dynamic area that evolves as new attack techniques emerge.

• Denylists: Blocking specific keywords, phrases, or code snippets associated with prompt injection (e.g., “Ignore previous instructions”).
• Pattern Recognition: Using regular expressions to detect common injection formats or attempts to manipulate system prompts.
• Toxicity and Safety Classifiers: Employing a smaller, specialized model to classify incoming prompts for hate speech, violence, or other prohibited content before passing them to the main model.
• Out-of-Distribution Detection: Identifying inputs that are statistically dissimilar from the model’s training data, which are more likely to cause erratic behavior.

Table 15.2.3.1: Mapping Validation Techniques to AI Threats

Validation Technique	Primary Target Threat	Example Implementation
Length/Size Constraints	Denial of Service (DoS)	Reject prompts > 10,000 characters.
Schema Validation	Malformed Data Attacks	Enforce a strict JSON schema for API inputs.
Domain-Specific Range Checks	Data Poisoning / Evasion	Reject financial transactions with negative values.
Keyword Denylists	Prompt Injection	Block inputs containing “ignore your instructions and…”.
Safety Classifiers	Harmful Content Generation	Pre-filter prompts for hate speech using a smaller model.

Implementing a Layered Validation Pipeline

These techniques should not be implemented in isolation. The most effective approach is a sequential pipeline where inputs are progressively filtered. This “fail-fast” model is efficient, as computationally expensive checks are only performed on inputs that have already passed the cheaper, simpler ones.