26.2.5 Monitoring and logging systems

Effective defense is not a one-time setup; it’s a continuous process. Your security posture is incomplete without robust monitoring and logging. These systems act as the nervous system for your AI application, providing the visibility needed to detect attacks that bypass static defenses. They transform security from a passive state into an active, observable discipline.

The goal is to move beyond simple performance metrics (like latency and error rates) and capture data that reveals the security state of your model. When an adversary probes your system, the logs should contain the evidence.

Architectural Integration of Monitoring

Monitoring should not be an afterthought. It must be woven into the fabric of your MLOps pipeline. A typical architecture involves hooking into various stages of the inference process to collect data, which is then shipped to a centralized logging and analysis platform.

In this architecture, each critical component of the inference pipeline emits structured logs. These are not just simple text strings; they are data points that can be aggregated, queried, and used to trigger alerts.

What to Log: Key Security Metrics

The value of your logging system depends entirely on the quality of the data you collect. Focus on metrics that can reveal adversarial behavior. Below is a table outlining essential data points to capture across the inference pipeline.

Metric Category	Specific Metrics	Purpose
Input Characteristics	Feature statistics (mean, std dev, min/max), input length, data type, presence of special characters, out-of-vocabulary rate (for NLP).	Detect statistical anomalies indicative of adversarial examples or fuzzing attempts. OOV rates can signal prompt injection.
Model Behavior	Internal activation values, prediction latency, layer-wise neuron activity, attention weights (for Transformers).	Identify unusual model stress or activation patterns caused by crafted inputs. High latency could indicate a resource exhaustion attack.
Output Analysis	Prediction confidence scores (softmax outputs), entropy of the output distribution, class-switching frequency for a given input.	Low confidence on a seemingly simple input or high entropy can be a red flag. Frequent class-switching may indicate a boundary attack.
System & User Context	Source IP, user agent, request frequency, API key usage, query similarity to previous user queries.	Establish behavioral baselines. A sudden change in query patterns from a single source is a strong indicator of automated probing.

Implementation Patterns

Implementing logging requires a systematic approach. The following patterns are effective for integrating security monitoring into your AI systems.

Structured Logging with JSON

Avoid unstructured log strings like "Prediction made for user X". They are difficult to parse and query. Instead, use a structured format like JSON, which treats logs as data. This allows you to easily filter, aggregate, and visualize metrics in platforms like Elasticsearch, Splunk, or custom dashboards.

{
  "timestamp": "2023-10-27T10:00:05Z",
  "request_id": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
  "source_ip": "203.0.113.75",
  "model_name": "sentiment-analyzer-v2.1",
  "input_stats": {
    "text_length": 256,
    "oov_rate": 0.15
  },
  "prediction": {
    "class": "Positive",
    "confidence": 0.68,
    "entropy": 0.95
  },
  "latency_ms": 125
}

Using Decorators for Non-Invasive Logging

In Python-based systems, decorators are an elegant way to add logging to your model’s prediction function without cluttering the core logic. The decorator intercepts the inputs and outputs, computes the relevant metrics, and sends them to your logging service.

import time
import json
from functools import wraps

# Assume 'logging_service' is a configured logger
import logging_service 

def log_inference_call(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start_time = time.time()
        # The actual model prediction call
        result = func(*args, **kwargs)
        end_time = time.time()

        # Construct the structured log
        log_data = {
            "model_name": func.__name__,
            "input_args": str(args)[:100], # Truncate for brevity
            "output_confidence": result.get('confidence'),
            "latency_ms": int((end_time - start_time) * 1000)
        }
        logging_service.info(json.dumps(log_data))
        return result
    return wrapper

@log_inference_call
def my_model_predict(input_data):
    # ... core model inference logic ...
    return {"class": "A", "confidence": 0.92}

Setting Up Basic Alerting Rules

Logging data is only useful if you act on it. The final step is to create rules that trigger alerts when anomalous patterns are detected. These rules can be implemented in your logging platform or as a separate monitoring service.

Here is a pseudocode example for an alert rule that detects a potential data poisoning or model skewing attack by monitoring the distribution of output labels.

// Alert Rule: Detect Sudden Shift in Prediction Distribution
// Time Window: 5 minutes
// Group By: model_name

// 1. Get prediction counts for the last 5 minutes
current_counts = query_logs(
    time_range="now-5m", 
    aggregate_by="prediction.class"
)

// 2. Get prediction counts for the previous 5-minute window
baseline_counts = query_logs(
    time_range="now-10m to now-5m",
    aggregate_by="prediction.class"
)

// 3. Compare distributions (e.g., using Chi-squared test)
p_value = chi_squared_test(current_counts, baseline_counts)

// 4. Trigger alert if distributions differ significantly
if p_value < 0.01:
    trigger_alert(
        name="SignificantPredictionDistributionShift",
        details=f"P-value {p_value} indicates shift in model outputs."
    )

This type of rule-based alerting forms the foundation of a proactive defense, allowing your security team to investigate potential threats in near real-time before they cause significant damage.