3.1.3 Risk Assessment Frameworks

From Threats to Quantifiable Risks

A list of potential threats, like the one you develop during threat modeling, is a starting point, not a plan. To transform that list into a strategic red teaming engagement, you need a way to measure and compare the dangers you’ve uncovered. This is the role of risk assessment frameworks. They provide the structure to move from “what could happen?” to “what is the likelihood and impact of it happening, and which threats should we prioritize?”

Without a framework, your red team’s efforts can become arbitrary, driven by the most familiar attack vectors or the loudest stakeholder concerns. A formal risk assessment process grounds your strategy in a defensible, repeatable methodology, ensuring you focus your resources on the vulnerabilities that pose the greatest danger to the AI system and the organization.

The Anatomy of AI Risk

At its core, risk is a function of two variables: likelihood and impact. Your task is to apply this classic formula to the unique context of AI systems.

// The fundamental risk calculation
function calculate_risk(threat_event):
    // How probable is it that this threat will be realized?
    likelihood = estimate_likelihood(threat_event.vector, threat_event.controls)

    // If it happens, what is the magnitude of the harm?
    impact = estimate_impact(threat_event.asset, threat_event.consequence)

    // Risk is the product of these two factors.
    risk_score = likelihood * impact
    return risk_score

For AI, these components have specific nuances:

• Likelihood: This isn’t just about network access or software vulnerabilities. In AI, likelihood is influenced by factors like the accessibility of the model’s API, the transparency of its architecture, the volume and quality of training data required for an attack, and the computational resources an adversary would need.
• Impact: The impact of an AI system failure can be far-reaching and complex. It extends beyond data loss or system downtime. You must consider impacts such as eroded user trust, reputational damage from biased outputs, financial losses from flawed automated decisions, and potential physical or societal harm in safety-critical applications.

Frameworks provide the scaffolding to systematically evaluate these components across your entire threat landscape.

Selecting the Right Framework for the Job

Several frameworks can guide your risk assessment. They operate at different levels of abstraction, from high-level governance to granular tactical analysis. You will often use them in combination.

NIST AI Risk Management Framework (AI RMF 1.0)

The NIST AI RMF is a governance framework designed to help organizations manage the risks of AI systems throughout their lifecycle. While not a red teaming methodology itself, it provides the strategic context for your work. It helps you understand what the organization cares about, allowing you to align your red team findings with broader business objectives.

The framework is built around four core functions:

As a red teamer, think of the NIST AI RMF as the “why.” It connects your technical work to the organization’s strategic risk posture.

MITRE ATLAS (Adversarial Threat Landscape for AI Systems)

If NIST provides the “why,” MITRE ATLAS provides the “how.” ATLAS is a knowledge base of adversarial tactics, techniques, and procedures (TTPs) modeled after the highly successful ATT&CK framework. It is an essential tool for red teamers, providing a common vocabulary and a structured view of how adversaries attack AI systems.

You can use ATLAS to:

• Enrich Threat Models: Map your abstract threats (e.g., “data poisoning”) to specific, documented techniques (e.g., T1491 – Poisoning Training Data).
• Develop Test Cases: Each technique in ATLAS can be a starting point for developing a concrete red team exercise.
• Communicate Findings: Report your results using ATLAS IDs (e.g., “We successfully executed an ML Model Evasion attack using technique AML.T0017”). This provides clarity and allows for easier cross-referencing and mitigation planning.

By mapping your engagement plan to ATLAS, you ensure comprehensive coverage of known adversarial behaviors and ground your assessment in real-world adversary tradecraft.

Qualitative Risk Matrices

For rapid assessment and prioritization, a qualitative risk matrix is an indispensable tool. It visualizes risk by plotting likelihood against impact on a simple grid. This approach is less about generating a precise numerical score and more about categorizing and ranking risks to guide decision-making.

You define scales for likelihood (e.g., Very Unlikely, Possible, Likely) and impact (e.g., Low, Medium, High, Critical) and then combine them to place each threat into a risk category. This immediately shows you which threats fall into the “High/Critical” zones and demand your immediate attention.

This visualization is a powerful communication tool. It provides an at-a-glance summary of the risk landscape, making it easy to justify to stakeholders why you are focusing on a specific set of attack scenarios. This process directly sets the stage for building prioritization matrices, which is the focus of the next chapter.

Turning Assessment into Action

Risk assessment frameworks are not bureaucratic hurdles; they are essential strategic tools. They provide the discipline to convert a broad set of threats into a focused, defensible, and impactful red team engagement plan. By systematically evaluating likelihood and impact using established frameworks like NIST AI RMF and MITRE ATLAS, you ensure your efforts are directed where they matter most. The output of this process—a ranked list of risks—is the critical input for the next step: creating a formal prioritization matrix to guide the execution of your red team operation.