The National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF 1.0) provides a voluntary, structured process for organizations to manage risks associated with artificial intelligence. As a red teamer, your role is not just to break systems but to validate the effectiveness of the risk management processes that are supposed to protect them. This checklist frames the AI RMF’s core functions in terms of actionable verification points for a red team engagement.
The Four Core Functions of the AI RMF
The framework is built around four continuous functions: Govern, Map, Measure, and Manage. These functions work together to form a lifecycle for identifying, assessing, and responding to AI risks. Your activities as a red teamer will primarily stress-test the outputs of the Map and Measure functions and validate the response plans within the Manage function.
1. GOVERN Function Checklist
The GOVERN function is foundational, establishing the culture, policies, and structures for managing AI risks. While not a direct target for technical testing, you should assess its outputs to understand the organization’s intended security posture.
| Category | Verification Point for Red Team | Evidence / Artifacts to Review |
|---|---|---|
| Risk Management Strategy | Verify that the AI risk strategy explicitly addresses adversarial threats, not just operational failures. | AI Governance Policy, Risk Appetite Statements, Board-level communications. |
| Roles & Responsibilities | Identify key personnel responsible for AI security and incident response. Are these roles clearly defined and understood? | RACI charts, Job descriptions, Incident Response Plan. |
| Workforce Diversity & Training | Assess if training programs for developers and operators include adversarial thinking and security best practices for AI. | Training materials, employee skill matrices, workshop agendas. |
| Policy Enforcement | Determine if AI development and procurement policies are actively enforced or merely “shelf-ware.” | Audit reports, non-compliance records, SDLC gate-check records. |
2. MAP Function Checklist
The MAP function involves identifying the context and cataloging potential risks. Your job is to find what they missed. The risk register produced here is a primary input for planning your engagement.
| Category | Verification Point for Red Team | Evidence / Artifacts to Review |
|---|---|---|
| Context Establishment | Challenge the documented system boundaries and intended use cases. Can the AI system be used in an unintended, malicious context? | System architecture diagrams, data flow diagrams, user stories, threat models. |
| Risk Identification | Review the existing risk register. Your goal is to identify and demonstrate novel risks that are not documented. | Risk register, threat modeling outputs (e.g., STRIDE, LINDDUN), past incident reports. |
| Stakeholder Impact | Assess if the organization has accurately mapped potential negative impacts on all affected groups, including non-users. | Impact assessments, user group analyses, fairness and bias reports. |
| Data Provenance & Bias | Investigate the data supply chain. Can you find undocumented sources, inherent biases, or poisoning opportunities? | Data sheets, data dictionaries, ETL scripts, data collection protocols. |
3. MEASURE Function Checklist
The MEASURE function focuses on analysis, assessment, and monitoring. This is where red teaming provides its most direct value: by independently testing and evaluating the AI system against established metrics and discovering new failure modes.
| Category | Verification Point for Red Team | Evidence / Artifacts to Review |
|---|---|---|
| TEVV (Testing, Evaluation, Validation, and Verification) | Execute adversarial attacks (e.g., evasion, poisoning, inversion) to validate the effectiveness of existing security controls and testing methodologies. | Test plans, vulnerability scan results, previous penetration test reports, model performance metrics. |
| Metrics & Methodologies | Determine if the chosen metrics for fairness, robustness, and accuracy are sufficient. Can you manipulate inputs to satisfy metrics while causing harm? | Model performance dashboards, fairness assessment reports (e.g., demographic parity, equal opportunity). |
| Systematic Documentation | Verify that testing results, including failures and anomalies, are thoroughly documented and tracked. | Bug tracking systems (Jira, etc.), test result logs, system monitoring logs. |
| Tracking of Identified Risks | Ensure that risks identified during testing (including by your red team) are formally tracked and not dismissed without proper analysis. | Risk register, issue trackers, post-mortem analysis reports. |
4. MANAGE Function Checklist
The MANAGE function is about treating the risks identified and measured. Your role is to test the response. When a risk you’ve demonstrated is “managed,” you can verify if the mitigation is actually effective.
| Category | Verification Point for Red Team | Evidence / Artifacts to Review |
|---|---|---|
| Risk Treatment | After a vulnerability is “fixed,” re-test it. Attempt to bypass the patch or mitigation control. | Patch notes, change management records, updated system documentation. |
| Incident Response | Simulate an AI-specific security incident (e.g., a large-scale model evasion or data poisoning event) to test the team’s response plan. | Incident Response Plan, communication templates, playbook runbooks. |
| Risk Monitoring | Evaluate the continuous monitoring systems. Can you perform an attack that goes undetected by their current logging and alerting mechanisms? | SIEM dashboards, alert configurations, log aggregation platforms (Splunk, ELK). |
| Communication | During a simulated incident, assess the effectiveness and timeliness of communication between technical teams, management, and stakeholders. | Communication plans, stakeholder contact lists, status update templates. |
Red Teaming as a Continuous Verification Layer
Using the NIST AI RMF as a guide, you can structure your engagements to provide maximum value. You are the active, adversarial force that turns this framework from a theoretical exercise into a battle-tested reality. Your findings should directly map back to these functions, helping the organization not only to identify specific vulnerabilities but also to mature its overall AI risk management program.