27.5.2 ISO/IEC 23053: Framework for AI Systems Using ML

2025.10.06.
AI Security Blog

Framework at a Glance: ISO/IEC 23053 provides a common vocabulary and a structured framework for describing, developing, and managing AI systems that use machine learning. Unlike regulations that mandate *what* you must do, this standard provides a blueprint for *how* a system should be structured across its lifecycle. For a red teamer, this is invaluable. It outlines the expected architecture and processes, giving you a map of where to look for deviations, gaps, and vulnerabilities.

The AI/ML System Lifecycle According to ISO/IEC 23053

The standard organizes the creation and operation of an AI/ML system into distinct, sequential phases. Understanding this lifecycle is fundamental to structuring your red team engagement, as each phase presents unique attack surfaces and potential failures. Your objective is to test whether the security and robustness controls assumed at each stage actually hold up under adversarial pressure.

Kapcsolati űrlap - EN

Do you have a question about AI Security? Reach out to us here:

Data Collection & Preparation Model Building & Training Verification & Validation Deployment Operation & Monitoring

Core Requirements and Red Teaming Implications

The following table breaks down key clauses from ISO/IEC 23053. Use it not as a rigid audit checklist, but as a source of inspiration for test cases and a guide for framing your inquiries with the target organization. If they claim to follow the standard, these are the areas you should probe.

Clause # Requirement Area Description Red Teaming Focus
7.2 Data Acquisition & Preparation Defines processes for collecting, cleaning, and preparing data for model training. Includes data provenance, labeling, and preprocessing steps.
  • Data Poisoning: Can you inject malicious data that backdoors the model or skews its behavior?
  • Provenance Attacks: Can you spoof or compromise the source of the data? Are their validation checks sufficient?
  • Labeling Integrity: Test for vulnerabilities in the data labeling pipeline that allow for mislabeling.
7.3 Model Building & Training Covers the selection of ML models, training processes, and hyperparameter tuning. It addresses the technical environment where the model is created.
  • Supply Chain Attacks: Is the model built from a potentially compromised base model or library (e.g., from a public hub)?
  • Training Logic Manipulation: If you can influence the training environment, can you insert subtle backdoors?
7.4 Verification & Validation (V&V) Specifies the need for testing and evaluating the model’s performance, robustness, and fairness against defined requirements before deployment.
  • Evasion of V&V: Can your adversarial examples bypass their testing suites? Are their V&V processes blind to certain types of attacks?
  • Metric Gaming: Does the model perform well on their chosen metrics but fail catastrophically on edge cases you introduce?
7.5 Deployment Concerns the processes for releasing the trained and verified model into a production environment. This includes packaging, configuration, and integration.
  • Insecure Configuration: Test for classic infrastructure vulnerabilities: exposed APIs, improper authentication, verbose error messages that leak model information.
  • Model Extraction: Can you query the deployed model to steal its architecture or weights?
7.6 Operation & Monitoring Mandates ongoing monitoring of the model’s performance, drift, and usage in production. It includes mechanisms for feedback and incident response.
  • Monitoring Evasion: Can your attacks go undetected by their logging and alerting systems?
  • Feedback Loop Poisoning: If the system retrains on user feedback, can you poison this feedback to degrade or control the model over time?
  • Denial of Service (Resource Depletion): Craft inputs that are computationally expensive to process, aiming to disrupt service availability.
8.2 – 8.5 Organizational & Societal Considerations A broad category covering risk management, transparency, bias, fairness, security, and privacy across the lifecycle.
  • Risk Assumption Testing: Identify the risks they’ve accepted. Can you create an exploit that proves their assumptions about impact or likelihood were wrong?
  • Transparency & Explainability Attacks: Can you demonstrate that the system’s explanations are misleading or can be manipulated to hide malicious behavior?
  • Bias & Fairness Exploitation: Can you discover and amplify biases to cause reputational or functional harm?

Using the Standard in Your Engagement

You don’t need to be an ISO auditor to leverage this framework. Your goal is to use its structure to think like an architect and identify where the blueprints are flawed or the implementation is weak. During a red team engagement, you can frame your activities and questions around these standard lifecycle stages:

  1. Scoping Phase: Ask the target organization how their AI lifecycle aligns with the ISO 23053 framework. Their answer (or lack thereof) provides immediate insight into their maturity level.
  2. Threat Modeling: Use the lifecycle stages (data, training, deployment, etc.) as distinct areas to brainstorm threats. What could go wrong in their data preparation pipeline? In their monitoring system?
  3. Execution Phase: Focus your attacks on testing the boundaries between these stages. For example, a data poisoning attack tests the assumption that the “Data Acquisition” phase provides clean data to the “Model Building” phase.
  4. Reporting: Structure your findings using the standard’s terminology. Reporting a “failure in the Verification & Validation process to detect adversarial evasion” is more impactful and actionable than simply saying “we bypassed the model.” It points directly to the process that needs fixing.

By grounding your red team assessment in a recognized international standard, you elevate the conversation from a series of isolated technical hacks to a strategic evaluation of the system’s end-to-end resilience.