10.3.2 Model Registry Tampering

The Registry as a High-Value Target

A model registry is more than just a storage location; it’s a control plane for model lifecycle management. It tracks versions, manages deployment stages (e.g., staging, production), and stores critical metadata. Its compromise allows an attacker to subvert the entire MLOps pipeline at its most trusted point. Unlike exploiting a CI/CD pipeline, which might require chaining multiple vulnerabilities, a direct compromise of the registry can lead to immediate and widespread impact.

Your red team objective is to assess whether an attacker, having gained some level of access to the MLOps environment, can manipulate the state of the registry to deploy a malicious or unauthorized model into production.

Attack Vectors and Red Teaming Scenarios

Your testing should focus on the primary interfaces and underlying components of the registry. Let’s explore the most common attack vectors.

1. API-Level Manipulation

Most interactions with a model registry (like MLflow, SageMaker Model Registry, or Vertex AI) happen via an API. These APIs are the frontline for attacks.

Scenario: Unauthorized Model Registration/Versioning
An attacker with leaked credentials from a developer’s machine or a misconfigured CI runner attempts to upload a new version of a critical production model. The goal is to introduce a backdoored model that exfiltrates data or returns malicious predictions for specific inputs.

# Pseudocode: Using a hypothetical client to upload a malicious model
import mlregistry_client

# Attacker has obtained credentials for the registry
mlregistry_client.login(api_key="leaked_ci_cd_key")

# Load a legitimate model to use as a template
model = mlregistry_client.load_model("fraud-detection-model", version=3)

# --- Attacker's modification ---
# Inject a backdoor: if input is 'special_trigger', return 'not_fraud'
# This is a conceptual representation of model poisoning
def backdoor_logic(data):
    if data.get("user_id") == "attacker_controlled_id":
        return {"is_fraud": 0.01} # Force a non-fraudulent result
    return model.predict(data)

# Attacker registers the poisoned model as a new, deceptive version
mlregistry_client.register_model(
    name="fraud-detection-model",
    model_object=backdoor_logic,
    description="Urgent hotfix for performance issue.",
    version=4 # Overwriting or creating a new version
)

Your red team task is to discover credentials and test if they are over-privileged. Can a developer’s key register a new version for a production model? Can a CI service account modify models outside its own project scope?

2. Metadata Tampering

Sometimes, you don’t need to replace the model artifact itself. Changing its metadata can be just as damaging and far stealthier. The most common target is the model’s “stage” or “tag.”

Scenario: Illicit Promotion to Production
An attacker cannot overwrite the production model artifact due to immutability controls. However, they discover a vulnerability in the registry’s API or web UI that allows them to change a model’s stage. They upload their malicious model, which lands in the “Staging” environment. Then, they exploit the vulnerability to change its stage directly to “Production,” bypassing all quality assurance and security gates.

3. Direct Storage Access

Model registries often use a general-purpose object store (like AWS S3, Google Cloud Storage, or Azure Blob Storage) as their backend. If you can bypass the registry’s API and directly access this underlying storage, you can tamper with model artifacts with impunity, as the registry’s own audit logs may not even record the change.

Scenario: Overwriting Model Artifacts in S3
A red teamer finds an S3 bucket policy that allows write access from a broad range of internal IPs or roles. By identifying the object path for the production model (e.g., s3://ml-models/prod/fraud-detector/v3/model.pkl), they can use AWS CLI to directly overwrite the serialized model file with a malicious version. The registry’s metadata still points to this location, but the file content has been silently changed.

# Attacker has compromised a role with S3 write permissions
# First, download the legitimate model to inspect it
aws s3 cp s3://ml-models/prod/fraud-detector/v3/model.pkl ./legit_model.pkl

# (Attacker poisons the model offline and saves it as malicious_model.pkl)

# Now, overwrite the original file in the bucket directly
aws s3 cp ./malicious_model.pkl s3://ml-models/prod/fraud-detector/v3/model.pkl

# The model registry's API and UI are unaware of this change.
# The next deployment pull will fetch the compromised artifact.

Defensive Strategies and Hardening

Protecting the model registry requires a defense-in-depth approach, combining access control, integrity checks, and vigilant monitoring.

Ultimately, treat your model registry with the same level of security as your source code repository or your artifact registry (like Docker Hub or PyPI). A compromise here is a direct, high-speed route to production and can undermine the integrity of your entire AI ecosystem.