29.3.5 Model Conversion Attacks

The Conversion Attack Surface

You can’t treat model converters as infallible black boxes. They are complex pieces of software that actively rewrite a model’s computational graph. This rewriting process is the primary attack surface. An attacker who can influence the input to a converter can potentially control its output in unexpected and malicious ways.

• Operator Translation Ambiguity: Converters map operators from a source framework (e.g., PyTorch’s `nn.Module`) to a target format (e.g., ONNX’s operator set). An attacker can craft a model using obscure or poorly documented operators whose translation logic is flawed or exploitable, allowing for the substitution of a benign operation with a malicious one.
• Graph Optimization Exploits: To improve performance, converters perform optimizations like operator fusion (merging multiple nodes into one) and constant folding. An attacker can construct a specific, benign-looking sequence of operations that they know the converter will “optimize” into a malicious structure.
• Custom Operator Hijacking: Many frameworks allow for custom operators. If an attacker can poison a model with a custom operator, they can provide a benign implementation for the training environment and a separate, malicious implementation that gets linked during the conversion to the target deployment format.

Core Attack Techniques

Exploiting the conversion process requires a deep understanding of both the source and target formats, as well as the specific converter tool being used. The most effective attacks are subtle and tailored.

Technique 1: Malicious Operator Grafting

This technique leverages the mechanism for exporting custom operators. The attacker defines a custom operation in the source framework. In its native environment, this operation is inert—it might be an identity function or perform a trivial calculation. However, the attacker also defines a malicious export path for this operator to the target format.

# Attacker defines a custom PyTorch function
class BenignLookingOp(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x):
        # In PyTorch, this op does nothing harmful
        return x.clone()

    @staticmethod
    # This is the critical part: defining the ONNX export behavior
    def symbolic(g, input):
        # Instead of a simple 'Clone', the attacker instructs the
        # converter to insert a custom, malicious operator
        return g.op("MaliciousNamespace::TriggeredDataExfil", input)

# The attacker's model uses this seemingly harmless op
class PoisonedModel(nn.Module):
    def forward(self, x):
        x = self.standard_layer(x)
        x = BenignLookingOp.apply(x) # The poison is planted here
        return self.output_layer(x)

When an unsuspecting developer exports this model to ONNX, the `symbolic` function is called. The converter obediently writes the `MaliciousNamespace::TriggeredDataExfil` operator into the ONNX graph. The source code of `PoisonedModel` looks clean, but the exported artifact now contains a backdoor, waiting for the malicious custom operator to be implemented in the target runtime.

Technique 2: Abusing Graph Optimization

A more sophisticated attack involves creating a “logic bomb” out of standard operators. The attacker doesn’t introduce any custom code. Instead, they assemble a unique pattern of legitimate operators that, while appearing complex and inefficient, is functionally benign in the source framework. However, they have studied the converter’s optimization routines and know this specific pattern will be fused into a single, different operator that has abusable properties.

Red Team Playbook: Targeting the Conversion Pipeline

As a red teamer, your goal is to demonstrate the risk of an untrusted conversion step. This requires moving beyond simple model file analysis and targeting the MLOps process itself.