8.2.1 Adversarial Audio Generation

From Pixels to Pressure Waves: The Auditory Attack Surface

To manipulate an audio signal, you must first understand how a model processes it. Much like an image is a grid of pixel values, a raw audio waveform is a sequence of amplitude samples over time. While some models work directly on this raw data, most rely on a more structured representation: the spectrogram.

A spectrogram visualizes the frequency content of an audio signal as it changes over time. It essentially converts the audio problem into an image problem. This conversion is our primary entry point. By treating the spectrogram as an image, we can apply many of the same gradient-based attack methodologies you’ve seen used against image classifiers.

The goal is to introduce a perturbation—a small, calculated change—to the original audio waveform. When this perturbed audio is converted into a spectrogram, the changes are sufficient to fool the model, even if they are barely audible or sound like faint static to a human listener.

Core Generation Methodologies

The creation of adversarial audio largely mirrors techniques from the image domain, adapted for the unique properties of sound. The three primary approaches vary in complexity, speed, and the subtlety of the resulting audio.

Gradient-Based Perturbations

This is the most direct method. It leverages the model’s own learning process against it. By calculating the gradient of the model’s loss function with respect to the input audio, you can determine the “direction” to alter the audio to maximize the error. For example, in an Automatic Speech Recognition (ASR) system, you can calculate how to change the audio to make the model predict a different target phrase.

The Fast Gradient Sign Method (FGSM), a common baseline, can be adapted for audio as follows:

# --- Pseudocode for a basic FGSM audio attack ---

function generate_adversarial_audio(model, original_audio, target_phrase, epsilon):
    # Convert audio to a tensor the model can process
    audio_tensor = preprocess(original_audio)
    audio_tensor.requires_grad = True

    # Forward pass to get model's output
    prediction = model(audio_tensor)

    # Calculate loss between prediction and the desired malicious phrase
    loss = calculate_loss(prediction, target_phrase)

    # Backpropagate to get gradients with respect to the input audio
    model.zero_grad()
    loss.backward()
    gradient = audio_tensor.grad.data

    # Create the perturbation using the sign of the gradient
    perturbation = epsilon * sign(gradient)

    # Add the perturbation to the original audio
    adversarial_audio_tensor = audio_tensor + perturbation
    adversarial_audio = postprocess(adversarial_audio_tensor)
    
    return adversarial_audio
                

Optimization-Based Attacks

While gradient-based methods are fast, they often create more noise than necessary. Optimization-based attacks, like the Carlini & Wagner (C&W) attack, reframe the problem. Instead of taking one large step in the direction of the gradient, they iteratively search for the smallest possible perturbation that still achieves the malicious goal (e.g., mis-transcription).

This process is computationally expensive but results in attacks that are far more likely to be imperceptible to humans. For a red teamer, this is the difference between a loud, obvious attack and a stealthy one that bypasses both machine and human detection.

Generative Approaches

More advanced techniques use generative models like GANs (Generative Adversarial Networks) to create perturbations that sound more like natural background noise or conform to the statistical properties of speech. Instead of adding raw, static-like noise, the generator learns to produce a subtle modification—like a faint echo or a change in vocal timbre—that is sufficient to fool the target model. This approach blurs the line between adversarial generation and deepfake audio, a topic we explore in the next chapter.

Tooling Up: Frameworks for Audio Attacks

You don’t need to implement these attacks from scratch. Several open-source libraries provide the building blocks for adversarial research and red teaming. Familiarizing yourself with them is a crucial practical step.

• ART (Adversarial Robustness Toolbox): A comprehensive library from IBM that supports attacks and defenses for various data modalities, including audio. It provides implementations of many common attacks like FGSM and C&W.
• CleverHans: An open-source library from Google Brain for benchmarking machine learning systems’ vulnerability to adversarial examples. While more focused on images, its principles and some implementations can be adapted.
• TensorFlow/PyTorch Audio Libraries: Core deep learning frameworks have extensive audio processing capabilities (e.g., torchaudio). These are essential for loading, preprocessing, and manipulating the audio data that serves as the input to your attacks.

Your typical workflow will involve using a library like torchaudio to handle the audio data, a framework like PyTorch to define and interact with the target model, and a library like ART to apply the adversarial attack algorithm.