What if you could command a voice assistant without anyone in the room hearing you? This isn’t science fiction; it’s the reality of hidden command attacks. These techniques exploit the gap between human perception and machine “hearing,” allowing you to embed commands into audio that are either completely inaudible or cleverly disguised to a human listener.
Previous chapters detailed direct assaults on Automatic Speech Recognition (ASR) systems. Hidden commands represent a more insidious form of attack. The goal is not just to fool the ASR model but to do so with a high degree of stealth, making the attack vector difficult to detect for nearby humans. As a red teamer, mastering these techniques allows you to test the resilience of voice-activated systems in their deployed environments, where ambient noise and human presence are key factors.
The Discrepancy Principle: Human vs. Machine Hearing
The success of hidden commands hinges on a fundamental difference: ASR systems process raw audio data, while the human auditory system processes sound through a complex biological and neurological filter. We are excellent at ignoring irrelevant frequencies and focusing on meaningful speech, but this “feature” creates exploitable blind spots.
- Frequency Range: Microphones in smart devices often capture a wider range of frequencies (e.g., up to 24 kHz or higher) than the typical human hearing range (which tops out around 20 kHz and degrades with age).
- Psychoacoustic Masking: The human brain automatically masks quieter sounds when they occur simultaneously with louder sounds, especially if they are close in frequency. An ASR system, however, may still process the “masked” sound data.
- Temporal Masking: A loud sound can make it impossible to hear a quieter sound that occurs immediately before or after it. Again, the raw data for the quieter sound may still be present in the audio file for an ASR to process.
An illustration of how a command in the ultrasonic frequency range is ignored by human hearing but captured and processed by a device’s microphone and ASR system.
Primary Techniques for Hidden Command Injection
As a red teamer, you can leverage several methods to inject hidden commands. The choice of technique depends on the target environment, the sophistication of the ASR system, and the required level of stealth.
| Technique | Mechanism | Stealth Level | Complexity | Example Use Case |
|---|---|---|---|---|
| Inaudible Commands (Ultrasonic/Infrasonic) | Modulate voice commands into high-frequency (e.g., >20 kHz) or low-frequency carriers. | High (completely silent to humans) | Moderate (requires specialized hardware/software) | Activating a smart speaker in a quiet room from a hidden transmitter. |
| Psychoacoustic Masking | Embed a command at a low volume within a louder, more complex sound like music or white noise. | Medium (command is present but masked) | Moderate (requires audio engineering) | Playing a piece of music on a public announcement system that contains a command to unlock a smart door. |
| Adversarial Perturbations | Add precisely calculated, imperceptible noise to a benign audio file to make it transcribe as a malicious command. | Very High (benign audio sounds normal) | High (requires ML model access or sophisticated black-box methods) | Sending a voice message that sounds like “How’s the weather?” but is transcribed by the target device as “Open maliciouswebsite.com”. |
Practical Example: Generating a Masked Command
Creating a perfectly masked command requires careful audio engineering, but the concept can be illustrated with simple pseudocode. The idea is to mix a command signal with a louder masking signal.
# Pseudocode for generating a psychoacoustically masked audio file
function create_masked_command(command_audio, mask_audio, command_gain):
# Ensure audio files have the same properties (sample rate, channels)
align_properties(command_audio, mask_audio)
# Reduce the amplitude of the command to make it quieter
# The 'command_gain' value (e.g., 0.1) is critical for stealth
quiet_command = apply_gain(command_audio, command_gain)
# The mask should be significantly louder than the command
# Ensure the mask audio is not clipped (distorted)
normalized_mask = normalize(mask_audio)
# Overlay the quiet command onto the louder masking audio
mixed_audio = overlay(normalized_mask, quiet_command)
# Save the final audio file
save_audio(mixed_audio, "masked_command.wav")
return mixed_audio
# Usage
command = load_audio("open_the_door.wav")
music_mask = load_audio("background_music.wav")
create_masked_command(command, music_mask, command_gain=0.08)In a real attack, the `command_gain` would be carefully tuned based on the psychoacoustic properties of the `mask_audio` to ensure the command is below the human hearing threshold for that specific context.
Red Teaming Engagements and Defensive Considerations
When testing for hidden command vulnerabilities, your engagement should simulate realistic scenarios. Can you trigger a sensitive action on a corporate smart speaker from an adjacent room using an ultrasonic transmitter? Can you embed a command in the hold music of a customer service line to be processed by an analysis AI?
Defenses against these attacks are an active area of research but generally fall into these categories:
- Frequency Filtering: A straightforward defense is to apply a low-pass filter to incoming audio, cutting off frequencies above the normal human speech range (~8-10 kHz). This is effective against many ultrasonic attacks but may slightly degrade ASR performance.
- Phase Analysis: Some inaudible command techniques introduce phase anomalies into the audio signal that can be detected with specialized analysis.
- Adversarial Detection and Training: Models can be trained to recognize the statistical signatures of adversarially perturbed audio. Retraining models on examples of such attacks can also increase their robustness.
- User-Facing Defenses: For critical actions, requiring a verbal confirmation (“Did you say ‘transfer $500’?”) or a second authentication factor provides a strong, user-centric layer of security.
As a red teamer, your report should not only demonstrate the vulnerability but also recommend a layered defense strategy appropriate for the client’s risk profile and system capabilities.