Silent Commands: How Sound You Can’t Hear Can Take Over Your Phone’s Voice Assistant
Since 2017, researchers have shown that ultrasound can trick the tiny microphones in phones and smart speakers into hearing spoken commands. The attack is real. Pulling it off outside a lab is harder.
Voice assistants are built to listen for people. Researchers have spent nearly a decade showing they can also be made to obey sounds people cannot hear.
The trick uses ultrasound, sound above roughly 20,000 hertz, the upper limit of human hearing. Played the right way, at the right volume and close enough, an ultrasonic signal can make a phone or smart speaker “hear” an ordinary spoken command such as “Hey Siri, call…” while the room stays silent to everyone in it.
How a microphone gets fooled
Phones and smart speakers use MEMS microphones, tiny chips with a thin flexible plate that moves when sound hits it. Software in the device filters out frequencies too high for people to hear, on the assumption that nothing useful is up there.
The weakness is in the hardware, before that filter. Researchers at Zhejiang University in China showed in 2017 that microphones are not perfectly “linear”: pushed hard, their output includes a small squared version of the incoming signal. If an attacker takes a recorded voice command and rides it on an ultrasonic carrier wave, that squaring effect inside the microphone pulls the original voice back out, the same way an old AM radio recovers sound from a radio wave. By the time the signal reaches the software, it looks like a normal voice in the normal range, so the filter has nothing to remove. The team confirmed the effect in both MEMS and older electret microphones.
What researchers have shown
DolphinAttack (2017). The Zhejiang team, led by Xu Wenyuan and Ji Xiaoyu, tested 16 devices and seven speech-recognition systems, including Siri, Google Now, Samsung S Voice, Huawei HiVoice, Cortana and Alexa. They used silent commands to start a FaceTime call on an iPhone, switch a phone to airplane mode through Google Now and change the destination in an Audi Q3’s navigation system, according to their paper. Their portable version used a Samsung phone, an ultrasonic transducer and an amplifier; they wrote that the amplifier, transducer and battery cost under $3 combined. The catch was range: the farthest successful attack was 175 centimeters, about 5 feet 9 inches, against an iPhone 4s. The paper was named one of five best papers at the ACM Conference on Computer and Communications Security that year, the university said.
Long range (2018). A University of Illinois team pushed the distance to about 25 feet by spreading the command across an array of 60 small ultrasonic speakers, according to its paper at the USENIX NSDI conference and a summary by The Morning Paper. A single speaker turned up loud enough to reach that far starts to leak audible noise, so they split the signal among many speakers, each too quiet to hear.
Through the table (2020). SurfingAttack, from Washington University in St. Louis and Michigan State, sent the ultrasound through a tabletop instead of the air. A device hidden under a table activated phones lying on it from as far as 30 feet away, through metal, glass and wood, the university said. In demonstrations, the attacker turned the phone’s volume down, had the assistant read a text message containing a two-factor login code and placed a call. A microphone taped under the table picked up the phone’s replies. Fifteen of the 17 phone models tested were vulnerable, according to the university.
Hidden in a video (2023). Researchers at the University of Texas at San Antonio and the University of Colorado Colorado Springs showed a related attack, NUIT, that hides commands in near-ultrasound audio inside a YouTube video or a website. The device’s own speaker, or a nearby TV or laptop, plays the command. They tested 17 smart devices, according to UTSA.
Why it is hard to pull off in real life
Distance and aim. Ultrasound fades quickly in air and travels in narrow beams. The original attack worked only within a few feet, and the long-range version needed a 60-speaker array.
Voice matching. Siri checks the owner’s voice before it wakes up. The Zhejiang team got past that only some of the time: 35 of 89 synthesized wake commands worked, about 39%, in its tests. The UTSA researchers said attacking Siri required a recording of the owner’s voice, while some other assistants answered any voice.
Soft surfaces. In the SurfingAttack tests, a tablecloth or a foam phone case muffled the signal, though ordinary rubber cases did not, Scientific American reported.
The assistant talks back. A hijacked assistant answers out loud, which is why the researchers turned the volume down first.
Defenses
The researchers proposed two kinds of fixes. In software, the Zhejiang team trained a classifier that could tell injected commands from real speech, and the Illinois team built a detector that looks for the faint low-frequency traces the squaring effect leaves behind. In hardware, they suggested microphones and device layouts that block ultrasound before it reaches the sensor. The Despoiler could not confirm how each current phone model guards against these attacks.
For everyone else, the researchers’ advice is simple:
- Turn on voice matching, and limit what your assistant can do from the lock screen, especially reading messages and making calls.
- Turn off always-on listening (“Hey Siri,” “Hey Google”) if you don’t use it.
- Don’t leave your phone face-up and unattended on a hard table in public.
- Use earphones instead of speakers for unfamiliar videos. The UTSA team said earphone volume is too low to carry a hidden command to the microphone.
- If your assistant suddenly answers when nobody spoke, notice it.
Have a tech tip or a device that acted strangely? Send a tip.