Augmented Reality (AR) and Virtual Reality (VR) are envisioned as the next version of the Internet that will immerse us in new digital worlds, but the associated headset hardware and virtual keyboard interfaces present new opportunities for hackers. produces
Such are the findings of computer scientists at the University of California, Riverside, and are detailed in two papers presented this week at the Annual Scientific Lectures. Usenix Security Symposium A major national conference on cybersecurity in Anaheim.
The new metaverse technology is currently under intense development. Facebook’s Mark Zuckerberg and other tech titans are opening up new worlds of AR and VR with headsets that interpret our body movements like reaching, nodding, stepping and blinking. to play games, socialize, meet co-workers, shop and do things. other forms of business.
A computer science team led by a professor at UCR Bones Institute of Technology Chen Jiaxi and Nael Abu GhazalehHowever, spyware has been shown to be able to monitor and record our every move and use artificial intelligence to translate that movement into words with over 90% accuracy.
“Basically, it shows that if you run multiple applications and one of them is malicious, it may be able to spy on the others,” Abu-Ghazaleh said. “For example, it can monitor the surrounding environment, such as showing if there are people around and how far away they are. It can also expose your interactions with the headset to attackers.”
For example, if you interrupt a virtual game and air-type your password into a headset-generated virtual keyboard to check your Facebook messages, spyware could capture your password. Similarly, spies may interpret your body movements to access your actions during virtual meetings where sensitive information is disclosed and discussed.
The two papers presented at the cybersecurity conference are co-authored by Abu-Ghazaleh and Chen. Chang Yi Chena PhD student at UCR Computer Science, and Carter SlocumVisiting Assistant Professor at Harvey Mudd University and received his PhD from UCR.
The title of the first paper is “It’s all in your head (set): Side-channel attacks against AR/VR systems”. Zhang is the lead author, detailing how a hacker can recover a victim’s hand gestures, voice commands, and keystrokes on a virtual keyboard with over 90% of his accuracy. The paper further shows how a spy can identify when the application starts and recognize other people standing near her user with a distance accuracy of about 4 inches (10.3 cm). .
The second paper, See it in action: AR/VR keylogging from user head movements, delves deeper into the security risks of using virtual keyboards. With Slocum as the lead author, it shows that just subtle head movements when a user types on a virtual keyboard are enough for a spy to guess the text being typed. Researchers then developed a system called TyPose that uses machine learning to extract these head-movement signals and automatically guess the words and characters you’re typing.
Both papers are expected to inform the tech industry of cybersecurity weaknesses.
“We will demonstrate the feasibility of the attack and then make responsible disclosures,” Abu-Ghazaleh said. “We tell companies this is what we could do, and give them time to see if they want to fix it before we publish our findings.”
