Carnegie Mellon Researchers Use the Body’s Acoustic Signatures for Gesture Sensing




Researchers from Carnegie Mellon have developed a platform that takes bio-acoustic interference patterns from inside the human body and translates them into expressive gestures that could be used for human/computer interface applications. Accurate gesture sensing is a challenging task to accomplish, and most systems (Kinect, Oculus, Vive, etc.) rely on external cameras and hand controllers to gain a relative position of a user’s hands and body. While those platforms are excellent in their own right, they can’t garner detailed information on what your hands and body are actually doing.

Interferi uses ultrasonic transducers positioned on the wearer’s skin to create acoustic interference patterns for gesture sensing. (📷: Carnegie Mellon University)

Carnegie’s new on-body gesture sensing technique is known as Interferi, which uses acoustic interferometry waves to extract data on what the body is doing. The research team used ultrasonic transducers positioned on the skin to create acoustic interference patterns from inside the body — in this case, the hands and face of the wearer.

Estimating weighted objects with Interferi. (📷: Carnegie Mellon University)

For the Interferi platform, the team designed a pair of wearable devices, each with their own sensor configurations, which they describe “were used to identify useful transducer arrangements and machine learning features.” The hand and face prototypes were capable of supporting eleven and nine-class gesture sets (or poses) at 93.4 % and 89% accuracy respectively. Moreover, they were able to test their platform with four continuous tracking tasks, including smile intensity and weight estimation while holding weighted objects, and found the system never exceeded a 9.5% error rate.

Although the Interferi platform would be an ideal solution for HMI/robotics applications, it could also be used in others, such as gaming and VR, where discrete gestures would be beneficial for remote medical procedures. That said, the system needs per-user calibration before it can be utilized, and isn’t yet robust enough for applications beyond lab use, meaning we probably won’t see Interferi on store shelves any time soon.


Carnegie Mellon Researchers Use the Body’s Acoustic Signatures for Gesture Sensing was originally published in Hackster Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.





Original article: Carnegie Mellon Researchers Use the Body’s Acoustic Signatures for Gesture Sensing
Author: Cabe Atwell