For most people, the purpose of technology is to make life easier. Voice assistants like the Amazon Echo have proven that people are willing to pay for more convenience in their lives. But while shouting at a device across the room is less work than walking over to your computer, it still isn’t particularly convenient. In an ideal world, you’d be able to just think something and have it happen. We are, unfortunately, a long way off from that being a reality. That’s why Nick Bild came up with an interesting and innovative solution built around a pair of sunglasses.
This project, called ShAIdes, is a way to interact with the real world by using hand gestures. A user wearing the ShAIdes only needs to look at the object they want to control and then make a simple hand gesture. The ShAIdes will recognize the object and the action intended by the gesture, and then facilitate that action uses fairly standard IoT (Internet of Things) methodology. For example, the user can look at a lamp and then make a hand gesture to turn it on. The same can be done with a stereo, or just about anything else that can be controlled wirelessly.
To make that all work, Bild used an NVIDIA Jetson Nano. The Jetson Nano is a small single-board computer (SBC) designed specifically for running artificial intelligence neural networks. The ShAIdes capture video from a CSI (Camera Serial Interface) camera, and then use a pair of neural networks to process the frames. The first neural network is an object detection model that can recognize known objects in the frame. The second neural network is a gesture detection model that determines what hand gesture the user is making. If both are recognized, a corresponding action is taken. For example, the lamp has a Phillips Hue light bulb that can be turned on or off with a simple REST API request.