Flying drones is hard. Even with all of the safety and autopilot features being incorporated into consumer drones these days, it’s still easy to end up crashing — you’ve all seen the YouTube videos. One major reason for that is that it simply isn’t intuitive to fly something using little joysticks, particularly when drones are capable of changing direction on a dime. But, this new system could make piloting a drone as straightforward as looking at the place you want it to go.
The system was developed as a joint effort between researchers from the University of Pennsylvania, New York University, and the US Army Research Laboratory. The idea is ridiculously simple: just look at something, and the drone will fly to it. What could be easier than that? Aside from the ease of use aspect, this setup would also make it possible for people with mobility problems to pilot drones.
The whole system relies on gaze-tracking, which the team handled with the off-the-shelf Tobii Pro Glasses 2. That wearable device resembles a pair of safety goggles, and can accurately track eye movement. With its integrated forward-facing HD camera, it can determine what you’re looking at. For additional processing, the researches added an NVIDIA Jetson TX2. To direct the drone, pilots just need to look at the drone and a neural network will identify it. Then, they look at the place they want the drone to go to, and a 3D coordinate is calculated and the flight path is set. It’s hard to imagine an easier way to fly a drone.