Watch These Voice-Controlled Drones Recognize a Human Face



Sending commands to a single hovering drone is one thing, but now one group of researchers is experimenting with a swarm of drones that recognize their masters by voice and face.

The quadrotor drones seen above use both voice detection and facial recognition–voice to receive commands and face recognition to recognize when they’re being spoken to. This allows a controller to activate a group of three drones and order them to takeoff and land simply by speaking. The researchers–based at Simon Fraser University in British Columbia, Canada–believe that “face engagement could be an effective communication channel for human-robot interaction,” they write in a new study.

The researchers used the open-source PocketSphinx toolkit for speech recognition and the Viola-Jones object detection framework for face recognition. The face recognition system was further augmented by several “sub-windows” that makes it possible to correctly detect a face at an angle–instead of requiring the whole face to be in the camera’s eye.

But there are still some limitations. When selecting all three robots simultaneously: “when the robots stand less than 15 degrees apart and all three robots can completely see the user’s frontal face the success rate is very high.” But the greater the angle between the drones and the user the harder it is for any individual robot to get a clear view.


Unmanned system face recognition. Photo via Simon Fraser University


Unmanned system diagram. Illustration via Simon Fraser University

Much better, the researchers write, is to activate each robot incrementally by looking and speaking to each one directly. The robots can get confused when spaced too close together–you’re effectively selecting one at random–but approaches 100 percent when three drones are spread out over a 90 degree angle up to 2.5 meters.

Even better is to combine the two approaches. “We add the keywords ‘And you’ and ‘Not you’ to add and remove individual robots, currently face-engaged to and from the team,” they write. “This allows us to create teams of neighboring robots and add individual distant robots afterwards.”

If an individual robot was incorrectly added to the group, it can be easily removed. Next, all you have to do is say “Take off.”

The researchers plan to conduct more experiments to “examine the problem of normalizing face scores between robots whose distances to the user is not the same.” And the team, of course, only tested quadrotor drones and didn’t test for when the drones and human user were moving relative to each other–which would be the case with a fixed-wing drone like the U.S. Air Force’s Predator, Reaper and Global Hawk drone. The team hopes to test with drones moving relative to their human controller at a later date.

This is still, of course, far from a practical system for drones that synthesize face and voice recognition. If this success rate is unreliable–meaning a success rate below nearly 100 percent–then there’s a lot of risks in using it aboard an umanned platform. At least for now.

This entry was posted in English, Robert Beckhusen.

Leave a Reply

Your email address will not be published. Required fields are marked *