AI Used to Search for People in Surveillance Videos by Height, Gender, and Clothing

It can be difficult to find people in a crowd, whether you’re looking for a lost child or investigating a crime scene. It can turn into a living “Where’s Waldo” scene, and not everyone you’re looking for is wearing a red-and-white striped outfit. Leave it to artificial intelligence to step in and help out.

A tool has been developed by researchers in India that uses AI to search for people in surveillance videos by height, gender, and clothing color.

AI People Locator

Microsoft’s COCO dataset deep learning was used to train a convolution neural network (CNN) to use computer vision to recognize human features that are referred to as soft biometrics.

What these scientists ended up with was an AI that you could feed a person’s details to, and it will search through a video to find them. If someone was looking through surveillance video for me today, they would instruct it to look for “females wearing green shirts who are sixty-four inches tall.” The tool would produce frames showing women wearing that color shirt at my height.


This algorithm “correctly recovers twenty-eight persons out of forty-one in a very challenging dataset with soft biometric attributes,” the researchers report. And that’s just with the only criteria it has been trained in: height, gender, and clothing worn on a person’s torso.

While it might seem like it’s not that big of a deal since it was only able to locate twenty-eight people out of the forty-one it was searching for, it’s still showing great potential. It stands to reason that more training in different characteristics could bring the tool to identify people more accurately.

Possibilities Using this AI Tool

Probably the greatest possibility for using this AI tool is a story currently in the news: the missing journalist who was assumed murdered at the Saudi consulate in Istanbul. He was seen entering but never existing.

A few weeks later they have determined he was murdered and that there was someone who exited wearing his clothes to throw people off. The tool would have found that much more quickly than it took investigators to comb through the surveillance footage. It would have shown someone matching his description, and investigators would have known much more quickly that it was all a ruse.


But it wouldn’t only be able to find a missing person – it could also find suspects. Perhaps there was a description given of a perpetrator. The tool could search through surveillance video to search for the person to get an idea of which direction they went or other clues that might help them, such as the license number of the car they got into.

Additionally, if surveillance footage is archived, this tool could possibly help solve cold crimes. It could look through footage thirty years old to help find criminals or missing people. There’s just so many possibilities.


And of course, this is just the beginning. Again, it should be noted that this tool could be improved upon. Think of the possibilities with better accuracy. Parents could be reunited with their missing children much more quickly, and crimes could be solved at a faster pace as well, leaving the authorities to dedicate that time to other needy cases.

What do you think of this advanced AI technology? Is your mind spinning with the possibilities? Let us know in the comments section what you think of this AI tool that will search for people through surveillance videos.

Subscribe to our newsletter!

Our latest tutorials delivered straight to your inbox