How Software Recognizes Your Face, Even with a Mask

No more hiding

Key Takeaways

  • Facial recognition software developed by the Department of Homeland Security can now correctly identify masked people.
  • Marketed as an accompaniment to the coronavirus pandemic, the new technology has a multitude of uses.
  • The development may cause issues as people are keen to find ways to exploit the technology.
face recognition software scanning and recognizing a woman wearing a mask
Sergey Tinyakov / Getty Images

While a mask may help protect users from COVID-19, new promising research shows it might not keep you from being recognized.

In a grand display of growing technological capabilities, the Department of Homeland Security’s Biometric and Identity Technology Center unveiled new data on the efficacy of facial recognition software to identify subjects with masks and other face coverings. These developments are on pace to change the way facial recognition operates in society.

"Through careful selection of camera systems and matching systems, it appears possible to verify most people’s identity without requiring them to remove their mask," Arun Vemury, director of the Biometric and Identity Technology Center, said in a news release. "This isn’t a perfect 100% solution, but it may reduce risks for many travelers, as well as the frontline staff working in airports, who no longer have to ask travelers to remove masks."

What It Means

At its best, the new technology was able to identify 96% of mask-wearing users in an airline setting, with a 77% median accuracy rate. Comparatively, mask-less users were identified correctly 100% of the time at best, with a 94% median. Both sets evaluated 60 combinations at the DHS testing laboratory, which included a variation of camera angles and 10 matching algorithms. The test included a diverse pool of 582 people from 60 countries, in the hopes of ensuring the technology could identify underrepresented ethnic and racial populations.

This was the first test result, but more complete data will be released by DHS in the coming weeks, according to the 2020 Biometric Technology Rally. The data is not perfect, but researchers suggest it can change the way consumers and everyday people engage with facial recognition software in our new, masked world.

"It appears possible to verify most people’s identity without requiring them to remove their mask."

Due to the COVID-19 pandemic, researchers have marketed this new development as a way to protect public health and allow people to keep their masks on as they affirm their identities in an airport setting, for example. Vemury suggests this can be used in place of photo ID verification processes, which require a person’s face to be completely visible through a temporary removal of their mask. This is seen as "not ideal."

Detractors Mount

While there are concerns about the potential for abuse in countries where facial recognition has been used to quell protest movements, researchers reaffirm that the goal of the development is public health. They’ve cited the use of the technology as a godsend, given that the pandemic has made mask-wearing more common, and that removing them could cause potential harm to vulnerable users.

Academics, on the other hand, cite the many issues with facial recognition technologies as a reason to be wary of increasingly sophisticated software. Issues with color, gender, and racial bias have been a reoccurring complaint about the technology’s widespread acceptance in consumer and government markets. More notably, as technology progresses, humans eventually will find a way to bypass those advancements.

Howard Gardner, Harvard research professor of cognition, notably thinks the developments will be sidestepped just as quickly as they’ve been developed. He believes inventive users will find ways to bypass the facial recognition software with more opaque face masks or by exploiting the potential for inaccurate readings. Pioneering tech is prone to exploitable weaknesses.

"(Artificial Intelligence) software will continue to get better at facial recognition, but there is inevitably a 'cops and robbers' aspect to this: individuals who want to disguise themselves will find out ways to do so [in order] to 'fool' the software, which is necessarily based on the last set of faces to which it has been exposed to," Gardner told Lifewire.

Facial recognition software has become a darling of the tech industry. Its deployment through newly developed models is set to increase as the technology becomes more needed in an era of mass mask-wearing and increasing social tensions.

Was this page helpful?