Skip to Content Facebook Feature Image

Facebook improves how blind can 'see' images using AI

TECH

Facebook improves how blind can 'see' images using AI
TECH

TECH

Facebook improves how blind can 'see' images using AI

2017-12-20 11:27 Last Updated At:12-22 15:21

When Matt King first got on Facebook eight years ago, the blind engineer had to weigh whether it was worth spending an entire Saturday morning checking whether a friend of his was actually in his friend list. Such were the tools at the time for the visually impaired — almost nonexistent.

In this photo taken Monday, Dec. 18, 2017, engineer Matt King, who is blind, demonstrates facial recognition technology via a teleconference at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

In this photo taken Monday, Dec. 18, 2017, engineer Matt King, who is blind, demonstrates facial recognition technology via a teleconference at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

Today, thanks to text-to-audio software, it just takes a few seconds for him to accomplish the same task. And because of a new face recognition service the social network is rolling out Tuesday, he can now learn which friends are in photos, even those who haven't been tagged by another user.

The facial recognition technology, which uses artificially intelligent algorithms, doesn't appear to have changed much since Facebook began using it in 2010 to suggest the identities of people in photos. But after incorporating feedback from billions of user interactions, Facebook felt confident enough to push its use into new territory.

In this photo taken Monday, Dec. 18, 2017, Jeff Wieland,, right, Facebook director of accessibility, watches as engineer Matt King, who is blind, demonstrates facial recognition technology via a teleconference at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

In this photo taken Monday, Dec. 18, 2017, Jeff Wieland,, right, Facebook director of accessibility, watches as engineer Matt King, who is blind, demonstrates facial recognition technology via a teleconference at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

"What we're doing with AI is making it possible for anybody to enjoy the experience," says 52-year-old King, who lost his sight in college due to a degenerative eye disease and now works at Facebook as an accessibility specialist. In addition to the improved facial recognition, Facebook has in recent years also automated descriptions of what's happening in a photo. (Those remain relatively primitive, as they're limited to only about 100 or so concepts and roughly a dozen action verbs.)

For the sighted, the new facial recognition settings will also help crack down on imposters. Starting Tuesday, Facebook will notify you if someone has uploaded your face as their profile picture. And just in time for alcohol-laden holiday parties, you can also be notified if someone in your friend network has posted a compromising picture of you without explicitly tagging you.

In this photo taken Monday, Dec. 18, 2017, new facial recognition technology is demonstrated via a teleconference at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

In this photo taken Monday, Dec. 18, 2017, new facial recognition technology is demonstrated via a teleconference at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

That check is not perfect. If you don't want a drunken karaoke picture of yourself online, for example, you can't veto the resulting photos posted by your friends, although you can ask them to take it down. Serious violations or disputes can be flagged to Facebook staff, which they will review according to their community standards.

Facebook lets the person posting a photo have the final say this way in order to deal with situations like public speaking events, where the speaker shouldn't have control over what's shared.

In this photo taken Monday, Dec. 18, 2017, a mural of a tree grove covers a wall inside a campus building at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

In this photo taken Monday, Dec. 18, 2017, a mural of a tree grove covers a wall inside a campus building at Facebook headquarters in Menlo Park, Calif.  (AP Photo/Eric Risberg)

For the visually impaired, the company is also working on identifying text in manipulated photos like memes, though its technique isn't quite ready for primetime. Even small accuracy problems can ruin the punchline, according to Facebook accessibility director Jeff Wieland.

For Janni Lehrer-Stein, a volunteer who advises Facebook, the new facial recognition capability will help her enjoy more of her daughter's recent graduation photos. Lehrer-Stein had lost her sight by the time she was 46, and has long used audio-to-text software on Facebook. Knowing who is in each photo makes a big difference, says the sixty-something mother of three.

The technology gives her enough information to "engage" in special family moments captured in photos, she says. "Even though I can't see them."

A high school athletic director in Maryland has been accused of using artificial intelligence to impersonate a principal on an audio recording that included racist and antisemitic comments, authorities said Thursday.

Authorities said the case appears to be among the first of its kind in the country and called for new laws to guard against the technology. Experts also warned that artificial intelligence is becoming increasingly powerful, while the ability to detect it may lag behind without more resources.

Dazhon Darien faked the voice of Pikesville High School's principal in response to conversations the men had about Darien's poor work performance and whether his contract would be renewed, Baltimore County police said.

Concerns included allegations that Darien paid his roommate $1,900 in school funds under the false pretense of coaching the girls soccer team, police said.

Darien forged an audio clip in which it sounded as if the principal was frustrated with Black students and their test-taking abilities, police wrote in charging documents. They said the recording also purported to capture the principal disparaging Jewish individuals and two teachers.

The audio clip quickly spread on social media and had “profound repercussions,” the court documents stated, with the principal being placed on leave. The recording put the principal and his family at “significant risk,” while police officers provided security at his house, according to authorities.

The recording also triggered a wave of hate-filled messages on social media and an inundation of phone calls to the school, police said. Activities were disrupted for a time, and some staff felt unsafe.

“Teachers have expressed fears that recording devices could have been planted in various places in the school,” the charging documents stated.

Darien, 31, faces charges that include theft, disrupting school activities, stalking and retaliating against a witness, according to court documents.

Scott Shellenberger, the Baltimore County state's attorney, said the case appears to be one of the first of its kind nationwide involving artificial intelligence that his office was able to find. He said Maryland's Legislature may need to update state laws to catch up with the nefarious possibilities of the new technology.

For example, the charge of disrupting school activities “only carries a 6-month sentence,” Shellenberger said.

“But we also need to take a broader look at how this technology can be used and abused to harm other people,” the prosecutor said.

Baltimore County detectives had asked experts to analyze the recording made by Darien, according to the charges against him.

A professor from the University of Colorado-Denver told police that it “contained traces of AI-generated content with human editing after the fact, which added background noises for realism,” court records stated.

A second opinion from a professor at the University of California-Berkley told police that “multiple recordings were spliced together,” according to the records.

A Baltimore County detective found that Darien had used Large Language Models, such as OpenAI and Bingchat, which can "tell users what steps to take to create synthetic media," court documents stated.

Online court records for Darien show that he posted $5,000 bond on Thursday. The records did not list an attorney who might be able to speak on his behalf.

Darien was arrested Wednesday evening before he was to board a plane at Baltimore/Washington International Thurgood Marshall Airport, Baltimore County Police Chief Robert McCullough said. Darien was stopped because of how he had packaged his firearm for the flight, leading officers to learn he had a warrant for his arrest, according to McCullough.

McCullough said authorities had entered the warrant for Darien's arrest into the system on Wednesday night with plans to serve it Thursday morning. The chief said he didn't know why Darien was catching a flight to Houston and did not suggest that he was trying to escape.

The Baltimore County school system is recommending Darien's termination, superintendent Myriam Rogers said Thursday.

Meanwhile, artificial intelligence is becoming increasingly powerful and yet “very easy to use,” said Siwei Lyu, director of a media forensics lab at the University at Buffalo.

“You can basically upload any subject’s voice up to this platform,” Lyu told The Associated Press on Thursday. “And then you can give it text and you can start creating voices of that person.”

A recording of someone talking for a minute or two can be gleaned from social media and used to recreate someone’s voice, Lyu said, noting that it’s not always perfect.

Lyu’s research focuses on identifying AI-generated voices and images. He said the models are becoming more powerful, while detection methods are trying to catch up.

“It’s kind of like a perpetual cat-and-mouse game,” Lyu said. “But if I project the speed of development based on today’s situation, detection will lag behind because we have less resources and are not getting as much attention as the generative side.”

This undated photo shows the The Pikesville High School sign on the school property. Baltimore County Police Chief Robert McCullough and other local officials speak at a news conference in Towson, Maryland, on Thursday, April 25, 2024. The officials discussed the arrest of a high school athletic director on charges that he used artificial intelligence to impersonate a principal on an audio recording that included racist and antisemitic comments. (Lloyd Fox/The Baltimore Sun via AP)

This undated photo shows the The Pikesville High School sign on the school property. Baltimore County Police Chief Robert McCullough and other local officials speak at a news conference in Towson, Maryland, on Thursday, April 25, 2024. The officials discussed the arrest of a high school athletic director on charges that he used artificial intelligence to impersonate a principal on an audio recording that included racist and antisemitic comments. (Lloyd Fox/The Baltimore Sun via AP)

Baltimore County Police Chief Robert McCullough and other local officials speak at a news conference in Towson, Maryland, on Thursday April 25, 2024. The officials discussed the arrest of a high school athletic director on charges that he used artificial intelligence to impersonate a principal on an audio recording that included racist and antisemitic comments. (Kim Hairston/The Baltimore Sun via AP)

Baltimore County Police Chief Robert McCullough and other local officials speak at a news conference in Towson, Maryland, on Thursday April 25, 2024. The officials discussed the arrest of a high school athletic director on charges that he used artificial intelligence to impersonate a principal on an audio recording that included racist and antisemitic comments. (Kim Hairston/The Baltimore Sun via AP)

Recommended Articles