Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said

SAN FRANCISCO (AP) — Tech behemoth OpenAI has touted its artificial intelligence-powered transcription tool Whisper as having near “human level robustness and accuracy.”

But Whisper has a major flaw: It is prone to making up chunks of text or even entire sentences, according to interviews with more than a dozen software engineers, developers and academic researchers. Those experts said some of the invented text — known in the industry as hallucinations — can include racial commentary, violent rhetoric and even imagined medical treatments.

Experts said that such fabrications are problematic because Whisper is being used in a slew of industries worldwide to translate and transcribe interviews, generate text in popular consumer technologies and create subtitles for videos.

More concerning, they said, is a rush by medical centers to utilize Whisper-based tools to transcribe patients’ consultations with doctors, despite OpenAI’ s warnings that the tool should not be used in “high-risk domains.”

The full extent of the problem is difficult to discern, but researchers and engineers said they frequently have come across Whisper’s hallucinations in their work. A University of Michigan researcher conducting a study of public meetings, for example, said he found hallucinations in eight out of every 10 audio transcriptions he inspected, before he started trying to improve the model.

A machine learning engineer said he initially discovered hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed. A third developer said he found hallucinations in nearly every one of the 26,000 transcripts he created with Whisper.

The problems persist even in well-recorded, short audio samples. A recent study by computer scientists uncovered 187 hallucinations in more than 13,000 clear audio snippets they examined.

That trend would lead to tens of thousands of faulty transcriptions over millions of recordings, researchers said.

This story was produced in partnership with the Pulitzer Center’s AI Accountability Network, which also partially supported the academic Whisper study. AP also receives financial assistance from the Omidyar Network to support coverage of artificial intelligence and its impact on society.

Such mistakes could have “really grave consequences,” particularly in hospital settings, said Alondra Nelson, who led the White House Office of Science and Technology Policy for the Biden administration until last year.

“Nobody wants a misdiagnosis,” said Nelson, a professor at the Institute for Advanced Study in Princeton, New Jersey. “There should be a higher bar.”

Whisper also is used to create closed captioning for the Deaf and hard of hearing — a population at particular risk for faulty transcriptions. That's because the Deaf and hard of hearing have no way of identifying fabrications “hidden amongst all this other text," said Christian Vogler, who is deaf and directs Gallaudet University’s Technology Access Program.

The prevalence of such hallucinations has led experts, advocates and former OpenAI employees to call for the federal government to consider AI regulations. At minimum, they said, OpenAI needs to address the flaw.

“This seems solvable if the company is willing to prioritize it,” said William Saunders, a San Francisco-based research engineer who quit OpenAI in February over concerns with the company's direction. “It’s problematic if you put this out there and people are overconfident about what it can do and integrate it into all these other systems.”

An OpenAI spokesperson said the company continually studies how to reduce hallucinations and appreciated the researchers' findings, adding that OpenAI incorporates feedback in model updates.

While most developers assume that transcription tools misspell words or make other errors, engineers and researchers said they had never seen another AI-powered transcription tool hallucinate as much as Whisper.

The tool is integrated into some versions of OpenAI’s flagship chatbot ChatGPT, and is a built-in offering in Oracle and Microsoft’s cloud computing platforms, which service thousands of companies worldwide. It is also used to transcribe and translate text into multiple languages.

In the last month alone, one recent version of Whisper was downloaded over 4.2 million times from open-source AI platform HuggingFace. Sanchit Gandhi, a machine-learning engineer there, said Whisper is the most popular open-source speech recognition model and is built into everything from call centers to voice assistants.

Professors Allison Koenecke of Cornell University and Mona Sloane of the University of Virginia examined thousands of short snippets they obtained from TalkBank, a research repository hosted at Carnegie Mellon University. They determined that nearly 40% of the hallucinations were harmful or concerning because the speaker could be misinterpreted or misrepresented.

In an example they uncovered, a speaker said, “He, the boy, was going to, I’m not sure exactly, take the umbrella.”

But the transcription software added: “He took a big piece of a cross, a teeny, small piece ... I’m sure he didn’t have a terror knife so he killed a number of people.”

A speaker in another recording described “two other girls and one lady.” Whisper invented extra commentary on race, adding "two other girls and one lady, um, which were Black.”

In a third transcription, Whisper invented a non-existent medication called “hyperactivated antibiotics.”

Researchers aren’t certain why Whisper and similar tools hallucinate, but software developers said the fabrications tend to occur amid pauses, background sounds or music playing.

OpenAI recommended in its online disclosures against using Whisper in “decision-making contexts, where flaws in accuracy can lead to pronounced flaws in outcomes.”

That warning hasn’t stopped hospitals or medical centers from using speech-to-text models, including Whisper, to transcribe what’s said during doctor’s visits to free up medical providers to spend less time on note-taking or report writing.

Over 30,000 clinicians and 40 health systems, including the Mankato Clinic in Minnesota and Children’s Hospital Los Angeles, have started using a Whisper-based tool built by Nabla, which has offices in France and the U.S.

That tool was fine-tuned on medical language to transcribe and summarize patients’ interactions, said Nabla’s chief technology officer Martin Raison.

Company officials said they are aware that Whisper can hallucinate and are addressing the problem.

It’s impossible to compare Nabla’s AI-generated transcript to the original recording because Nabla’s tool erases the original audio for “data safety reasons,” Raison said.

Nabla said the tool has been used to transcribe an estimated 7 million medical visits.

Saunders, the former OpenAI engineer, said erasing the original audio could be worrisome if transcripts aren't double checked or clinicians can't access the recording to verify they are correct.

“You can't catch errors if you take away the ground truth,” he said.

Nabla said that no model is perfect, and that theirs currently requires medical providers to quickly edit and approve transcribed notes, but that could change.

Because patient meetings with their doctors are confidential, it is hard to know how AI-generated transcripts are affecting them.

A California state lawmaker, Rebecca Bauer-Kahan, said she took one of her children to the doctor earlier this year, and refused to sign a form the health network provided that sought her permission to share the consultation audio with vendors that included Microsoft Azure, the cloud computing system run by OpenAI’s largest investor. Bauer-Kahan didn't want such intimate medical conversations being shared with tech companies, she said.

“The release was very specific that for-profit companies would have the right to have this,” said Bauer-Kahan, a Democrat who represents part of the San Francisco suburbs in the state Assembly. “I was like ‘absolutely not.’ ”

John Muir Health spokesman Ben Drew said the health system complies with state and federal privacy laws.

Schellmann reported from New York.

AP is solely responsible for all content. Find AP’s standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.

The Associated Press and OpenAI have a licensing and technology agreement allowing OpenAI access to part of the AP’s text archives.

Assistant professor of information science Allison Koenecke, an author of a recent study that found hallucinations in a speech-to-text transcription tool, sits for a portrait in her office at Cornell University in Ithaca, N.Y., Friday, Feb. 2, 2024. (AP Photo/Seth Wenig)

A computer screen displays text produced by an artificial intelligence-powered transcription program called Whisper at Cornell University in Ithaca, N.Y., Friday, Feb. 2, 2024. In this example, the speaker said, "and after she got the telephone he began to pray" while the program transcribes that as "I feel like I'm going to fall. I feel like I'm going to fall, I feel like I'm going to fall…." (AP Photo/Seth Wenig)

A computer screen displays text produced by an artificial intelligence-powered transcription program called Whisper at Cornell University in Ithaca, N.Y., Friday, Feb. 2, 2024. In this example, the speaker said, "as the um, the, her father dies not too long after he remarried…." while the program transcribes that as " It's fine. It's just too sensitive to tell. She does die at 65…." (AP Photo/Seth Wenig)

A computer screen displays text produced by an artificial intelligence-powered transcription program called Whisper at Cornell University in Ithaca, N.Y., Friday, Feb. 2, 2024. In this example, the speaker said, "and after she got the telephone he began to pray" while the program transcribes that as "I feel like I'm going to fall. I feel like I'm going to fall, I feel like I'm going to fall…." (AP Photo/Seth Wenig)

Assistant professor of information science Allison Koenecke, an author of a recent study that found hallucinations in a speech-to-text transcription tool, works in her office at Cornell University in Ithaca, N.Y., Friday, Feb. 2, 2024.

Assistant professor of information science Allison Koenecke, an author of a recent study that found hallucinations in a speech-to-text transcription tool, works in her office at Cornell University in Ithaca, N.Y., Friday, Feb. 2, 2024. The text preceded by "#Ground truth" shows what was actually said while the sentences preceded by ""text"" was how the transcription program interpreted the words. (AP Photo/Seth Wenig)

NEW YORK (AP) — U.S. stocks are slipping Monday, while oil prices leap on worries that war in the Middle East will slow the global flow of crude and make inflation even worse.

Crude prices jumped 6%, which will likely mean higher prices soon at gasoline pumps. That would hurt not only U.S. households, whose spending makes up the bulk of the U.S. economy, but also businesses with big fuel bills.

The S&P 500 slipped 0.3%, with some of the sharpest losses hitting cruise lines and airlines. It had sunk as much as 1.2% in the morning before trimming its loss.

The Dow Jones Industrial Average was down 199 points, or 0.4%, as of 10:45 a.m. Eastern time, and the Nasdaq composite was 0.1% lower.

Prices climbed for natural gas, meanwhile, which could mean higher heating bills for the remainder of the winter, after a major supplier of liquefied natural gas to Europe said it would stop production because of the war. Gold climbed 2.1% as investors looked for safer things to own and as U.S. officials tried to persuade the world that this war will not last forever.

“This is not Iraq,” U.S. Defense Secretary Pete Hegseth said Monday. “This is not endless.”

Typically, Treasury yields also fall when investors are feeling nervous. But yields instead climbed, in part because higher oil prices will put upward pressure on inflation, which is already worse than what nearly everyone would like. That could tie the Federal Reserve’s hands and keep it from cutting interest rates.

Lower interest rates can boost the economy and job market, while also worsening inflation. Higher rates can do the opposite.

Past military conflicts in the Middle East have not caused long-term drops for markets. For this war to knock down U.S. stocks in a significant and sustained way, the price of oil would perhaps need to jump above $100 per barrel, according to strategists at Morgan Stanley led by Michael Wilson.

Oil prices are still well below there. A barrel of benchmark U.S. crude rose 6.4% to $71.31. Brent crude, the international standard, climbed 7.8% to $78.59 per barrel.

That helped the U.S. stock market pare some of its steep, opening loss. Morgan Stanley says the S&P 500 has climbed an average of 2%, 6% and 8% in the one, six and 12 months following “geopolitical risk events” historically. That's going back to the Korean War, which began in 1950, and the 1956 Suez crisis.

At the moment, though, fear is still running through markets.

Stocks of airlines were some of Monday’s sharpest losers. Not only do higher oil prices threaten their already big fuel bills, the fighting in the Middle East also closed airports and left travelers stranded.

United Airlines fell 3.2%, and American Airlines lost 4.2%.

Norwegian Cruise Line Holdings fell even more, 9.8%. It needs customers to have plenty of cash to spend after paying for their gasoline bills and other essentials.

The cruise operator also reported weaker revenue for its latest quarter than analysts expected, though its profit was better. Its forecast for profit this upcoming fiscal year was lower than analysts expected.

Hotels, discount retailers and other companies that benefit when customers have more cash in their pocket from lower fuel bills also lagged the market. MGM Resorts fell 3.1%, and Dollar Tree lost 2.5%.

Stocks in the housing industry also struggled as higher Treasury yields could translate into more expensive mortgage rates. Paint company Sherwin-Williams fell 2.9%, and homebuilder D.R. Horton lost 3.9%.

Helping to limit Wall Street's losses were oil companies, which benefited from the rising prices for crude. Exxon Mobil climbed 0.9%, and Occidental Petroleum rose 2.1%.

Companies that make equipment for the military also strengthened. Lockheed Martin climbed 3.1%, and RTX rallied 4.3%.

In stock markets abroad, indexes fell across much of Europe and Asia. Germany’s DAX lost 2.5%, France’s CAC 40 fell 2.3% and Hong Kong’s Hang Seng dropped 2.1% for some of the world’s larger losses.

Stocks in Shanghai were an outlier and rose 0.5%.

In the bond market, the yield on the 10-year Treasury rose to 4.04% from 3.97% late Friday. A report showing growth for U.S. manufacturing was better than economists expected last month also helped to lift yields.

AP Business Writers Matt Ott and Elaine Kurtenbach contributed.

James Denaro, center, and others work on the floor at the New York Stock Exchange in New York, Monday, March 2, 2026. (AP Photo/Seth Wenig)

Trader John Bishop works on the floor of the New York Stock Exchange, Friday, Feb. 20, 2026. (AP Photo/Richard Drew)

Iraqi Shiite carry a mock coffin of Iranian supreme Leader Ayatollah Ali Khamenei, who was killed by a U.S. airstrike in Tehran, during a symbolic funeral, in Najaf, Iraq, Sunday, March 1, 2026. (AP Photo/Anmar Khalil)

People walk in front of an electronic stock board showing Japan's Nikkei index at a securities firm Monday, March 1, 2026, in Tokyo.(Yohei Fukai/Kyodo News via AP)

A pedestrian walks outside the New York Stock Exchange during a snow storm, Monday, Feb. 23, 2026, in New York. (AP Photo/Seth Wenig)

FILE - Fishermen work in front of oil tankers south of the Strait of Hormuz Jan. 19, 2012, offshore the town of Ras Al Khaimah in United Arab Emirates. (AP Photo/Kamran Jebreili, File)

Snow falls outside the New York Stock Exchange, Monday, Feb. 23, 2026, in New York. (AP Photo/Seth Wenig)