Skip to Content Facebook Feature Image

Pennsylvania sues AI company, saying its chatbots illegally hold themselves out as licensed doctors

News

Pennsylvania sues AI company, saying its chatbots illegally hold themselves out as licensed doctors
News

News

Pennsylvania sues AI company, saying its chatbots illegally hold themselves out as licensed doctors

2026-05-06 00:33 Last Updated At:00:40

HARRISBURG, Pa. (AP) — Pennsylvania has sued an artificial intelligence chatbot maker, saying its chatbots illegally hold themselves out as doctors and are deceiving the system's users into thinking they are getting medical advice from a licensed professional.

The lawsuit, filed Friday, asks the statewide Commonwealth Court to order Character Technologies Inc., the company behind Character.AI, to stop its chatbots “from engaging in the unlawful practice of medicine and surgery.”

Gov. Josh Shapiro's administration called it a “first of its kind enforcement action” by a governor and it comes amid growing pressure by states on tech companies to rein in its chatbots' potentially dangerous messages, especially to children.

That includes a consumer protection lawsuit filed by Kentucky against Character Technologies, and warnings by state attorneys general that chatbots are potentially violating a raft of state laws.

Pennsylvania's lawsuit said an investigator from the state agency that licenses professionals created an account on Character.AI, searched on the word “psychiatry” and found a large number of characters, including one described as a “doctor of psychiatry."

That character held itself out as able to assess the investigator “as a doctor" who is licensed in Pennsylvania, the lawsuit said.

“Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Gov. Josh Shapiro said in a statement. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional."

Character.AI declined to comment on the lawsuit Tuesday but sent a statement saying it prioritizes responsible product development and the well-being of its users. It posts disclaimers to inform users that characters on its website are not real people and that everything they say “should be treated as fiction,” the statement said.

Those disclaimers also say users should not rely on characters for professional advice, it said.

In December, attorneys general from 39 states and Washington, D.C., wrote to Character Technologies and 12 other AI and tech firms — including Anthropic, Meta, Apple, Microsoft, OpenAI, Google and xAI — to warn them about a rise in misleading and manipulative chatbot messages that violate state laws.

In the letter, they said “it is illegal to provide mental health advice without a license, and doing so can both decrease trust in the mental health profession and deter customers from seeking help from actual professionals.”

There are a growing number of wrongful death legal actions against AI chatbot makers across the country and Character Technologies has faced several lawsuits over child safety, including the lawsuit filed by Kentucky.

In January, Google and Character Technologies agreed to settle a lawsuit from a Florida mother who alleged a chatbot pushed her teenage son to kill himself. Last fall, Character.AI banned minors from using its chatbots amid growing concerns about the effects of artificial intelligence conversations on children.

Follow Marc Levy at http://twitter.com/timelywriter

FILE - Pennsylvania Gov. Josh Shapiro speaks to the crowd at a Centre County Democratic Party event at the Penn Stater hotel, April 11, 2026, in State College, Pa. (AP Photo/Marc Levy, File)

FILE - Pennsylvania Gov. Josh Shapiro speaks to the crowd at a Centre County Democratic Party event at the Penn Stater hotel, April 11, 2026, in State College, Pa. (AP Photo/Marc Levy, File)

WASHINGTON (AP) — Senate Republicans have added $1 billion in White House security upgrades to legislation that would fund immigration enforcement agencies, a proposed boost for President Donald Trump’s ballroom project after a man was charged with trying to assassinate him at the White House Correspondents' Association dinner last week.

The GOP bill released late Monday would designate the money for the U.S. Secret Service for “security adjustments and upgrades” related to the ballroom project, which Trump and Republicans have been pushing since Cole Tomas Allen allegedly stormed the April 25 media dinner at the Washington Hilton with guns and knives. The legislation says the money would support enhancements to the ballroom project, “including above-ground and below-ground security features,” but also specifies that the money may not be used for non-security elements.

White House spokesperson Davis Ingle praised Republicans for including the money for the “long overdue” project, saying it would “provide the United States Secret Service with the resources they need to fully and completely harden the White House complex, in addition to the many other critical missions for the USSS.”

The money is part of a larger bill to pay for Immigration and Customs Enforcement and Border Patrol, as Democrats have been blocking funds for both agencies since mid-February. Congress passed bipartisan legislation to fund the rest of the Homeland Security Department on April 30 after a record-long shutdown, but Republicans are using a partisan budget maneuver to push through the ICE and Border Patrol dollars on their own. The House has not released its bill yet, but the Senate is expected to start voting on its version of the legislation next week.

It is unclear exactly how the $1 billion would be used, and the amount far exceeds the proposed $400 million for construction of the ballroom. The White House has said in court documents that the East Wing project would be “heavily fortified,” including bomb shelters, military installations and a medical facility underneath the ballroom. Trump has said it should include bulletproof glass and be able to repel drone attacks.

The National Trust for Historic Preservation has sued to block construction of the project, but a federal appeals court said last month that it can continue in the meantime.

The White House has said that private money would pay for the construction but public money would be used for security measures. Some Republicans have suggested that public money pay for all of it, arguing the security breach at the dinner shows the president needs a secure place to host events.

“It would be insane” to hold the dinner at a hotel again, said Republican Sen. Lindsey Graham of South Carolina, who introduced a bill to pay for the ballroom’s construction with Sen. Katie Britt, R-Ala.

Democrats have said they will oppose any efforts to pay for the ballroom.

“While Americans are struggling to make ends meet as a result of President Trump’s failed policies, Republicans are focused on providing tens of billions of dollars for the President’s vanity ballroom project and cruel mass deportation campaign,” said Illinois Sen. Dick Durbin, the top Democrat on the Senate Judiciary Committee, which oversees the U.S. Secret Service.

Associated Press writer Darlene Superville contributed to this report.

Construction of the new White House Ballroom is seen from a window in the East Room Monday, May 4, 2026, at the White House in Washington. (AP Photo/Mark Schiefelbein)

Construction of the new White House Ballroom is seen from a window in the East Room Monday, May 4, 2026, at the White House in Washington. (AP Photo/Mark Schiefelbein)

Recommended Articles