If you believe artificial intelligence poses grave risks to humanity, then a professor at Carnegie Mellon University has one of the most important roles in the tech industry right now.
Zico Kolter leads a 4-person panel at OpenAI that has the authority to halt the ChatGPT maker's release of new AI systems if it finds them unsafe. That could be technology so powerful that an evildoer could use it to make weapons of mass destruction. It could also be a new chatbot so poorly designed that it will hurt people's mental health.
“Very much we’re not just talking about existential concerns here,” Kolter said in an interview with The Associated Press. “We’re talking about the entire swath of safety and security issues and critical topics that come up when we start talking about these very widely used AI systems.”
OpenAI tapped the computer scientist to be chair of its Safety and Security Committee more than a year ago, but the position took on heightened significance last week when California and Delaware regulators made Kolter's oversight a key part of their agreements to allow OpenAI to form a new business structure to more easily raise capital and make a profit.
Safety has been central to OpenAI's mission since it was founded as a nonprofit research laboratory a decade ago with a goal of building better-than-human AI that benefits humanity. But after its release of ChatGPT sparked a global AI commercial boom, the company has been accused of rushing products to market before they were fully safe in order to stay at the front of the race. Internal divisions that led to the temporary ouster of CEO Sam Altman in 2023 brought those concerns that it had strayed from its mission to a wider audience.
The San Francisco-based organization faced pushback — including a lawsuit from co-founder Elon Musk — when it began steps to convert itself into a more traditional for-profit company to continue advancing its technology.
Agreements announced last week by OpenAI along with California Attorney General Rob Bonta and Delaware Attorney General Kathy Jennings aimed to assuage some of those concerns.
At the heart of the formal commitments is a promise that decisions about safety and security must come before financial considerations as OpenAI forms a new public benefit corporation that is technically under the control of its nonprofit OpenAI Foundation.
Kolter will be a member of the nonprofit's board but not on the for-profit board. But he will have “full observation rights” to attend all for-profit board meetings and have access to information it gets about AI safety decisions, according to Bonta's memorandum of understanding with OpenAI. Kolter is the only person, besides Bonta, named in the lengthy document.
Kolter said the agreements largely confirm that his safety committee, formed last year, will retain the authorities it already had. The other three members also sit on the OpenAI board — one of them is former U.S. Army General Paul Nakasone, who was commander of the U.S. Cyber Command. Altman stepped down from the safety panel last year in a move seen as giving it more independence.
“We have the ability to do things like request delays of model releases until certain mitigations are met,” Kolter said. He declined to say if the safety panel has ever had to halt or mitigate a release, citing the confidentiality of its proceedings.
Kolter said there will be a variety of concerns about AI agents to consider in the coming months and years, from cybersecurity – “Could an agent that encounters some malicious text on the internet accidentally exfiltrate data?” – to security concerns surrounding AI model weights, which are numerical values that influence how an AI system performs.
“But there’s also topics that are either emerging or really specific to this new class of AI model that have no real analogues in traditional security,” he said. “Do models enable malicious users to have much higher capabilities when it comes to things like designing bioweapons or performing malicious cyberattacks?”
“And then finally, there’s just the impact of AI models on people,” he said. “The impact to people’s mental health, the effects of people interacting with these models and what that can cause. All of these things, I think, need to be addressed from a safety standpoint.”
OpenAI has already faced criticism this year about the behavior of its flagship chatbot, including a wrongful-death lawsuit from California parents whose teenage son killed himself in April after lengthy interactions with ChatGPT.
Kolter, director of Carnegie Mellon's machine learning department, began studying AI as a Georgetown University freshman in the early 2000s, long before it was fashionable.
“When I started working in machine learning, this was an esoteric, niche area,” he said. “We called it machine learning because no one wanted to use the term AI because AI was this old-time field that had overpromised and underdelivered.”
Kolter, 42, has been following OpenAI for years and was close enough to its founders that he attended its launch party at an AI conference in 2015. Still, he didn't expect how rapidly AI would advance.
“I think very few people, even people working in machine learning deeply, really anticipated the current state we are in, the explosion of capabilities, the explosion of risks that are emerging right now,” he said.
AI safety advocates will be closely watching OpenAI's restructuring and Kolter's work. One of the company's sharpest critics says he's “cautiously optimistic,” particularly if Kolter's group "is actually able to hire staff and play a robust role.”
“I think he has the sort of background that makes sense for this role. He seems like a good choice to be running this,” said Nathan Calvin, general counsel at the small AI policy nonprofit Encode. Calvin, who OpenAI targeted with a subpoena at his home as part of its fact-finding to defend against the Musk lawsuit, said he wants OpenAI to stay true to its original mission.
“Some of these commitments could be a really big deal if the board members take them seriously,” Calvin said. “They also could just be the words on paper and pretty divorced from anything that actually happens. I think we don’t know which one of those we’re in yet.”
Carnegie Mellon University Head of Machine Learning, Zico Kolter delivers a keynote speech at AI Horizons Summit in Bakery Square on Thursday, Sept. 11, 2025 in Pittsburgh. (Sebastian Foltz/Pittsburgh Post-Gazette via AP)
Carnegie Mellon University Head of Machine Learning, Zico Kolter delivers a keynote speech at AI Horizons Summit in Bakery Square on Thursday, Sept. 11, 2025 in Pittsburgh. (Sebastian Foltz/Pittsburgh Post-Gazette via AP)
WASHINGTON (AP) — The Supreme Court is taking up one of the term’s most consequential cases, President Donald Trump’s executive order on birthright citizenship declaring that children born to parents who are in the United States illegally or temporarily are not American citizens, and he was in the courtroom on Wednesday for some of the arguments.
The justices are hearing Trump’s appeal of a lower-court ruling from New Hampshire that struck down the citizenship restrictions, one of several courts that have blocked them. They have not taken effect anywhere in the country.
Trump is the first sitting president to attend oral arguments at the nation’s highest court. He spent just over an hour inside the courtroom, hearing arguments by the government’s lawyer, Solicitor General D. John Sauer. He left shortly after Sauer wrapped up and the plaintiff was invited to present her case.
The case frames another test of Trump's assertions of executive power that defy long-standing precedent for a court that has largely ruled in the president's favor — but with some notable exceptions that Trump has responded to with starkly personal criticisms of the justices. A definitive ruling is expected by early summer.
The birthright citizenship order, which Trump signed the first day of his second term, is part of his Republican administration’s broad immigration crackdown.
Birthright citizenship is the first Trump immigration-related policy to reach the court for a final ruling. The justices previously struck down global tariffs Trump had imposed under an emergency powers law that had never been used that way.
Trump reacted furiously to the late February tariffs decision, saying he was ashamed of the justices who ruled against him and calling them unpatriotic.
He issued a preemptive broadside against the court on Sunday on his Truth Social platform. “Birthright Citizenship is not about rich people from China, and the rest of the World, who want their children, and hundreds of thousands more, FOR PAY, to ridiculously become citizens of the United States of America. It is about the BABIES OF SLAVES!,” the president wrote. “Dumb Judges and Justices will not a great Country make!”
Trump's order would upend the long-standing view that the Constitution’s 14th Amendment, ratified in 1868, and federal law since 1940 confer citizenship on everyone born on American soil, with narrow exceptions for the children of foreign diplomats and those born to a foreign occupying force.
The 14th Amendment was intended to ensure that Black people, including former slaves, had citizenship, though the Citizenship Clause is written more broadly. “All persons born or naturalized in the United States, and subject to the jurisdiction thereof, are citizens of the United States and of the State wherein they reside,” it reads.
In a series of decisions, lower courts have struck down the executive order as illegal, or likely so, under the Constitution and federal law. The decisions have invoked the high court's 1898 ruling in Wong Kim Ark, which held that the U.S.-born child of Chinese nationals was a citizen.
The Trump administration argues that the common view of citizenship is wrong, asserting that children of noncitizens are not “subject to the jurisdiction” of the United States and therefore are not entitled to citizenship.
The court should use the case to set straight “long-enduring misconceptions about the Constitution’s meaning,” wrote Sauer, the solicitor general.
No court has accepted that argument, and lawyers for pregnant women whose children would be affected by the order said the Supreme Court should not be the first to do so.
“We have the president of the United States trying to radically reinterpret the definition of American citizenship,” said Cecillia Wang, the American Civil Liberties Union legal director who is facing off against Sauer at the Supreme Court.
More than one-quarter of a million babies born in the U.S. each year would be affected by the executive order, according to research by the Migration Policy Institute and Pennsylvania State University’s Population Research Institute.
While Trump has largely focused on illegal immigration in his rhetoric and actions, the birthright restrictions also would apply to people who are legally in the United States, including students and applicants for green cards, or permanent resident status.
Associated Press writer Darlene Superville contributed to this report.
Follow the AP’s coverage of the U.S. Supreme Court at https://apnews.com/hub/us-supreme-court.
Pro and anti-Trump demonstrators rally outside the U.S. Supreme Court, before justices hear oral arguments on whether President Donald Trump can deny citizenship to children born to parents who are in the United States illegally or temporarily, on Capitol Hill, in Washington, Wednesday, April 1, 2026. (AP Photo/J. Scott Applewhite)
President Donald Trump leaves the U.S. Supreme Court, Wednesday, April 1, 2026, in Washington. (AP Photo/Mark Schiefelbein)
President Donald Trump's motorcade arrives at the U.S. Supreme Court, Wednesday, April 1, 2026, in Washington. (AP Photo/Tom Brenner)
Pro and anti-Trump demonstrators rally outside the U.S. Supreme Court, before justices hear oral arguments on whether President Donald Trump can deny citizenship to children born to parents who are in the United States illegally or temporarily, on Capitol Hill, in Washington, Wednesday, April 1, 2026. (AP Photo/J. Scott Applewhite)
President Donald Trump's motorcade arrives at the U.S. Supreme Court, Wednesday, April 1, 2026, in Washington. (AP Photo/Tom Brenner)
Demonstrators holding opposing views verbally engage ahead of President Donald Trump's arrival at the U.S. Supreme Court, Wednesday, April 1, 2026, in Washington. (AP Photo/Tom Brenner)
President Donald Trump's limo exits the White House en route to the Supreme Court, Wednesday, April 1, 2026, in Washington. (AP Photo/Mark Schiefelbein)
President Donald Trump's motorcade arrives at the U.S. Supreme Court, Wednesday, April 1, 2026, in Washington. (AP Photo/Tom Brenner)
Pro and anti-Trump demonstrators rally outside the U.S. Supreme Court, before justices hear oral arguments on whether President Donald Trump can deny citizenship to children born to parents who are in the United States illegally or temporarily, on Capitol Hill, in Washington, Wednesday, April 1, 2026. (AP Photo/J. Scott Applewhite)
People arrive to walk inside the U.S. Supreme Court, on Capitol Hill in Washington, Wednesday, April 1, 2026. The Supreme Court justices will hear oral arguments today on whether President Donald Trump can deny citizenship to children born to parents who are in the United States illegally or temporarily. (AP Photo/J. Scott Applewhite)
People arrive to walk inside the U.S. Supreme Court, on Capitol Hill in Washington, Wednesday, April 1, 2026. The Supreme Court justices will hear oral arguments today on whether President Donald Trump can deny citizenship to children born to parents who are in the United States illegally or temporarily. (AP Photo/J. Scott Applewhite)
President Donald Trump answers questions from reporters after signing an executive order in the Oval Office of the White House Tuesday, March 31, 2026, in Washington, as Commerce Secretary Howard Lutnick listens. (AP Photo/Alex Brandon)
The U.S. Supreme Court is seen as the moon rises Tuesday, March 31, 2026, in Washington. (AP Photo/Jose Luis Magana)
