AI is Only 30% Away From Matching Human-Level General Intelligence on GAIA Benchmark

MOUNTAIN VIEW, Calif.--(BUSINESS WIRE)--Dec 23, 2024--

H2O.ai, the leader in open-source Generative AI and the most accurate Predictive AI platforms, today announced that h2oGPTe Agent has secured the #1 position on the GAIA (General AI Assistants) benchmark leaderboard with an unprecedented score of 65% — outperforming Google’s Langfun Agent (49%), Microsoft Research (38%), and Hugging Face (33%) leading entries. This remarkable achievement underscores H2O.ai's dominance in the emerging domain of general-purpose AI agents, setting a new gold standard for the industry.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20241223840924/en/

Why GAIA Matters

The GAIA benchmark measures how useful AI systems are in solving real-world tasks that require a lot of time, thought and effort for skilled humans. It consists of hundreds of challenges that require laborious research, data analysis, document handling and reasoning. Degree-holding human respondents achieve a score of 92% and require several human-days to solve all 300 test set problems.

h2oGPTe Agent outpaced competitors by delivering consistent robustness, accuracy and efficiency, highlighting its readiness for enterprise use cases that depend heavily on skilled human assistants.

Enterprise h2oGPTe Agent: A Landmark Achievement

This achievement solidifies H2O.ai’s leadership in the global race to build intelligent, adaptable AI assistants capable of transforming businesses.

Sri Ambati, Founder and CEO of H2O.ai, shared his enthusiasm:

“Today we are announcing that AI is only 30% away from matching human-level general intelligence on the GAIA benchmark. Open-ended questions in GAIA are a better measure of intelligence than MMLU, which relies on multiple choice. To share how exciting this is: the entire Gen AI ecosystem was barely able to pass a tenth in accuracy on one of the toughest AGI benchmarks merely a year ago.

“Makers at H2O.ai built h2oGPTe Agentic AI wielding the best models in the world for reasoning, multi-modal image, video, language understanding, code generation and execution to ace the GAIA benchmark with a stunning 15% accuracy leap over the previous record set by researchers from Google Deepmind using the same Claude-3.5-Sonnet. h2oGPTe Agent also beat Microsoft Research’s agent Magentic-1 that used OpenAI’s o1 model by 27%.

“Agentic AI is eating SaaS and with h2oGPTe Agentic AI now being generally available, all our enterprise customers can solve a wide range of sophisticated business and research problems.”

H2O.ai's success on GAIA underscores its philosophy of simplicity and adaptability:

H2O.ai’s win reaffirms its leadership in AI innovation, particularly in agentic systems poised to reshape business workflows.

Enterprise h2oGPTe 1.6 includes the Agent feature and is available on all public clouds, virtual private clouds and for on-premise deployments: https://h2o.ai/platform/enterprise-h2ogpte/

Read technical blog https://h2o.ai/blog/2024/h2o-ai-tops-gaia-leaderboard/

About H2O.ai

Founded in 2012, H2O.ai is at the forefront of the AI movement to democratize Generative AI. H2O.ai’s open-source Generative AI and Enterprise h2oGPTe, combined with Document AI and the award-winning autoML Driverless AI, have transformed more than 20,000 global organizations, and over half of the Fortune 500, including AT&T, Commonwealth Bank of Australia, Chipotle, Singtel, Workday, Progressive Insurance, and AES.

H2O.ai partners include Dell, Deloitte, Ernst & Young (EY), PricewaterhouseCoopers (PwC), NVIDIA, Snowflake, AWS, Google Cloud Platform (GCP) and Microsoft Azure. H2O.ai’s AI for Good program supports nonprofit groups, foundations, and communities in advancing education, healthcare, and environmental conservation. With a vibrant community of 2 million data scientists worldwide, H2O.ai aims to co-create valuable AI applications for all users.

H2O.ai has raised $256 million from investors, including Commonwealth Bank, Nvidia, Goldman Sachs, Wells Fargo, Capital One, Nexus Ventures and New York Life.

Example GAIA challenge solved by the h2oGPTe Agent (Graphic: Business Wire)

Example GAIA challenge solved by the h2oGPTe Agent (Graphic: Business Wire)

h2oGPTe Agent Tops GAIA Benchmark Test Results Dec 2024 (Graphic: Business Wire)

NEW YORK (AP) — For one Wisconsin couple, the loss of government-sponsored health subsidies next year means choosing a lower-quality insurance plan with a higher deductible. For a Michigan family, it means going without insurance altogether.

For a single mom in Nevada, the spiking costs mean fewer Christmas gifts this year. She is stretching her budget already while she waits to see if Congress will act.

Less than three weeks remain until the expiration of COVID-era enhanced tax credits that have helped millions of Americans pay their monthly fees for Affordable Care Act coverage for the past four years.

The Senate on Thursday rejected two proposals to address the problem and an emerging health care package from House Republicans does not include an extension, all but guaranteeing that many Americans will see much higher insurance costs in 2026.

Here are a few of their stories.

Chad Bruns comes from a family of savers. That came in handy when the 58-year-old military veteran had to leave his firefighting career early because of arm and back injuries he incurred on the job.

He and his wife, Kelley, 60, both retirees, cut their own firewood to reduce their electricity costs in their home in Sawyer County, Wisconsin. They rarely eat out and hardly ever buy groceries unless they are on sale.

But to the extent that they have always been frugal, they will be forced to be even more so now, Bruns said. That is because their coverage under the health law enacted under former President Barack Obama is getting more expensive -– and for worse coverage.

This year, the Brunses were paying $2 per month for a top-tier gold-level plan with less than a $4,000 deductible. Their income was low enough to help them qualify for a lot of financial assistance.

But in 2026, that same plan is rising to an unattainable $1,600 per month, forcing them to downgrade to a bronze plan with a $15,000 deductible.

Kelley Bruns said she is concerned that if something happens to their health in the next year, they could go bankrupt. While their monthly fees are low at about $25, their new out-of-pocket maximum at $21,000 amounts to nearly half their joint income.

“We have to pray that we don’t have to have surgery or don’t have to have some medical procedure done that we’re not aware of,” she said. “It would be very devastating.”

Dave Roof’s family of four has been on ACA insurance since the program started in 2014. Back then, the accessibility of insurance on the marketplace helped him feel comfortable taking the leap to start a small music production and performance company in his hometown of Grand Blanc, Michigan. His wife, Kristin, is also self-employed as a top seller on Etsy.

The coverage has worked for them so far, even when emergencies come up, such as an ATV accident their 21-year-old daughter had last year.

But now, with the expiration of subsidies that kept their premiums down, the 53-year-old Roof said their $500 per month insurance plan is jumping to at least $700 a month, along with spiking deductibles and out-of-pocket costs.

With their joint income of about $75,000 a year, that increase is not manageable, he said. So, they are planning to go without health insurance next year, paying cash for prescriptions, checkups and anything else that arises.

Roof said his family is already living cheaply and has not taken a vacation together since 2021. As it is, they do not save money or add it to their retirement accounts. So even though forgoing insurance is stressful, it is what they must do.

“The fear and anxiety that it’s going to put on my wife and I is really hard to measure,” Roof said. “But we can’t pay for what we can’t pay for.”

If you ask Katelin Provost, the American middle class has gone from experiencing a squeeze to a “full suffocation.”

The 37-year-old social worker in Henderson, Nevada, counts herself in that category. As a single mom, she already keeps a tight budget to cover housing, groceries and day care for her 4-year-old daughter.

Next year, that is going to be even tougher.

The monthly fee on her plan is going up from $85 to nearly $750. She decided she is going to pay that higher cost for January and reevaluate afterward, depending on whether lawmakers extend the subsidies, which as of now appears unlikely. She hopes they will.

If Congress does not act, she will drop herself off the health insurance and keep it only for her daughter because she cannot afford the higher fee for the two of them over the long term.

The strain of one month alone is enough to have an impact.

“I’m going to have to reprioritize the next couple of months to rebalance that budget,” Provost said. “Christmas will be much smaller.”

FILE - Pages from the U.S. Affordable Care Act health insurance website healthcare.gov are seen on a computer screen in New York, Aug. 19, 2025. (AP Photo/Patrick Sison, File)