AWS Trainium2 Instances Now Generally Available

LAS VEGAS--(BUSINESS WIRE)--Dec 3, 2024--

At AWS re:Invent, Amazon Web Services, Inc. (AWS), an Amazon.com, Inc. company (NASDAQ: AMZN), today announced the general availability of AWS Trainium2-powered Amazon Elastic Compute Cloud (Amazon EC2) instances, introduced new Trn2 UltraServers, enabling customers to train and deploy today’s latest AI models as well as future large language models (LLM) and foundation models (FM) with exceptional levels of performance and cost efficiency, and unveiled next-generation Trainium3 chips.

This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20241203165432/en/

“Trainium2 is purpose built to support the largest, most cutting-edge generative AI workloads, for both training and inference, and to deliver the best price performance on AWS,” said David Brown, vice president of Compute and Networking at AWS. “With models approaching trillions of parameters, we understand customers also need a novel approach to train and run these massive workloads. New Trn2 UltraServers offer the fastest training and inference performance on AWS and help organizations of all sizes to train and deploy the world’s largest models faster and at a lower cost.”

As models grow in size, they are pushing the limits of compute and networking infrastructure as customers seek to reduce training times and inference latency—the time between when an AI system receives an input and generates the corresponding output. AWS already offers the broadest and deepest selection of accelerated EC2 instances for AI/ML, including those powered by GPUs and ML chips. But even with the fastest accelerated instances available today, customers want more performance and scale to train these increasingly sophisticated models faster, at a lower cost. As model complexity and data volumes grow, simply increasing cluster size fails to yield faster training time due to parallelization constraints. Simultaneously, the demands of real-time inference push single-instance architectures beyond their capabilities.

Trn2 is the highest performing Amazon EC2 instance for deep learning and generative AI

Trn2 offers 30-40% better price performance than the current generation of GPU-based EC2 instances. A single Trn2 instance combines 16 Trainium2 chips interconnected with ultra-fast NeuronLink high-bandwidth, low-latency chip-to-chip interconnect to provide 20.8 peak petaflops of compute, ideal for training and deploying models that are billions of parameters in size.

Trn2 UltraServers meet increasingly demanding AI compute needs of the world’s largest models

For the largest models that require even more compute, Trn2 UltraServers allow customers to scale training beyond the limits of a single Trn2 instance, reducing training time, accelerating time to market, and enabling rapid iteration to improve model accuracy. Trn2 UltraServers are a completely new EC2 offering that use ultra-fast NeuronLink interconnect to connect four Trn2 servers together into one giant server. With new Trn2 UltraServers, customers can scale up their generative AI workloads across 64 Trainium2 chips. For inference workloads, customers can use Trn2 UltraServers to improve real-time inference performance for trillion-parameter models in production. Together with Anthropic, AWS is building an EC2 UltraCluster of Trn2 UltraServers, named Project Rainier, which will scale out distributed model training across hundreds of thousands of Trainium2 chips interconnected with third-generation, low-latency petabit scale EFA networking—more than 5x the number of exaflops that Anthropic used to train their current generation of leading AI models. When completed, it is expected to be the world’s largest AI compute cluster reported to date available for Anthropic to build and deploy their future models on.

Anthropic is an AI safety and research company that creates reliable, interpretable, and steerable AI systems. Anthropic’s flagship product is Claude, an LLM trusted by millions of users worldwide. As part of Anthropic’s expanded collaboration with AWS, they’ve begun optimizing Claude models to run on Trainium2, Amazon’s most advanced AI hardware to date. Anthropic will be using hundreds of thousands of Trainium2 chips—over five times the size of their previous cluster—to deliver exceptional performance for customers using Claude in Amazon Bedrock.

Databricks’ Mosaic AI enables organizations to build and deploy quality agent systems. It is built natively on top of the data lakehouse, enabling customers to easily and securely customize their models with enterprise data and deliver more accurate and domain-specific outputs. Thanks to Trainium's high performance and cost-effectiveness, customers can scale model training on Mosaic AI at a low cost. Trainium2’s availability will be a major benefit to Databricks and its customers as demand for Mosaic AI continues to scale across all customer segments and around the world. Databricks, one of the largest data and AI companies in the world, plans to use Trn2 to deliver better results and lower TCO by up to 30% for its customers.

Hugging Face is the leading open platform for AI builders, with more than 2 million models, datasets, and AI applications shared by a community of more than 5 million researchers, data scientists, machine learning engineers, and software developers. Hugging Face has collaborated with AWS over the last couple of years, making it easier for developers to experience the performance and cost benefits of AWS Inferentia and Trainium through the Optimum Neuron open-source library, integrated in Hugging Face Inference Endpoints and now optimized within the new HUGS self-deployment service, available on the AWS Marketplace. With the launch of Trainium2, Hugging Face users will have access to even higher performance to develop and deploy models faster.

poolside is set to build a world where AI will drive the majority of economically valuable work and scientific progress. poolside believes that software development will be the first major capability in neural networks that reaches human-level intelligence. To enable that, they're building FMs, an API, and an assistant to bring the power of generative AI to developers' hands. A key to enable this technology is the infrastructure they’re using to build and run their products. With AWS Trainium2, poolside’s customers will be able to scale their usage of poolside at a price performance ratio unlike other AI accelerators. In addition, poolside plans to train future models with Trainium2 UltraServers, with expected savings of 40% compared to EC2 P5 instances.

Trainium3 chips—designed for the high-performance needs of the next frontier of generative AI workloads

AWS unveiled Trainium3, its next-generation AI training chip. Trainium3 will be the first AWS chip made with a 3-nanometer process node, setting a new standard for performance, power efficiency, and density. Trainium3-powered UltraServers are expected to be 4x more performant than Trn2 UltraServers, allowing customers to iterate even faster when building models and deliver superior real-time performance when deploying them. The first Trainium3-based instances are expected to be available in late 2025.

Enabling customers to unlock the performance of Trainium2 with AWS Neuron software

The Neuron SDK includes compiler, runtime libraries, and tools to help developers optimize their models to run on Trainium. It provides developers with the ability to optimize models for optimal performance on Trainium chips. Neuron is natively integrated with popular frameworks like JAX and PyTorch so customers can continue using their existing code and workflows on Trainium with fewer code changes. Neuron also supports over 100,000 models on the Hugging Face model hub. With the Neuron Kernel Interface (NKI), developers get access to bare metal Trainium chips, enabling them to write compute kernels that maximize performance for demanding workloads.

Neuron software is designed to make it easy to use popular frameworks like JAX to train and deploy models on Trainium2 while minimizing code changes and tie-in to vendor-specific solutions. Google is supporting AWS's efforts to enable customers to use JAX for large-scale training and inference through its native OpenXLA integration, providing users an easy and portable coding path to get started with Trn2 instances quickly. With industry wide open-source collaboration and the availability of Trainium2, Google expects to see increased adoption of JAX across the ML community—a significant milestone for the entire ML ecosystem.

Trn2 instances are generally available today in the US East (Ohio) AWS Region, with availability in additional regions coming soon. Trn2 UltraServers are available in preview.

To learn more, visit:

About Amazon Web Services

Since 2006, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud. AWS has been continually expanding its services to support virtually any workload, and it now has more than 240 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, media, and application development, deployment, and management from 108 Availability Zones within 34 geographic regions, with announced plans for 18 more Availability Zones and six more AWS Regions in Mexico, New Zealand, the Kingdom of Saudi Arabia, Taiwan, Thailand, and the AWS European Sovereign Cloud. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—trust AWS to power their infrastructure, become more agile, and lower costs. To learn more about AWS, visit aws.amazon.com.

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Amazon strives to be Earth's Most Customer-Centric Company, Earth's Best Employer, and Earth's Safest Place to Work. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Career Choice, Fire tablets, Fire TV, Amazon Echo, Alexa, Just Walk Out technology, Amazon Studios, and The Climate Pledge are some of the things pioneered by Amazon. For more information, visit amazon.com/about and follow @AmazonNews.

AWS Trainium2 chip (Photo: Business Wire)

Amazon EC2 Trn2 UltraServers (Photo: Business Wire)

Uvira, CONGO (AP) — A climate of fear reigned Saturday in Uvira, a strategic city in eastern Congo, days after it fell to the Rwanda -backed M23 group, as fighting in the region escalated despite a U.S. mediated peace deal.

The Associated Press gained rare access to the city, which was the Congo government’s last major foothold in South Kivu province after the provincial capital of Bukavu fell to the rebels in February. Its capture allows the rebels to consolidate a broad corridor of influence across the east.

M23 said it took control of Uvira earlier this week, following a rapid offensive launched at the start of the month. Along with the more than 400 people killed, about 200,000 have been displaced, regional officials say.

On Saturday, the situation in Uvira still had not returned to normal. There was absolute silence and no traffic, apart from military jeeps circulating on the empty streets. The banks were closed and people have not resumed their jobs — only a few dared to go out during the day, and no one ventured outside after sunset, with armed M23 fighters patrolling the city.

“Some people left the city, but we stayed," Maria Esther, a 45-year-old mother of 10, told AP. “But the situation hasn’t returned to normal, we haven’t resumed our usual activities because there’s no money circulating.”

Joli Bulambo, another resident of Uvira, said: “People thought that the situation that had happened in Goma with the deaths would be the same here in Uvira, but fortunately, there were not many deaths because God helped."

The rebels’ latest offensive comes despite a U.S.-mediated peace agreement signed last week by the Congolese and Rwandan presidents in Washington.

The United States accused Rwanda of violating the agreement by backing a deadly new rebel offensive in the mineral-rich eastern Congo, and warned that the Trump administration will take action against “spoilers” of the deal.

The accord didn’t include the rebel group, which is negotiating separately with Congo and agreed earlier this year to a ceasefire that both sides accuse the other of violating. However, it obliges Rwanda to halt support for armed groups like M23 and work to end hostilities.

Marco Rubio, U.S. Secretary of State, said on X on Saturday: “Rwanda’s actions in eastern DRC are a clear violation of the Washington Accords signed by President Trump, and the United States will take action to ensure promises made to the President are kept.”

There was no immediate reaction from Rwanda.

The rebels’ advance pushed the conflict to the doorstep of neighboring Burundi, which has maintained troops in eastern Congo for years, heightening fears of a broader regional spillover.

More than 100 armed groups are vying for a foothold in mineral-rich eastern Congo, near the border with Rwanda, most prominently M23. The conflict has created one of the world’s most significant humanitarian crises, with more than 7 million people displaced, according to the U.N. agency for refugees.

Local U.N. partners report that more than 200,000 people have been displaced across the province since Dec. 2. Civilians also have crossed into Burundi, and there have been reports of shells falling in the town of Rugombo, on the Burundian side of the border, raising concerns about the conflict spilling over into Burundian territory.

Congo, the U.S. and U.N. experts accuse Rwanda of backing M23, which has grown from hundreds of members in 2021 to around 6,500 fighters, according to the U.N.

Congo’s Foreign Minister Thérèse Kayikwamba Wagner on Friday accused Rwanda of trampling on the peace agreement, which she described as bringing “hope of a historic turning point.”

She warned, however, that the “entire process … is at stake,” and urged the Security Council to impose sanctions against military and political leaders responsible for the attacks, ban mineral exports from Rwanda and prohibit it from contributing troops to U.N. peacekeeping missions.

“Rwanda continues to benefit, especially financially but also in terms of reputation, from its status as a troop-contributing country to peacekeeping missions,” Wagner told AP.

Bertrand Bisimwa, deputy coordinator of the AFC/M23 rebel movement told AP in an exclusive interview Friday that peace commitments have remained largely theoretical. “Regardless of the ceasefire agreements we sign and the mutual commitments we make, nothing is implemented on the ground,” he said.

Asked about the expansion of M23 operations toward the Uvira region, Bisimwa said the region was a long-standing hot spot of ethnic tensions and violence. “For a long time, people were attacked and killed because of their community affiliation,” he said.

On Friday, Rwanda’s Foreign Minister Olivier Nduhungirehe told diplomats that Congo had declared it would continue fighting in M23 recaptured territories and it was only after M23 retaliated that the international community “suddenly woke up.”

“The DRC has openly declared that it would not observe any ceasefire and would instead continue fighting to recapture territories held by the AFC/M23, even as the peace process unfolded," he said.

While Rwanda denies the claim that it backs M23, it acknowledged last year that it has troops and missile systems in eastern Congo, allegedly to safeguard its security. U.N. experts estimate there are up to 4,000 Rwandan forces in Congo.

Associated Press writers Ruth Alonga in Goma, Congo, and Evelyne Musambi in Nairobi, Kenya, contributed to this report.

A boy walks past a burned-out vehicle as residents return to their homes in Luvungi, Democratic Republic of Congo, Saturday, Dec. 13, 2025. (AP Photo/Moses Sawasawa)

Displaced people who fled the war between FARDC and M23 rebels walk with their belongings as they return to their homes in Luvungi, Democratic Republic of Congo, Saturday, Dec. 13, 2025. (AP Photo/Moses Sawasawa)

A man rides a bicycle along a street as people return to homes in Uvira Democratic Republic of Congo, Saturday, Dec. 13, 2025. (AP Photo/Moses Sawasawa)

Displaced people who fled the war between FARDC and M23 rebels walk with their belongings as they return to their homes in Luvungi, Democratic Republic of Congo, Saturday, Dec. 13, 2025. (AP Photo/Moses Sawasawa)

People buy and sell as residents return to their homes in Uvira, Democratic Republic of Congo, Saturday, Dec. 13, 2025. (AP Photo/Moses Sawasawa)

Soldiers patrol as thousands of people fleeing fighting in Congo's South Kivu province arrive in Cibitoke, Kansega, Burundi, Thursday, Dec. 11, 2025. (AP Photo/Berthier Mugiraneza)

Internally displaced people (IDPs) fleeing fighting in Congo's South Kivu province arrive in Cibitoke, Kansega, Burundi, Thursday, Dec. 11, 2025. (AP Photo/Berthier Mugiraneza)

Internally displaced people (IDPs) who fled fighting in Congo's South Kivu province prepare a meal in Cibitoke, Kansega, Burundi, Thursday, Dec. 11, 2025. (AP Photo/Berthier Mugiraneza)

Internally displaced people (IDPs) fleeing fighting in Congo's South Kivu province arrive in Cibitoke, Kansega, Burundi, Thursday, Dec. 11, 2025. (AP Photo/Berthier Mugiraneza)

FILE - Democratic Republic of the Congo's Foreign Minister Therese Kayikwamba Wagner attends a signing ceremony for a peace agreement between Rwanda and the Democratic Republic of the Congo at the State Departmentin Washington, June 27, 2025. (AP Pho to/Mark Schiefelbein, File)