Meta published a detailed look on Monday, October 19, at Meta Prometheus, the gigawatt-scale AI training cluster it has built in New Albany, Ohio. According to Meta’s engineering post, hundreds of thousands of GPUs are running there today, and the goal is about one million at full scale. The post is unusually specific about what goes wrong when you run that many chips together, and what Meta did about it. This article covers what Meta Prometheus is, the hardware inside it, the power and cooling problems, the reliability claims, and what the numbers do and don’t show.
What Meta Prometheus is and what it trains
Two years ago the Ohio site was a conventional data center that served web traffic. Meta converted it into an AI cluster, and the company says the first phase is finished while later phases are still being built. It describes the result as a gigawatt-scale supercluster. A gigawatt is roughly the output of one large power plant, so the label is about electricity as much as chips.
Meta Prometheus trains the Muse family of models, including Muse Wonder, which Meta calls its frontier models for agentic tasks. Two kinds of work run there: pre-training, where a model learns from enormous piles of data, and reinforcement learning, where it gets refined through trial and feedback. Both need thousands of GPUs moving in lockstep for long stretches, so one flaky chip can stall a lot of expensive hardware. That’s why reliability matters as much as chip count.
The hardware inside Meta Prometheus
The cluster mixes three generations of Nvidia rack-scale systems, each with its own internal name. Catalina uses GB200 NVL72 racks, with 192 GB of HBM3e memory per GPU and about 120 kW for each two-rack pair. Clemente uses GB300 NVL72, with 288 GB per GPU and about 135 kW per rack. Meta puts Clemente roughly 5% to 10% ahead of Catalina on speed.
The newest tier is San Miguel, built on Vera Rubin NVL72 with 288 GB of HBM4 per GPU. Meta lists its draw at more than 170 kW per rack and its efficiency at 10% to 20% better performance per watt than GB300. The trend line is plain: each generation packs more compute into a rack and pulls more power through it. A typical office server rack draws a small fraction of that, so almost everything around the chips has to be redesigned.
Power swings and cooling at Meta Prometheus
Training does not draw power evenly. Meta says synchronized jobs can swing the cluster’s load by 30% to 40%, and starting or pausing a large job can move tens of megawatts within seconds. A grid connection and a building full of electronics are not built for that kind of jolt.
Meta’s answer is a cluster-wide power capping system it calls Dynamo Dimmer. Instead of halting jobs when the cluster nears its electrical ceiling, it lowers GPU power limits gradually. Backing it up are battery units inside the racks, capacitor trays, battery storage at the facility and coordination with local utilities.
Cooling follows the same logic. Meta says the whole site uses closed-loop liquid cooling. Free-air buildings and rapid-deployment structures use air-assisted liquid cooling, while the purpose-built AI building uses facility-level liquid cooling.
Networking at Meta Prometheus scale
Meta runs a dedicated Ethernet back-end network that connects GPUs within and across buildings. Each building’s fabric supports up to 20,736 GPUs. Meta says it moved from its earlier Disaggregated Scheduled Fabric to a Non-Scheduled Fabric built on 51.2T switches.
A separate backend aggregation network links the buildings, over distances from 2 km to more than 100 km, at multiple petabits per second for each pair of regions. Meta’s scale figures are large: millions of miles of fiber, tens of thousands of switches and millions of optical modules. The practical point is that a training run can span buildings and still behave like one machine.
Reliability: the 99% claim
The most striking number in the post concerns failures. Meta says interruptions peaked in mid-2025 at nearly 1,000 events per 8,000 GPUs per day. After a year of engineering work, it reports a 99% reduction. Its stated target is two interruptions per 8,000 GPUs per day, which works out to about 100,000 GPU hours between interruptions.
The company gives two efficiency figures from runs of Muse Wonder. Goodput, the share of time the cluster spends doing useful training work, topped 85% for pre-training and 90% for reinforcement learning. Mean time between interruptions was about 47,000 GPU hours in those runs, later improved to more than 54,000 with firmware fixes.
A few design choices explain the gains. Every NVL72 rack is a 72-GPU domain, yet Meta only uses 64 of them for a job and holds eight in reserve. When a chip fails, a spare takes over and the job keeps going. Meta also validated more than 600,000 NVLink connections before putting the hardware into production, and says firmware updates can reach more than 100,000 GPUs within hours.
What the numbers do and don’t show
Every figure above comes from Meta. Nobody outside the company has audited the interruption counts, the goodput or the GPU totals, and “hundreds of thousands” is not a precise number. We also found no dollar figure for the build in the post. Treat the reliability story as a credible engineering report and an unverified scorecard.
The money question sits in the background. Meta’s spending plans were the center of our look at whether ad growth can keep up with $130 billion-plus in AI spending, and clusters like this are where that money goes. Lenders are watching the wider build-out, as the record price of insurance against a CoreWeave default showed last week.
Supply is the other limit. The SemiAnalysis newsletter, which The Neuron summarized on Tuesday, expects about 50 GW of new data center capacity in a single year, with another 60 GW already being built. It argues that chips and memory, not power, could end up as the main bottleneck. If so, a cluster that fails less often gets more work from the same silicon. That’s the case Meta is making.
What to watch next
Three things are worth following. The first is the Vera Rubin ramp, since Meta says that capacity is now arriving. The second is Hyperion in Richland Parish, Louisiana, which Meta says is being designed with lessons from Meta Prometheus. The third is whether the reliability claims show up in the Muse models themselves, through faster releases or lower costs. Meta’s next earnings call is the first chance to hear how management ties this spending to revenue.
Editor’s take
Meta Prometheus is a useful reminder that frontier AI is as much a plumbing and electrical problem as a software one. The most interesting line in the post isn’t the million-GPU goal, it’s the admission that nearly 1,000 interruptions a day was the starting point. A 99% cut is impressive if it holds up, but it comes from the builder, not an independent reviewer. Read the post as evidence of how hard this work is, and wait for outside confirmation before treating the scorecard as settled.
