H100 rental price
median price per GPU-hour$2.56 / GPU-hr
Live chart · updates with every published snapshot
Most memory on an AI accelerator
The largest on-package memory capacity on any reviewed datacenter AI accelerator, held by the current specification leader.
on-package memory capacity256 GBon the AMD Instinct MI325X
View chart data
| As of | on-package memory capacity |
|---|---|
| Jul 26, 00:00 UTC | 256 GB |
About this metric
Model size is bounded by memory before it is bounded by anything else: a model must fit in accelerator memory, in one device or across many, before a single token can be generated. This chart tracks the ceiling, the largest on-package memory capacity on any datacenter AI accelerator in the reviewed cross-vendor specification matrix.
Capacities come from first-party vendor specifications, reviewed and republished per accelerator on the underlying board alongside memory bandwidth. Only shipping datacenter parts with published specifications are included; roadmap claims and unannounced configurations are not, so the ceiling moves when hardware ships, not when it is teased.
Every jump in the memory ceiling changes inference economics: a model that previously needed two accelerators suddenly needs one, and serving costs drop by roughly half for that workload. The vendor holding the ceiling also holds a real pricing lever, which is why memory capacity, not raw compute, has become the headline number in accelerator launches.
Related charts