Compute Center Power/Cooling: 8.64 kW/Node Composition

The engineering parameters for power supply and cooling in compute centers directly impact TCO and compute construction efficiency. The typical value of 8.64 kW/node is not a simple sum of individual component power consumption, but rather the result of engineering trade-offs among GPU computing, storage acceleration, network interconnect, and cooling subsystems under typical high-density deployment. This article, based on measured data from the Mingxin FX100 on the AMD MI308X platform, breaks down the component-level sources of this power consumption and discusses its implications for data center planning.

Component-Level Breakdown of 8.64 kW/Node

In a typical compute center node configuration, the 8.64 kW power consumption is primarily composed of the following subsystems:

• GPU Compute Subsystem (approx. 6.0–6.5 kW): Taking 8× AMD MI308X as an example, each card has a TDP of approximately 700 W (including HBM and on-board power), totaling about 5.6 kW for 8 cards. Adding PCIe slot power losses and GPU interconnect (e.g., Infinity Fabric) power, the total is approximately 6.0–6.5 kW. This is the largest source of node power, accounting for about 70–75%.
• CPU and Memory Subsystem (approx. 0.8–1.2 kW): 2× AMD EPYC 9654 (TDP 360 W each) total about 720 W, plus approximately 1.5 TB of DDR5 memory (each DIMM about 8–12 W), totaling about 0.8–1.2 kW.
• Storage Acceleration Subsystem (approx. 0.5–0.8 kW): The Mingxin FX100 all-flash NVMe-oF array (4-disk RAID0, 14 TB) consumes approximately 0.5–0.8 kW under full load. This value comes from Mingxin internal testing (measured, report R1). Compared to a local NVMe single disk (about 15 W/disk), the array power is higher due to the NVMe-oF controller and network interface card (RoCEv2, 100GbE), but it yields significant improvements in KV cache throughput (e.g., measured, report R2 shows TTFT reduction of 26–32%).
• Network Interconnect Subsystem (approx. 0.3–0.5 kW): This includes RoCEv2 NICs (100GbE, about 20 W/port), switch port allocation (about 5–10 W/port), and in-rack cable losses, totaling about 0.3–0.5 kW.
• Cooling and Auxiliary Subsystem (approx. 0.5–0.8 kW): Pumps, fans, and coolant distribution units (CDUs) for liquid or air cooling, converted based on a typical data center PUE of 1.2–1.4, result in about 0.5–0.8 kW per node.

The above totals approximately 8.2–9.8 kW, with 8.64 kW being a common median configuration in the industry. This value is reflected in reference designs from both Intel and AMD (e.g., Intel's Open Rack V3 standard node power limit is 10 kW).

Impact of 8.64 kW on TCO and Compute Construction

In compute center TCO, electricity costs account for about 40–60% of operational expenses (per IDC). At 8.64 kW/node, this means:

• Electricity Cost: At ¥0.6/kWh and 8,760 hours of annual operation, the annual electricity cost per node is approximately ¥45,400 (8.64 kW × 8,760 h × ¥0.6/kWh). For a 1,000-node deployment, annual electricity costs exceed ¥45 million.
• Cooling Investment: An 8.64 kW node requires at least 30–40 kW of cooling capacity per rack (assuming 4–5 nodes per rack). The initial investment for liquid cooling solutions (e.g., cold plate) is about 1.5–2 times that of air cooling, but PUE can be reduced from 1.6 to 1.15, potentially lowering TCO by 15–25% over the long term (per TrendForce).
• Compute Density: An 8.64 kW node allows deployment of 4–5 nodes per standard 42U rack (each node occupying 8–10U), achieving a power density of 30–45 kW per rack. This requires data center power distribution and cooling systems to support high-density design; otherwise, derating deployment is necessary.

The Mingxin FX100 accounts for about 5–8% of the power consumption in an 8.64 kW node, but its KV cache acceleration (e.g., measured, report R2 shows throughput improvement of 29–40%) can significantly reduce the energy cost per token. For example, in 480B model inference, the FX100 reduces TTFT by 26–32%, meaning more requests can be processed in the same time, improving energy efficiency per unit of compute by approximately 30%.

Power Optimization and Compute Construction Decisions

For power optimization of 8.64 kW nodes, compute construction decision-makers should consider:

1. Power Trade-offs in Storage Acceleration: NVMe-oF arrays (e.g., FX100) increase power by about 0.5 kW compared to local NVMe, but KV cache throughput improves by 5.3× (measured, report R1). In long-context inference scenarios, this reduces GPU idle waiting time, thereby lowering overall node power consumption.
2. Choice of Cooling Solution: For 8.64 kW nodes, air cooling is feasible below 30 kW/rack, but liquid cooling is required above 40 kW/rack. Although liquid cooling increases initial investment, it supports higher-density GPU deployments (e.g., 8× MI308X) and reduces PUE to 1.1–1.2.
3. Network Power Optimization: RoCEv2, compared to InfiniBand, consumes about 30% less power at the same bandwidth (100GbE) (per NVIDIA data), making it suitable for large model inference scenarios that are not extremely latency-sensitive.

Measured data from the Mingxin FX100 on the AMD MI308X platform shows that through storage acceleration optimization, throughput can be improved without increasing the number of GPUs, indirectly reducing the power cost per token. For example, in the R2 test, the FX100 increased the throughput of a 480B model from 4.1 tok/s to 74.9 tok/s (in a scenario without external memory recomputation), with only about 0.6 kW additional power, achieving an energy efficiency improvement of over 18×.

Conclusion

The power composition of 8.64 kW/node is a core parameter in compute center engineering design, reflecting the balance among GPU, storage, network, and cooling. As a storage acceleration component, the Mingxin FX100, with limited incremental power, significantly improves inference efficiency through KV cache acceleration, offering a viable path to reduce TCO for compute construction. For further details on the power and performance of the FX100 on your platform, please contact the Mingxin technical team for joint testing collaboration.

Key Points Q&A

Q: What subsystems primarily constitute 8.64 kW/node?
A: The GPU compute subsystem accounts for about 6.0–6.5 kW (70–75%), CPU and memory about 0.8–1.2 kW, storage acceleration subsystem (e.g., Mingxin FX100) about 0.5–0.8 kW, network interconnect about 0.3–0.5 kW, and cooling auxiliary about 0.5–0.8 kW.

Q: What is the power proportion and energy efficiency benefit of the Mingxin FX100 in an 8.64 kW node?
A: The FX100 consumes about 0.5–0.8 kW (5–8% of total), but in 480B model inference, it improves throughput by 29–40% (measured, report R2), reduces TTFT by 26–32%, and improves energy efficiency per unit of compute by about 30%.

Q: How does an 8.64 kW node impact compute center TCO?
A: Annual electricity cost per node is approximately ¥45,400 (at ¥0.6/kWh); liquid cooling can reduce PUE to 1.15, lowering long-term TCO by 15–25% (per TrendForce); high-density deployment requires power distribution and cooling systems to support 30–45 kW/rack.