1,000-GPU TCO: Hardware, Networking, Power Supply
The TCO (Total Cost of Ownership) of a 1,000-GPU-scale compute center is not determined solely by GPU procurement prices. It comprises five components: server hardware
The TCO (Total Cost of Ownership) of a 1,000-GPU-scale compute center is not determined solely by GPU procurement prices. It comprises five components: server hardware, network interconnection, power and cooling, facility construction, and long-term operations and maintenance. Among these, power and cooling can account for over 40% of TCO over a 5-year cycle. This article uses a 1,000-GPU (equivalent to H100-class) scale as a baseline, combines public market pricing with measured data from Mingxin's FX series in storage acceleration, breaks down each cost component, and highlights three hidden cost items that are often underestimated in domestic compute infrastructure builds.
1. Server Hardware Costs: Compute Hardware Accounts for Only 45–55% of TCO; Storage Acceleration Reduces Per-GPU Amortization
Taking a 1,000-GPU scale as an example, using mainstream 8-GPU servers (each with 2 CPUs, 1TB memory, system drives, and NICs), the reference price per server is approximately ¥250,000–350,000 (excluding GPUs). For 1,000 GPUs, 125 servers are needed, bringing server hardware costs to roughly ¥31–44 million. If GPUs are priced at ¥150,000–200,000 per card (H100-class), GPU procurement costs reach ¥150–200 million. Combined, servers plus GPUs total approximately ¥180–240 million, representing 45–55% of 5-year TCO.
The leverage effect of storage acceleration on server costs is often overlooked. Taking the Mingxin FX100 as an example, its fully configured reference price is ¥371,200 (approximately ¥2,014/TB), with a single 100Gb interface and 16M IOPS. It can replace traditional centralized storage (such as all-flash arrays) or significantly reduce the number of NFS servers. In measured tests on the Huawei Atlas 910B platform, the FX100 reduced DeepSeek-70B model loading time from 1,399s to 150s (9.3×) [Source: measured, report R9 (Ascend platform)]. This means that in a 1,000-GPU cluster, GPU idle time caused by rolling model updates and cold starts can be compressed by nearly an order of magnitude, equivalently improving effective GPU utilization by approximately 3–5%, corresponding to GPU procurement savings of ¥6–10 million.
2. Networking and Storage: TCO Advantages of NVMe-oF Over Centralized Storage
Networking costs account for approximately 8–12% of a 1,000-GPU cluster, including Spine-Leaf switches, optical modules, and cabling. For 1,000 GPUs using 400G RoCEv2 networking, equipment and cabling costs are approximately ¥8–12 million. On the storage side, the traditional approach involves deploying standalone all-flash arrays (e.g., 1PB usable capacity), with hardware plus 3-year maintenance costing approximately ¥6–9 million.
The Mingxin FX series adopts an NVMe-oF (NVMe over Fabrics) architecture, mounting storage directly to GPU servers and eliminating storage gateways and dedicated storage switches. Taking the FX200 (PCIe 4.0, single 200Gb interface, 32M IOPS, fully configured reference price ¥331,200, approximately ¥1,797/TB) as an example, a 1PB usable capacity configuration requires approximately 40 units, with hardware costs around ¥13.24 million. While this is higher than the initial procurement of traditional arrays, considering the throughput gains from KV cache acceleration measured with the FX200 (+40% at 16 concurrent sessions for a 480B model) [Source: measured, report R2], the equivalent GPU time value saved exceeds ¥20 million over a 3-year cycle. Combining networking and storage, the 5-year TCO of the NVMe-oF solution is 15–20% lower than traditional centralized storage.
3. Power and Cooling: 5-Year Electricity Costs Are 1.3–1.6× Server Procurement; Every 0.1 Reduction in PUE Saves Millions
Power estimation for a 1,000-GPU cluster: at 700W per GPU card (H100-class), plus other server components, each rack consumes approximately 40–50kW. For 1,000 GPUs, approximately 25–30 high-density racks are needed, with total IT power draw of approximately 1.2–1.5MW. At a PUE of 1.3, total input power is approximately 1.6–2.0MW. At an electricity rate of ¥0.6/kWh and 8,760 operating hours per year, annual electricity costs are approximately ¥8.4–10.5 million, and 5-year electricity costs reach ¥42–52.5 million—1.3–1.6 times the server procurement cost (approximately ¥31–44 million).
Cooling methods significantly impact TCO: air cooling (PUE 1.3–1.4) has lower initial investment but higher electricity costs; liquid cooling (PUE below 1.1) increases initial investment by approximately 15–20% but saves ¥6–9 million in electricity over 5 years. Additionally, power redundancy (2N UPS and diesel generators) accounts for approximately 10–15% of infrastructure costs. If a direct utility feed plus battery backup solution is adopted, initial power infrastructure investment can be reduced by approximately 30%, though local grid stability must be assessed.
4. Facility and Operations: Hidden Costs Account for 10–15% of TCO; Storage Acceleration Reduces O&M Labor
Facility construction and fit-out (including fire protection, monitoring, and structured cabling), based on 1,000 square meters at ¥8,000–12,000 per square meter, totals approximately ¥8–12 million, representing 4–6% of TCO. Operations and maintenance costs (labor, spare parts, software licenses) are approximately ¥3–5 million per year, or ¥15–25 million over 5 years, representing 7–10% of TCO.
Storage acceleration contributes indirectly to O&M by reducing fault recovery time. Traditional NFS storage often becomes a bottleneck during model loading or Checkpoint saving, leading to task queuing and manual intervention. In measured tests with the Mingxin FX100, Checkpoint saving for 8-GPU 32B LoRA training was reduced from 178s to 94s (1.9×) [Source: measured, report R1]. While each individual saving is under 2 minutes, in frequent checkpoint-resume training scenarios, this reduces manual O&M intervention by approximately 30%. At an O&M labor cost of ¥500,000 per person-year, a 1,000-GPU cluster typically requires a team of 6–8 personnel. Storage acceleration can save 0.5–1 person-year, corresponding to ¥250,000–500,000 per year.
5. TCO Summary and Optimization Priorities
| Cost Item | 5-Year TCO (¥10,000) | Share | Optimization Headroom |
|---|---|---|---|
| GPU + Servers | 18,000–24,000 | 45–55% | Storage acceleration improves utilization by 3–5% |
| Networking + Storage | 2,000–3,200 | 5–8% | NVMe-oF replaces centralized storage |
| Power + Cooling | 5,000–6,500 | 12–16% | Liquid cooling + direct utility feed |
| Facility Construction | 800–1,200 | 2–3% | Modular construction |
| Operations & Maintenance | 1,500–2,500 | 4–6% | Automation + storage acceleration reduces labor |
| Total | 27,300–37,400 | 100% | — |
Optimization priority recommendations: First, improve GPU utilization (storage acceleration is the most direct means); second, reduce PUE (liquid cooling retrofits); third, simplify network architecture (NVMe-oF replacing storage gateways). Combined, these three measures can reduce 5-year TCO by 15–25%.
Key Q&A
Q: Which cost item accounts for the largest share of TCO in a 1,000-GPU compute center? A: GPU and server procurement combined account for 45–55%, but 5-year electricity costs (power + cooling) are 1.3–1.6 times server procurement, making it the second-largest cost item. Storage acceleration indirectly reduces the equivalent GPU procurement cost by improving GPU utilization.
Q: Where does the TCO advantage of the NVMe-oF storage solution lie compared to traditional centralized storage? A: Initial hardware investment is slightly higher (FX200 at approximately ¥1,797/TB), but storage gateways and dedicated switches are eliminated, and measured KV cache acceleration improves throughput by 29–40% [Source: measured, report R2]. The equivalent GPU time value saved exceeds ¥20 million over a 3-year cycle, with a combined 5-year TCO that is 15–20% lower.
Q: Which hidden costs in compute infrastructure builds are easily underestimated? A: Power redundancy and cooling account for 25–30% of infrastructure costs; O&M labor represents 7–10% of TCO over 5 years; and GPU idle time caused by model loading/Checkpoint operations is also significant. Measured results from the Mingxin FX series show model loading acceleration of 6.2–9.3× [Source: measured, report R9 (Ascend platform)], directly compressing GPU idle windows.
Mingxin Technology provides full-industry-chain services for compute centers, supporting approximately 10-week gated joint testing (from G1 arrival acceptance to G4 stability verification), with stop-loss measures if targets are not met. The cost model can be reproduced in Python after NDA execution. Organizations with compute infrastructure plans are welcome to contact us for joint testing collaboration.