A Measured Perspective on Inference Storage Energy Consumption: Ascend 910B vs. NVIDIA GPU

In discussions of energy consumption in inference storage systems, comparisons between the Ascend 910B platform and NVIDIA GPUs are often reduced to a contest over chip energy efficiency ratios. But according to measured data from Xinyuanyi's own test platform, the storage pathway's impact on total system energy consumption is just as critical as the chip itself—FX100 achieves 6.2–9.3× model loading acceleration on the Huawei Atlas 910B platform (measured, report R9), and 29–40% KV tiering throughput improvement on the AMD MI308X platform (measured, report R2/R3). The storage energy mechanisms behind these numbers are the real variables decision-makers should focus on. This article starts from architectural differences and measured data to lay out the genuine divide between the two platform types in inference storage energy consumption.

Architectural Roots of the Storage Energy Difference Between Ascend 910B and NVIDIA GPU

Storage energy consumption in an inference system is not a single metric; it is jointly determined by data pathway efficiency, VRAM capacity strategy, and memory access patterns. According to NVIDIA GPUDirect Storage Documentation, GPU direct-attached storage bypasses the CPU bounce buffer, letting data transfer directly between storage and VRAM, reducing the extra power draw from memory copies. The Ascend 910B platform likewise has direct-attach-like capabilities, but the two vendors differ in the implementation depth of their storage protocol stacks—NVIDIA deeply couples GDS with its CUDA ecosystem, whereas Ascend's storage acceleration relies more on upper-layer framework adaptation.

According to the FlashAttention paper (arXiv:2205.14135), the bottleneck in attention computation is fundamentally HBM bandwidth rather than compute throughput, and this conclusion has direct implications for energy analysis: when insufficient KV Cache hit rates cause repeated disk reads, storage bandwidth consumption translates into GPU idle-wait power rather than effective compute power. The 480B model long-context cold-recovery scenario recorded in Xinyuanyi's R2 measurement is precisely a quantified manifestation of this mechanism—the recompute baseline TTFT p50 reached as high as 149.5s (concurrency level 16), while after integrating FX100 it dropped to 11.85s (measured, report R2), and the share of GPU ineffective power during the wait dropped markedly as well.

How Measured Data Changes the Discussion Framework for Energy Comparisons

Cross-platform energy comparisons easily fall into the trap of "numbers that were never measured." Xinyuanyi completed measurements separately on the Ascend 910B platform and the AMD MI308X platform, but the two test sets used different workload shapes and cannot be directly converted into a cross-platform "performance per watt" comparison. What can be confirmed is that storage acceleration delivers significant gains on both platform types:

The table above shows that storage acceleration on the Ascend platform manifests as a 6.2–9.3× reduction in loading time (measured, report R9), while on the AMD platform it manifests as a 29–40% inference throughput improvement (measured, report R2/R3). Although the two cannot be directly compared, they point to the same conclusion: the storage system is an important lever for energy optimization on both platform types. According to public research from Epoch AI, among the total energy growth brought by AI compute scaling, the share attributable to storage and data movement is trending upward—this third-party research framing suggests that inference storage energy optimization should not be excluded from selection considerations.

General Criteria for Inference Storage Energy Optimization

For decision-makers evaluating both the Ascend 910B and NVIDIA GPU platforms, energy comparison should rest on three verifiable dimensions:

First, data pathway efficiency. According to NVIDIA GPUDirect Storage Documentation, direct-attached storage reduces the number of CPU interventions, and each bypass of the bounce buffer means saving the power of one memory read/write. Although the Ascend platform has no fully corresponding public documentation, the magnitude of 6.2–9.3× model loading acceleration in the R9 measurement (measured, report R9) indirectly indicates that its storage pathway has equivalent room for optimization in reducing CPU-mediated data movement.

Second, VRAM capacity strategy and KV Cache hit rate. According to the PagedAttention paper (arXiv:2309.06180), KV Cache paging management increases effective capacity by reducing VRAM fragmentation, and according to the Mooncake paper (arXiv:2407.00079), a KVCache-centric disaggregated architecture can reduce redundant computation through prefix cache reuse. Xinyuanyi FX100's KV tiering acceleration externalizes exactly these mechanisms to the storage layer—in the R1 measurement, the LMCache parallel read patch reduced TTFT at single-card concurrency 16 from 37.97s to 9.30s (a 4.1× improvement, with bandwidth increased 5.3×) (measured, report R1). When the KV Cache hit rate rises, the GPU shifts from "waiting on disk reads" to "continuous computation," and energy per token naturally declines.

Third, energy opportunities in training checkpoint save and restore. In training scenarios, I/O waits during checkpoint saving likewise generate ineffective power draw. The R1 measurement shows that in 8-GPU 32B LoRA training, full-model snapshot saving dropped from 178s to 94s (1.9× acceleration, sustained write bandwidth 3.26→6.40 GB/s) (measured, report R1). On both Ascend 910B and NVIDIA platforms, this acceleration magnitude means the GPU is released from save-wait and can return to a compute state faster.

Conclusion

The comparison between Ascend 910B and NVIDIA GPU in inference storage energy consumption should not be reduced to a contest over chip energy efficiency ratios. Xinyuanyi's measured data on both platform types shows that the storage pathway's impact on total system energy consumption is clear and quantifiable—6.2–9.3× model loading acceleration on the Ascend platform (measured, report R9), and 29–40% KV tiering throughput improvement on the AMD platform (measured, report R2/R3). For decision-makers evaluating both platform types, we recommend incorporating storage energy into the selection matrix and basing judgments on measured in-band data rather than single-chip specifications. Xinyuanyi offers an approximately 10-week gated joint test (including in-band validation of TTFT reduction ≥25% and throughput +29–40%), enabling evaluation of storage acceleration's impact on energy consumption under real workloads.

Key Questions and Answers

Q: Can the difference between Ascend 910B and NVIDIA GPU in inference storage energy consumption be directly quantified and compared? A: No. The workload shapes Xinyuanyi measured on the two platform types differ; the Ascend platform validated 6.2–9.3× model loading acceleration (measured, report R9), while the AMD platform validated 29–40% KV tiering throughput improvement (measured, report R2/R3). The two cannot be converted into cross-platform performance per watt.

Q: How does storage acceleration affect the energy consumption of an inference system? A: By raising the KV Cache hit rate and improving data pathway efficiency, it reduces ineffective GPU power during disk-read waits or recomputation. In the 480B model cold-recovery scenario in the R2 measurement, after integrating FX100, TTFT dropped from 149.5s to 11.85s (measured, report R2), markedly shortening GPU wait time.

Q: How should the storage energy performance of the two platform types be evaluated during selection? A: We recommend focusing on three dimensions: whether the data pathway supports direct-attached storage, the hit-rate performance of the KV Cache tiering strategy, and the acceleration magnitude of training checkpoint saving. Xinyuanyi offers a gated joint test that can validate, under real workloads, in-band measured performance of TTFT reduction ≥25% and throughput +29–40%.

References

1. MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
2. NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
3. Epoch AI — https://epoch.ai/
4. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — https://arxiv.org/abs/2205.14135
5. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — https://arxiv.org/abs/2407.00079
6. Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180