AI Accelerator Selection: Interface and Bandwidth

When selecting storage for domestic AI clusters, the matching between storage interface generations (PCIe 3.0/4.0/5.0/6.0) and compute accelerator bandwidth is a key variable determining inference throughput, training efficiency, and total cost of ownership (TCO). Based on measured data from Mingxin FX100 on AMD ROCm and Huawei Ascend platforms, KV Cache tiered acceleration improves 480B model inference throughput by 29–40% (measured, report R2/R3), while model loading time is compressed by 6.2–9.3x compared to the NFS baseline (measured, report R9). The core driver of these results is not simply flash media speed, but the alignment of bandwidth between the NVMe-oF protocol and the compute accelerator interface generation, eliminating I/O bottlenecks. This article analyzes the selection logic from this perspective, helping technical decision-makers optimize domestic AI investments from a storage dimension.

Why Storage Interface Generation Matters More Than Flash Media Speed?

In domestic AI scenarios, a common misconception is to focus only on the sequential read/write bandwidth of flash media (e.g., 7 GB/s for a single PCIe 4.0 NVMe drive) while ignoring the interface generation matching between the compute accelerator and the storage system. In actual workloads, the core bottlenecks for inference and training often appear in random reads of KV Cache, continuous streaming transfers during model loading, and write latency during checkpointing—these operations are limited by PCIe link bandwidth and NVMe-oF protocol efficiency, not the raw media limits of a single drive.

Taking the Mingxin FX100 (PCIe 3.0×4, single interface 100Gb) as an example, its design logic is: on an 8-card AMD Instinct MI308X platform (ROCm 7.2), it provides sustained bandwidth through an NVMe-oF array (4-disk RAID0, 14 TB), matching the 100 GbE network. Measurements show that in a long-context cold recovery workload for a 480B model, the FX100 reduces TTFT by 26–32% compared to a local single NVMe drive baseline (measured, report R2). This is not because the flash is faster, but because the distributed architecture of NVMe-oF avoids saturation of a single drive's I/O queue depth—with 16 concurrent requests on a local single drive, TTFT p50 reaches 35.73s, while the FX100 reduces it to 26.35s. Although the interface generation (PCIe 3.0) is not the latest, protocol optimizations (RoCEv2 and LMCache parallel read patch) achieve a 4.1x improvement in TTFT (measured, report R1).

For domestic GPU platforms (e.g., Ascend 910B), matching storage interface generations is even more critical. In the R9 test, the FX100 compressed DeepSeek-70B model loading time from 1399s (NFS baseline) to 150s (9.3x acceleration) via NVMe-oF. The core reason is avoiding the TCP/IP protocol stack overhead of NFS by directly utilizing RDMA transport. This suggests to selectors: in Ascend or ROCm clusters, storage systems should prioritize RDMA support (e.g., RoCEv2) over pursuing flash media PCIe generation upgrades.

Storage Bandwidth Matching Principles for Domestic GPU Platforms (ROCm, Ascend)

The memory bandwidth and compute capability of domestic AI accelerators (e.g., AMD MI308X, Huawei Ascend 910B) are rapidly improving, but storage I/O often becomes an overlooked bottleneck. Based on measured data from the Mingxin FX product line, the following matching principles can be summarized:

1. Inference Scenarios: Aligning KV Cache Bandwidth Requirements with Interface Generation

In inference for a 480B MoE model, the read bandwidth requirement for KV Cache grows linearly with concurrency. R2 tests show that under TP8 configuration with 16 concurrent requests, TTFT p50 drops from 10.17s (local NVMe) to 7.53s (FX100), corresponding to a 40% throughput improvement. This requires the storage interface bandwidth to be no less than the compute accelerator's PCIe link capability—for example, the MI308X's PCIe 5.0×16 interface has a theoretical bandwidth of approximately 32 GB/s, but in actual inference workloads, the random read pattern of KV Cache results in an effective bandwidth of only 0.98 GB/s (local single drive baseline), while the FX100, with its parallel read patch, increases this to 5.23 GB/s (5.3x, measured, report R1). Therefore, selection should focus on the storage system's IOPS and random read latency, not sequential bandwidth.

2. Training Scenarios: Bandwidth Matching for Checkpoint Writes

Training checkpoint saving is a typical write-intensive workload. In the R1 test, for an 8-card 32B LoRA training task, the write time for each 65.6 GB full model snapshot dropped from 178s (local NVMe) to 94s (FX100), with sustained write bandwidth improving by 96% (3.26→6.40 GB/s). This benefits from the distributed write capability of NVMe-oF, avoiding write amplification on a single drive. For Ascend 910B clusters, if NFS is used in similar workloads, write latency may increase by 2–3x due to the TCP/IP protocol stack, so storage solutions supporting RDMA are recommended.

3. Interface Generation Selection: Practical Boundaries from PCIe 3.0 to 6.0

The Mingxin FX product line covers PCIe 3.0 to 6.0 (FX100 to FX400), but higher generations are not always better. The FX100 (PCIe 3.0) can already support inference acceleration for 480B models under a 100 GbE network, while the FX400 (PCIe 6.0, 140M IOPS) targets mass production needs by the end of 2026. The interface generation of current domestic AI accelerators (e.g., MI308X is PCIe 5.0, Ascend 910B is PCIe 4.0) determines the matching upper limit for the storage system—if the compute accelerator only supports PCIe 4.0, deploying PCIe 5.0 storage may waste bandwidth. Selection recommendation: Use the compute accelerator's interface generation as a baseline and choose a storage solution with matching bandwidth to avoid over-investment.

Impact of Storage Selection on TCO for Domestic AI Clusters from Measured Data

In the TCO of domestic AI clusters, storage costs account for approximately 15–25% (per IDC), but storage efficiency directly affects GPU utilization. In the R3 test, the Mingxin FX100 achieved a 35–36% throughput improvement at the full-machine level with TP4×2 (measured, report R3). This means, with the same number of GPUs, inference throughput can increase by 1.35x, equivalently reducing GPU procurement costs by approximately 26%. For an 8-card MI308X cluster (single card approximately ¥200,000), the storage investment (FX100 fully configured at approximately ¥371,200) can be recovered within 6–12 months through improved GPU efficiency.

Additionally, model loading acceleration (6.2–9.3x, measured, report R9) reduces service startup and fault recovery times. In an Ascend 910B cluster, loading a 70B model dropped from 23 minutes (NFS) to 2.5 minutes (FX100), directly enhancing the cluster's elastic scaling capability and reducing operational complexity.

Conclusion

Selection of domestic AI accelerators should not focus solely on GPU compute power and memory; storage interface generation and bandwidth matching are equally important dimensions. The Mingxin FX series, through NVMe-oF protocol and tiered acceleration technology, has validated 29–40% inference throughput improvements and 6.2–9.3x loading acceleration on ROCm and Ascend platforms. We welcome compute center operators and technical teams to conduct gated joint testing, verifying a TTFT reduction of ≥25% within 10 weeks (G3 primary gate), ensuring quantifiable and reproducible investment.

Key Questions and Answers

Q: How much does storage interface generation (PCIe 3.0/4.0/5.0) affect inference performance for domestic AI accelerators?
A: In 480B model inference, the Mingxin FX100 (PCIe 3.0), through NVMe-oF protocol optimization, reduces TTFT by 26–32% (measured, report R2) and improves throughput by 29–40% (measured, report R3). Interface generation itself is not the bottleneck; protocol efficiency (e.g., RDMA) is more critical.

Q: On the Ascend 910B platform, how can storage selection optimize model loading time?
A: On the Ascend platform, the Mingxin FX100 compresses DeepSeek-70B model loading time from 1399s (NFS baseline) to 150s (9.3x, measured, report R9). The core is using RDMA transport instead of the TCP/IP protocol stack. It is recommended to prioritize NVMe-oF solutions supporting RoCEv2.

Q: How does storage investment affect TCO for domestic AI clusters?
A: In an 8-card MI308X cluster, the FX100 achieves a 35–36% throughput improvement (measured, report R3), equivalently reducing GPU procurement costs by approximately 26%. The storage investment (approximately ¥371,200) can be recovered within 6–12 months through improved GPU efficiency.