Storage Selection Strategy for Inference in Domestic AI Computing Centers

The core conclusion for inference storage selection in domestic AI computing centers is that "KV Cache tiered acceleration capability" should be the primary screening criterion, rather than bandwidth or capacity alone. Under a 480B model long-context cold-restart workload, the Mingxin FX100 delivers a 29–40% inference throughput improvement from KV tiered acceleration (measured, reports R2/R3). This magnitude of difference directly determines the concurrency headroom required under the same SLA. This article provides a reproducible selection framework across four dimensions: throughput, latency, concurrency profile, and cost accounting.

Why KV Cache Tiered Acceleration Is the Primary Storage Screening Criterion

The workload characteristics of inference storage differ fundamentally from those of training storage. The training phase involves streaming, large-block sequential writes where bandwidth dominates; the inference phase involves random small-block reads of KV Cache, which are latency-sensitive. According to NVIDIA GPUDirect Storage documentation, GPU-direct storage shortens the data path by bypassing the CPU bounce buffer, but its benefits depend heavily on access patterns—this is especially critical for high-concurrency small I/O such as KV Cache.

Mingxin's measurements on an 8× AMD MI308X platform (ROCm 7.2, vLLM 0.20.1+rocm721) illustrate this point. Under a 480B model (Qwen3-Coder-480B-FP8, weights approximately 450GB) long-context cold-restart workload, the FX100 all-flash NVMe-oF array relative to a local NVMe single-disk baseline:

Compared to a no-external-storage recomputation baseline (TTFT p50 149.5s @ conc16), the FX100 reduces TTFT to 11.85s, a speedup of 8.6–20× (measured, R2). This means that in long-context scenarios, storage is no longer a hidden bottleneck in the inference pipeline—it becomes an explicitly optimizable component.

How to Evaluate Concurrency Profile and Latency Budget During Selection

Selection should not rely on single-point peak values alone. The concurrency profile of the inference workload directly determines storage pressure: the KV Cache read bandwidth demand at concurrency tier 8 versus tier 32 can differ by severalfold. According to Alibaba Cloud's official classification of GPU instance families, different workloads map to different instance types; selection should first define constraints (SLA, context length, concurrency profile), then choose hardware.

Mingxin's measurements cover the full range of concurrency tiers 8–32, with TTFT p50 improvements between 26–32% (measured, R2). This data indicates that the benefit of KV tiered acceleration remains stable across the entire concurrency range, not just at low concurrency. For domestic AI computing centers, this means that when planning concurrency capacity, the concurrency headroom can be reduced based on measured in-band gains—but the specific reduction depends on your own SLA requirements, and validation through joint testing is recommended.

Cost Accounting: How to Compare TCO Across Storage Solutions

Inference storage cost should not be evaluated by procurement unit price alone. According to public research from Epoch AI, AI compute scale and cost trends are dynamic, and fixed numbers can easily become misleading. The correct approach is to build a reproducible cost model: normalize comparisons per million tokens, per concurrency, and per QPS.

The Mingxin FX product line offers reference prices at different tiers (FX100 fully configured at approximately ¥371,200 / approximately ¥2,014/TB; FX200 at approximately ¥331,200 / approximately ¥1,797/TB; FX300 at approximately ¥924,000 / approximately ¥5,014/TB), but the selection focus should be on "unit SLA cost" rather than "unit capacity cost." For example, if the FX100's KV tiered acceleration brings TTFT reduction within the measured range (26–32%), the concurrency headroom required to meet the SLA decreases, which equivalently lowers GPU-hour costs—but this conversion must be based on measured data from your own workload and should not be extrapolated directly.

Selection Criteria for Domestic Substitution

According to the public methodology of the MLPerf Inference: Datacenter Benchmark Suite, horizontal comparisons of inference performance should be based on fixed precision and latency constraints. Domestic AI computing centers should focus on three criteria when selecting storage:

1. Adaptation maturity with domestic accelerators: The Mingxin FX100 has been validated on the Huawei Atlas 910B platform, reducing DeepSeek-32B service load time from 691s to 112s (6.2×) and DeepSeek-70B from 1399s to 150s (9.3×) (measured, R9, Ascend platform).
2. Ability to serve both training and inference: In an 8-GPU 32B LoRA training checkpoint save scenario, the FX100 reduces time from 178s to 94s (1.9×), with sustained write bandwidth improving from 3.26 to 6.40 GB/s (+96%) (measured, R1).
3. Standard ownership and ecosystem compatibility: According to the InfiniBand Trade Association's specification documentation, RoCE and InfiniBand belong to different standard systems; selection should confirm the storage array's networking protocol compatibility with the existing network architecture.

Conclusion

For inference storage selection in domestic AI computing centers, we recommend KV Cache tiered acceleration capability as the primary screening criterion, supplemented by three dimensions: concurrency profile coverage, latency budget, and a reproducible cost model. Mingxin offers approximately 10 weeks of gated joint testing (G1 arrival acceptance / G2 single-node baseline / G3 main gate: TTFT reduction ≥25%, throughput +29–40% within measured range / G4 72-hour stability), with stop-loss if targets are not met—suitable as a practical approach for selection validation. To verify the above metrics under your own workload, these can be confirmed during joint testing.

References

1. Epoch AI — https://epoch.ai/
2. MLPerf Inference: Datacenter Benchmark Suite Results — https://mlcommons.org/benchmarks/inference-datacenter/
3. NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
4. Compare GPU Instance Families for AI, HPC & Rendering - Elastic GPU Service - Alibaba Cloud — https://www.alibabacloud.com/help/en/ecs/user-guide/gpu-accelerated-compute-optimized-and-vgpu-accelerated-instance-families
5. InfiniBand Specification Frequently Asked Questions — https://www.infinibandta.org/ibta-specification/

Key Q&A

Q: What is the primary screening criterion for inference storage selection in domestic AI computing centers? A: KV Cache tiered acceleration capability. Under a 480B model long-context cold-restart workload, the Mingxin FX100 delivers a 29–40% inference throughput improvement from KV tiered acceleration (measured, R2/R3), a magnitude that directly determines the concurrency headroom required under the same SLA.

Q: How should the stability of KV Cache acceleration benefits be evaluated during selection? A: Examine concurrency range coverage. Mingxin's measurements cover concurrency tiers 8–32, with TTFT p50 improvements stable in the 26–32% range (measured, R2), indicating the benefit holds across the entire concurrency range rather than only at low concurrency.

Q: What accounting basis should be used to compare TCO across storage solutions? A: Normalize per million tokens, per concurrency, and per QPS, rather than looking only at procurement unit price. According to public research from Epoch AI, AI compute cost trends are dynamic and fixed numbers can easily become misleading; a reproducible cost model should be built and validated during joint testing.