Storage Requirements and Tiered Acceleration Solutions for AI Inference in Industrial Automation

AI inference in industrial automation has evolved from offline quality inspection to production-line-level real-time decision-making. The core storage challenge is no longer sequential bandwidth but low-latency data access under high concurrency and rapid KV Cache reconstruction. In a long-context cold-restore workload with a 480B production-grade model, the Mingxin FX100 delivered measured inference throughput gains of 29–40% (measured, report R2/R3) and reduced time-to-first-token (TTFT) by 26–32% (measured, report R2), providing a quantifiable storage-side solution to this challenge.

Why Storage Requirements for Industrial AI Inference Differ from Data Centers

AI inference workloads in industrial automation—such as visual quality inspection, predictive equipment maintenance, and flexible assembly path planning—exhibit three characteristics distinct from general cloud inference. First, task fragmentation: the input for a single inference (e.g., an industrial camera image, a vibration waveform) is typically KB to MB in size, but concurrency is extremely high—dozens of cameras triggering inference simultaneously on a single production line is common. Second, latency sensitivity: quality inspection results must be returned before downstream robotic arm actions, with end-to-end latency budgets often in the hundreds of milliseconds, leaving even tighter budgets for the storage system. Third, polarized model scales: edge-side deployments commonly use 7B–14B small models, while factory-level central scheduling or digital twin platforms deploy 70B+ large models, placing entirely different stress profiles on storage.

According to SNIA (Storage Networking Industry Association) definitions for tiered storage and computational storage, storage system performance evaluation must consider not only peak bandwidth but also access latency and concurrent processing capability under specific workloads. Storage requirements for industrial AI inference can therefore be decomposed into three dimensions: fast loading of model weights and checkpoints, efficient read/write of KV Cache during inference, and latency stability under multi-path concurrency.

Why KV Cache Access Becomes the Storage Bottleneck in Industrial Inference

Attention computation in large-model inference is limited by HBM bandwidth rather than compute, as explicitly noted in the FlashAttention work (arXiv:2205.14135). As sequence length grows, KV Cache consumes more GPU memory; once it exceeds GPU memory capacity, it must spill to the storage system. At that point, every cache miss means reading hundreds of MB or even GB of data from storage, and storage latency is added directly to TTFT.

In industrial scenarios, this bottleneck is amplified. Production-line inference often reuses the same batch of historical context (e.g., equipment state sequences, process parameter trajectories), so prefix cache hit rates are theoretically high. However, once the cache is evicted or must be shared across nodes, the cost of cold reads becomes unacceptable. The Mooncake architecture work (arXiv:2407.00079) identifies a KVCache-centric disaggregated storage-compute architecture as the direction to address this, but its premise is that the storage side can provide sufficiently low latency and high concurrent read bandwidth.

Mingxin's measured R2 results provide specific quantification: with a 480B model, TP8 deployment, and three concurrency levels, TTFT p50 dropped from 10.17–35.73 seconds to 7.53–26.35 seconds (measured, report R2). This reduction means that in industrial scenarios, cold-start inference that previously required tens of seconds to produce the first token is compressed into an acceptable interactive range.

Tiered Acceleration: The Latency Economics from Local NVMe to NVMe-oF

Selecting a storage solution for industrial AI inference is fundamentally a trade-off between latency and cost. The baseline approach (local NVMe single drive) offers the lowest latency but limited capacity and no sharing; traditional NFS provides ample capacity but suffers from high latency due to network protocol stack overhead. NVIDIA's GPUDirect Storage documentation describes a data path for GPU-direct storage that bypasses the CPU bounce buffer to reduce data movement latency, pointing a direction for storage-side optimization.

Mingxin FX100 measured data demonstrates a third path: connecting all-flash arrays directly to GPU servers via NVMe-oF (NVMe over Fabric) with RDMA, achieving capacity elasticity of shared storage while maintaining latency close to local NVMe. Measured R9 results (Ascend 910B platform) show model inference loading accelerated 6.2–9.3× relative to the NFS baseline: DeepSeek-32B service loading dropped from 691 seconds to 112 seconds, and DeepSeek-70B from 1399 seconds to 150 seconds (measured, report R9).

For KV Cache cold-read scenarios, the LMCache parallel read patch measured in R1 shows significant effect: under single-GPU, concurrency 16, cold disk read (Qwen2.5-32B), TTFT dropped from 37.97 seconds to 9.30 seconds, and read bandwidth increased from 0.98 GB/s to 5.23 GB/s (measured, report R1). This magnitude of improvement turns the "line change equals cold start" pain point in industrial scenarios from unusable to acceptable.

Solution Selection: Matching Storage Tiers by Concurrency Level and Model Scale

Based on multiple Mingxin measured reports, storage solutions for industrial AI inference can be selected using the following logic:

The key demarcation point in the table above: when concurrency exceeds 8 paths and model size exceeds 70B, KV Cache spillover access becomes the dominant latency factor. At this point, the storage system's random read latency and concurrent throughput directly determine inference service quality. Mingxin FX100 measured throughput gains of 35–36% (measured, report R3) on a 480B model with TP4×2 full-machine configuration, representing the quantified benefit in this scenario.

For extreme scenarios without external recomputation (i.e., relying solely on GPU memory and local disks with no external storage), the R2 measured control data provides additional reference: the recomputation baseline TTFT p50 was 149.5 seconds (concurrency 16), while the FX100 solution achieved 11.85 seconds, with throughput improving from 4.1 tok/s to 74.9 tok/s (measured, report R2). This comparison demonstrates that storage-side optimization can yield order-of-magnitude inference performance improvements under specific workloads.

Conclusion

Storage selection for industrial automation AI inference should not follow the data center's "high bandwidth" mindset. Instead, it should be designed in tiers around KV Cache access latency and concurrent throughput. The Mingxin FX product line spans PCIe 3.0 to PCIe 6.0 (FX100 to FX400), offering bandwidth options from single-interface 100Gb to 400Gb, and supports an approximately 10-week gate-based joint testing collaboration model (G1 arrival acceptance through G4 stability verification), enabling industrial customers to validate benefits under actual workloads before deployment. Teams with production-line-level AI inference storage needs are welcome to contact us for joint testing.

Key Q&A

Q: What is the fundamental difference between storage requirements for industrial AI inference and data centers? A: Industrial scenarios are dominated by fragmented small requests, high concurrency, and hundred-millisecond latency budgets. The storage bottleneck lies not in sequential bandwidth but in random read latency and concurrent processing capability, with polarized model scales (edge 7B–14B, factory-level 70B+).

Q: What are the measured benefits of Mingxin FX100 in KV Cache acceleration? A: In a 480B model long-context cold-restore workload, inference throughput improved by 29–40% (measured, report R2/R3), TTFT decreased by 26–32% (measured, report R2); relative to the no-external-storage recomputation baseline, throughput increased from 4.1 tok/s to 74.9 tok/s (measured, report R2).

Q: How can industrial customers verify whether a storage solution fits their workloads? A: Mingxin offers an approximately 10-week gate-based joint testing process (G1 arrival acceptance / G2 single-node baseline / G3 main gate: TTFT reduction ≥25%, throughput +29–40% measured in-band / G4 72-hour stability), with stop-loss if targets are not met. The measurement model is reproducible in Python after NDA.

References

1. NVIDIA GPUDirect Storage Documentation — https://docs.nvidia.com/gpudirect-storage/index.html
2. SNIA — Storage Networking Industry Association — https://www.snia.org/
3. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — https://arxiv.org/abs/2205.14135
4. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — https://arxiv.org/abs/2407.00079
5. Efficient Memory Management for Large Language Model Serving with PagedAttention — https://arxiv.org/abs/2309.06180