Scality has introduced its AI Inference Factory, a new architectural framework designed to streamline the deployment of on-premises artificial intelligence workloads. By integrating its Autonomous Data Infrastructure (ADI) with a disaggregated serving layer, the company aims to provide enterprises with a high-performance alternative to public cloud AI services, addressing common challenges related to data sovereignty, escalating token-based pricing, and GPU utilization efficiency.
The core of the new offering is the separation of the prefill and decode stages of the inference process. In traditional setups, these two stages often compete for the same GPU resources, leading to inefficiencies where new prompts can interrupt ongoing token generation. Scality’s architecture allows organizations to decouple these functions, assigning dedicated GPU pools to each task while utilizing a shared key-value (KV) cache pool managed by the ADI platform. This approach ensures that decode GPUs can retrieve existing context without needing to recompute it, significantly reducing latency and improving overall throughput.
Scality CTO Giorgio Regni noted that the integration allows the storage layer to operate with performance characteristics that rival high-bandwidth memory (HBM). By offloading the KV cache to the ADI storage layer, organizations can maintain much larger context windows than a single GPU’s memory would typically permit. Testing conducted by the company demonstrated that this architecture could restore a 14,000-token context in approximately 166 milliseconds, a speed that remains competitive with local HBM access while offering vastly superior scalability for concurrent sessions.
The AI Inference Factory provides a comprehensive software stack, including validated open-weight models, a control plane for authentication and metering, and the underlying ADI object storage. The platform is designed to be hardware-agnostic, running on standard enterprise-grade servers from major vendors such as Dell, HPE, Lenovo, and Supermicro. By supporting popular frameworks and agentic tools, the system allows teams to maintain control over their model versions and quantization settings, which can often be subject to change in managed cloud environments.
This release positions Scality alongside other storage vendors focusing on AI-optimized infrastructure. While competitors like Cloudian and MinIO have also developed solutions for AI data management, Scality emphasizes its specific focus on prefill-decode disaggregation as a primary differentiator for optimizing GPU utilization. The platform is currently available either as a software license or as a fully managed service, offering a path for government agencies and large enterprises to build production-ready AI environments on their own terms.
As organizations move beyond experimental AI projects into mission-critical production, the demand for predictable performance and data control is increasing. Scality’s move to provide a unified, petabyte-scale storage layer that integrates directly with the inference serving path reflects a broader industry shift toward making enterprise data infrastructure the backbone of the AI lifecycle.
