Overview
- The companies announced the jointly validated KV‑cache platform on Nov. 5–6, 2026 and published reference benchmarks plus an IBM Redbook that detail the architecture and test results.
- The integrated system uses Supermicro Petascale storage, IBM Storage Scale software, NVIDIA Dynamo inference software and NVIDIA Spectrum‑X networking to store and serve model key‑value state from shared flash.
- Company‑validated tests report large gains for long prompts, including up to 56x faster time‑to‑first‑token at 130,000 tokens, a 22x rise in concurrent requests per second, and a 95% cut in total time for a 200‑request batch.
- Validation included noisy‑neighbor and high‑concurrency scenarios to measure resilience and consistency, and Supermicro says its factory rack integration and on‑site services can shorten deployment from weeks to days.
- The platform builds on NVIDIA’s ICMSP work and the partners’ cooperation since mid‑2025; it aims to lower GPU recomputation costs for RAG, multi‑turn agents and other long‑context enterprise AI while availability, pricing and region rollout remain undisclosed.