Overview
- Xiaomi announced HySparse 2 and said MiMo‑V3 will adopt it, positioning the release as a shift from model drops to architecture work focused on long contexts and agentic inference.
- The company reports that at one million tokens HySparse 2 reduces prefill FLOPs to about one fifth of the prior approach and shrinks KV cache size to roughly one quarter.
- Key techniques include KV Bridging, which builds cross-decoder key/value from self-decoder states, KV Reuse that shares cache inside mixed blocks, token-level selection to pick important tokens, and a recent-token forced window to let local and global tokens share the same KV cache.
- Xiaomi says HySparse 2 raises retrieval and reasoning scores (MRCRv2 and RULER‑v2) while lowering metrics for agent and long‑context perplexity (AgentPPL and LongPPL), and the paper is available on arXiv for verification.
- Xiaomi released the open‑source MiMo‑V2.6 Pro and Flash models immediately before this announcement and framed those models as practical steps toward exploring recursive self‑improvement by scaling reinforcement‑learning compute, a change that could lower compute costs for long‑context, interactive AI applications.