Particle.news

Inkling 2 Mini Brings 26B Open Weights to Laptops but Windows Audio Path Sets Live‑Overlay Latency Floor

Local, looped‑transformer Inkling 2 Mini lets developers run suggestion and short‑translation polish on consumer laptops without a network hop.

Overview

  • Thinking Machines released Inkling 2 Mini as an open‑weight 26.4 billion‑parameter mixture‑of‑experts model with a roughly 20 GB mixed‑precision checkpoint and Apache 2.0 licensing that runs on Apple Silicon with 32 GB+, single DGX Spark nodes, or discrete GPUs and is available on Hugging Face and via Ollama and Tinker.
  • The model uses a 24‑layer, looped transformer design that replays its middle 12 layers three times so each token gets 48 layer applications which raises its effective per‑token compute while keeping the stored weights small.
  • Inkling 2 Mini supports text and images, a one‑million‑token context, and company‑reported strong agentic coding scores, and its inference cost is now small enough to make on‑device suggestion, short translation polish, and reply drafting practical.
  • On Windows the dominant latency and correctness limits for live bilingual overlays remain the shared‑mode audio engine and the need for separate mic and loopback tracks, so developers must use dual IAudioClient instances with AUDCLNT_STREAMFLAGS_LOOPBACK, non‑blocking callbacks, and lock‑free rings to preserve timing and speaker identity.
  • Measured end‑to‑speech‑to‑visible‑text budgets on well‑tuned machines typically sit between 150 and 400 ms, so the model lowers network dependence and improves privacy but developers must still optimize the audio capture path, STT placement, and system settings to achieve responsive overlays.