Overview
- Resp an published the gateway as a pre‑alpha open‑source project on GitHub on Tuesday, October 20, 2026, and is encouraging a plugin ecosystem while also offering a paid hosted platform with observability and unified billing.
- The rewrite grew out of a January 2026 failure when a routine LiteLLM upgrade produced intermittent connection errors that Respan rolled back and never fully diagnosed, prompting a move away from a large Python wrapper codebase.
- The gateway uses a very small Rust core that dispatches requests and runs features as plugins that handle providers, routing, realtime protocols, modalities, and security controls so teams can extend behavior without changing the core.
- Plugins are compiled into the gateway binary, which means developers must rebuild and manage those plugins in their build pipelines and accept that plugin code runs inside the gateway process with access to model requests and other sensitive data.
- Resp an says some provider integrations are tiny (the Mistral provider is 22 lines) and the gateway can translate between provider protocols, but plugin registration order and compiled dependencies create operational and security trade‑offs that teams will need to manage.