Overview
- OpenAI unveiled a limited‑preview Decisions API on September 30 that asks models to pick from predefined options instead of generating free text and that the company says keeps image understanding and safety protections.
- TypeSafe’s Jev, released earlier this month, returns typed choices with probabilities and confidence scores and advertises end‑to‑end latencies of 70–500 milliseconds and input‑priced billing around $0.042 per million tokens.
- Early integrations and user reports show large speed and cost gains for classification and agent tasks, with median speedups of about 7× and examples such as a hackathon demo estimating monitoring costs of $2.94 with Jev versus $372 with a frontier LLM.
- TypeSafe’s own documentation warns of specific failure modes such as unreliable counting, arithmetic and date comparisons and recommends pinning versions, keeping math in application code, and logging full probability distributions.
- The announcements have triggered a fast industry shift with startups and platforms building similar models, prompting engineers to use shadow testing, batching, human fallbacks and probability logging while researchers test real‑world calibration and adversarial robustness.