Overview
- Aleph Alpha published Kolibri, an open-weight large language model that the company says uses 78.1 billion parameters with a Mixture-of-Experts design that activates about 3.46 billion parameters per token to reduce compute needs.
- The model was trained with roughly 23 percent German-language pretraining data and a tokenizer tuned for German so that German text is split into fewer tokens and processed more cheaply and efficiently.
- Kolibri supports long contexts, logical reasoning, programmatic integrations and on-premise deployment so that governments and regulated firms can keep sensitive data on local infrastructure.
- Aleph Alpha has agreed to combine with Canadian firm Cohere and the product rollout comes as the startup has already restructured its leadership, cut about 50 jobs and ended non-core partnerships to focus on core products.
- The open weights and a Model Card are available on Hugging Face under an Apache-2.0-style license, and the pending Cohere deal could reshape customer relationships, product roadmaps and where engineering work is centered, with Heidelberg kept as a tech hub.