MoE model vLLM

How to Run Kolibri-1 Locally: Aleph Alpha's German-English MoE Model

Kolibri-1 is Aleph Alpha's 78B MoE model with 3.46B active params. Here's what hardware you need and how to serve it with vLLM.