Colibri
By JustVugg
Pure-C inference engine, Apache 2.0 and roughly 25,500 GitHub stars, that treats VRAM, RAM, and storage as one memory hierarchy: attention and embedding layers stay resident while routed Mixture-of-Experts weights stream from SSD on demand, putting models the size of Kimi K3, GLM-5.2, and DeepSeek V4 Flash within reach of a machine with no GPU.
Best for
- running frontier MoE models on consumer hardware
- inference on machines with little VRAM or no GPU at all
- trading throughput for model size on a fast SSD
- zero-dependency builds across Linux, macOS, and Windows
- Aug 19, 2026Added to the catalogue: pure-C inference engine that streams MoE experts from SSD, running Kimi K3 and GLM-5.2 class models without a GPU, ~25,500 GitHub stars.
Other Local & on-device AI
Ollama
Run open-weight LLMs locally with a single command. Bundles model weights, quantizations, and an OpenAI-compatible HTTP API into a clean CLI.
LM Studio
Desktop GUI for downloading and chatting with local LLMs. The friendly way to try open-weight models without touching a terminal.
llama.cpp
C/C++ inference engine for LLaMA-family models. The library that quietly powers most local AI apps — fast, low-level, runs on almost anything.
Jan
Open-source ChatGPT alternative that runs entirely offline. Built on llama.cpp with a clean desktop UI and an OpenAI-compatible API.
MLX
Apple's array framework for Apple Silicon. Designed to run ML workloads natively on M-series Macs with unified memory between CPU and GPU.
GPT4All
Open-source desktop app for running LLMs locally with a chat UI, document RAG, and a browsable model catalog.
Open WebUI
Self-hosted, extensible ChatGPT-style web interface for local and remote models, with offline operation and RAG.
AnythingLLM
All-in-one desktop/self-hosted app for document chat (RAG) and agents over local or cloud models.