Unsloth
By Unsloth AI
Open-source fine-tuning engine that trains LLMs about twice as fast with roughly 70 percent less VRAM and exports directly to GGUF and Ollama formats, with close to 69,000 GitHub stars.
Best for
- fine-tuning open models on a single GPU
- QLoRA and LoRA training with limited VRAM
- no-code dataset creation and training runs
- exporting tuned models to GGUF or Ollama
- running and comparing models fully offline
- Jul 22, 2026Added — open-source fine-tuning engine, roughly 2x faster training with 70% less VRAM, close to 69,000 GitHub stars.
Other Local & on-device AI
Ollama
Run open-weight LLMs locally with a single command. Bundles model weights, quantizations, and an OpenAI-compatible HTTP API into a clean CLI.
LM Studio
Desktop GUI for downloading and chatting with local LLMs. The friendly way to try open-weight models without touching a terminal.
llama.cpp
C/C++ inference engine for LLaMA-family models. The library that quietly powers most local AI apps — fast, low-level, runs on almost anything.
Jan
Open-source ChatGPT alternative that runs entirely offline. Built on llama.cpp with a clean desktop UI and an OpenAI-compatible API.
MLX
Apple's array framework for Apple Silicon. Designed to run ML workloads natively on M-series Macs with unified memory between CPU and GPU.
GPT4All
Open-source desktop app for running LLMs locally with a chat UI, document RAG, and a browsable model catalog.
Open WebUI
Self-hosted, extensible ChatGPT-style web interface for local and remote models, with offline operation and RAG.
AnythingLLM
All-in-one desktop/self-hosted app for document chat (RAG) and agents over local or cloud models.