llama.cpp

Ollama

ollama/ollama

Popular

Easiest way to pull and run open models locally behind an OpenAI-compatible API; the default local runtime.

LM Studio

lmstudio.ai

Popular

Desktop app for discovering, downloading and running local models with a chat UI and local API server.

MLX

ml-explore/mlx

Rising

Apple's array framework for machine learning on Apple Silicon, with mlx-lm for fast local LLMs.

Jan

janhq/jan

Open-source, offline-first ChatGPT alternative desktop app running local models via llama.cpp.

LocalAI

mudler/LocalAI

Self-hosted, OpenAI-compatible API for running LLMs, image and audio models on consumer hardware.

llamafile

mozilla-ai/llamafile

Mozilla project that packages a model and llama.cpp into a single cross-platform executable.

KoboldCpp

LostRuins/koboldcpp

Single-file llama.cpp-based runner with a web UI, popular for local storytelling and roleplay.

LiteRT

google-ai-edge/LiteRT

Popular

Google's on-device ML runtime formerly called TensorFlow Lite, for Android, iOS, embedded Linux and accelerators.

TensorFlow Lite Micro

tensorflow/tflite-micro

Popular

LiteRT for Microcontrollers: tiny interpreter running quantized models in kilobytes of RAM; the classic TinyML runtime.