ollama Web
https://ollama.com/
Ollama local models
How to use it
Install, then `ollama run llama3.1` to chat with models entirely offline; exposes a local API on port 11434.
Open resource →
ollama GitHub
https://github.com/ollama/ollama/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
lmstudio Web
https://lmstudio.ai/
LM Studio
How to use it
Desktop app to download and chat with local models via a friendly UI and local server — no terminal needed.
open-webui GitHub
https://github.com/open-webui/open-webui/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
anything-llm GitHub
https://github.com/Mintplex-Labs/anything-llm/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
anythingllm Web
https://anythingllm.com/
AnythingLLM
How to use it
Self-hosted RAG app: upload documents and chat with them using local or API models.
LocalAI GitHub
https://github.com/mudler/LocalAI/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
vllm GitHub
https://github.com/vllm-project/vllm/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
llama.cpp GitHub
https://github.com/ggml-org/llama.cpp/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
gpt4all GitHub
https://github.com/nomic-ai/gpt4all/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
nomic Web
https://www.nomic.ai/gpt4all/
GPT4All
How to use it
One-click desktop app to run local models and chat with your documents privately.
jan Web
https://jan.ai/
Jan local AI
How to use it
Open-source offline ChatGPT alternative — download models and chat fully locally.
jan GitHub
https://github.com/janhq/jan/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
textgen GitHub
https://github.com/oobabooga/textgen/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
sglang GitHub
https://github.com/sgl-project/sglang/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
sglang Web
https://docs.sglang.ai/
SGLang docs
How to use it
High-throughput serving framework for LLMs — deploy models behind fast OpenAI-compatible APIs.
TensorRT-LLM GitHub
https://github.com/NVIDIA/TensorRT-LLM/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
nvidia Web
https://developer.nvidia.com/tensorrt-llm/
NVIDIA TensorRT-LLM
How to use it
GPU-optimized LLM inference for NVIDIA hardware — maximum throughput serving.
mlx GitHub
https://github.com/ml-explore/mlx/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.
mlx-lm GitHub
https://github.com/ml-explore/mlx-lm/
Open-source code repository
How to use it
Open the repo page, read the README for install instructions (usually a git clone or pip install command), then run it in a terminal. Check the 'Issues' and 'Releases' tabs for updates.