Ollama

Open-source local-LLM runtime that lets developers run hundreds of language models — including DeepSeek-R1, Llama 3.1, Gemma 4, Qwen 3, Kimi K2.5, GLM-5, MiniMax, and gpt-oss — directly on local hardware via a unified CLI and HTTP API. Architecture sits on `llama.cpp` (with GGUF model format) for general inference and uses Apple's MLX framework to accelerate on Apple Silicon. Ships as a single binary with no daemon required and exposes an OpenAI-compatible API surface for drop-in integration with existing tooling.

Closes the "every chat costs per-token" gap by moving inference to commodity hardware while keeping data on the operator's machine. Solves the privacy, cost, and latency tax that comes with routing all LLM traffic through hosted APIs. The 2026 framing has shifted further — Ollama now anchors the local end of hybrid deployment pipelines where the same agent runs against a hosted frontier model OR a local Ollama instance depending on the task's privacy profile.

Private RAG pipelines over S3-stored corpora (the model runs local while embeddings/documents live on S3-compatible storage), agentic workflows where every tool call can route to local models (e.g. via the March 2026 GitHub Copilot Ollama integration), on-prem deployments where data residency rules forbid cloud inference, developer workstation prototyping before promoting to a paid frontier API, and offline edge inference where network connectivity is intermittent.