AI & automation
Ollama
Runs open-weight LLMs locally instead of calling a hosted API
What it does
Ollama downloads and serves open-weight language models, giving an OpenAI-compatible API for running inference without sending prompts to an external provider.
How it runs here
Runs in the ollama namespace with model weights stored on a 20Gi Longhorn volume, exposing its API on the standard Ollama port for in-cluster clients.
Connects to
Stands on its own.
Why I chose it
Local inference keeps prompts on owned hardware with no per-token cost; on CPU-only Pi nodes that means a small model such as a 1.7B Qwen running at single-digit tokens per second.