Self-Hosted Models

Voxos can use an explicitly registered local model server or a public HTTPS endpoint through one generic OpenAI compatible text provider. The server must implement the streaming POST /v1/chat/completions contract and structured tool calls. Authentication is optional; when a bearer key is entered, Voxos stores it in the encrypted user vault and sends it only to that configured endpoint.

These instructions were checked against the linked official runtime documentation on 2026-08-19.

Why One OpenAI-Compatible Provider

Ollama, LM Studio, vLLM, llama.cpp, and many hosted gateways expose an OpenAI-compatible API. A single endpoint-based integration keeps the Voxos runtime independent of the model runner and supports both local and non-local servers.

Voxos requires an explicit list of model IDs. GET /v1/models is useful for finding those IDs, but Voxos does not import it automatically because API compatibility and the models’ tool behavior vary between runtimes. A model listed by the endpoint is not necessarily capable of reliable structured tool calling.

Custom models are treated as text and tool-calling models. Voxos does not infer image input, browser-execution, or image-generation support from a model name. Use a model and server configuration whose documentation explicitly supports streaming tool calls. Test important tools before relying on scheduled or unattended runs.

Configure Voxos

1. Start the model server and load or pull a tool-capable instruct model.
2. Verify its OpenAI-compatible model list from the machine or container that runs the Voxos backend.
3. Open Providers in Voxos.
4. Find OpenAI compatible.
5. For a local or private endpoint, register the complete base URL as described in Endpoint Reachability.
6. Enter the API base URL, including its /v1 path.
7. Enter one model ID per line. Commas are also accepted.
8. Enter a bearer API key when the server requires one. Leave it empty for an endpoint that intentionally has no authentication.
9. Save the endpoint, then open Agent and select the provider, primary model, allowed subagent models, and default subagent model.

Example values for Ollama running on the same host as the Voxos Compose stack:

Base URL: http://host.docker.internal:11434/v1
Model ID: gpt-oss:20b
API key:  leave empty

The URL is resolved by the Voxos API and durable run worker, not by the web browser. localhost therefore means the backend process or its container.

Endpoint Reachability

Unregistered endpoints must use HTTPS on port 443, contain no URL credentials, query, or fragment, and resolve only to global IP addresses. Requests do not follow redirects. Local, private, link-local, and non-standard-port endpoints therefore require an exact deployment-owned registration.

For host or Compose development, add complete base URLs to the JSON array in OUTBOUND_ADDITIONAL_TRUSTED_ENDPOINTS_JSON. Each entry must use the text_provider purpose and local_container kind:

OUTBOUND_ADDITIONAL_TRUSTED_ENDPOINTS_JSON=[{"purpose":"text_provider","kind":"local_container","url":"http://host.docker.internal:11434/v1"}]

Use http://localhost:<port>/v1 instead when Voxos runs directly on the host. The saved URL must normalize to the registered URL; another path, port, host, or endpoint purpose is not trusted.

Root Docker Compose configures host.docker.internal and maps it to the host gateway. Use:

http://host.docker.internal:<port>/v1

For a server on another private machine, register its exact URL as a local_container endpoint and ensure every API and durable-worker replica has a network route to it.

Never expose an unauthenticated model server to the public internet. Binding a server to 0.0.0.0 can expose it to the LAN; use that only when a container or another machine must reach it, then restrict the listening port with a host firewall and enable authentication or a protected reverse proxy.

Ollama: Recommended Cross-Platform Start

Ollama is the simplest common setup across Linux, macOS, and Windows. Its OpenAI compatibility reference documents Chat Completions, streaming, tools, and /v1/models. Its tool-calling guide covers single, parallel, multi-turn, and streamed tool calls.

gpt-oss:20b is a useful tool-capable starting point for machines with enough memory. Ollama’s current model page describes the 20B variant as fitting in about 16 GB of memory. Choose a smaller tool-capable model if the host cannot run it comfortably.

Linux

Install with the current command from the official Linux guide:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull gpt-oss:20b
curl http://localhost:11434/v1/models

The installer normally creates a system service. If the Voxos backend runs directly on the host, keep Ollama on its default loopback listener and use http://localhost:11434/v1.

For a containerized Voxos backend, Ollama must listen on an address reachable through the host gateway. Run sudo systemctl edit ollama.service and add:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Restrict port 11434 to the Docker host/network before using this listener.

macOS

Install the current application from the official macOS guide, start it, then run:

ollama pull gpt-oss:20b
curl http://localhost:11434/v1/models

The current Ollama requirement is macOS 14 or newer. Apple Silicon uses CPU and GPU acceleration; Intel Macs use CPU execution.

For host-mode Voxos, use http://localhost:11434/v1. If Docker Desktop cannot reach the default loopback listener, follow the official Ollama server configuration and set:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"

Restart the Ollama application, use http://host.docker.internal:11434/v1 in Voxos, and keep the port protected from untrusted LAN clients.

Windows

Install OllamaSetup.exe from the official Windows guide. The current native application supports Windows 10 22H2 or newer and makes ollama available in PowerShell and Command Prompt. Then run:

ollama pull gpt-oss:20b
Invoke-RestMethod http://localhost:11434/v1/models

Use http://localhost:11434/v1 when Voxos runs directly in Windows. For Docker Desktop, create the user environment variable OLLAMA_HOST=0.0.0.0:11434, restart Ollama, and configure http://host.docker.internal:11434/v1. Limit inbound access to the local machine and Docker networking in Windows Firewall.

LM Studio: Desktop Alternative

LM Studio is a GUI-first alternative on supported macOS, Windows, and Linux systems. Its current system requirements cover Apple Silicon macOS, x64/ARM Windows, and x64/ARM64 Linux. The OpenAI-compatible endpoint guide documents /v1/models, /v1/responses, and /v1/chat/completions.

After installing LM Studio, download and load a tool-capable model in the app, open Developer, and start the server. The CLI equivalent is:

lms get openai/gpt-oss-20b
lms load openai/gpt-oss-20b
lms server start

The default base URL is http://localhost:1234/v1. Use the exact ID returned by http://localhost:1234/v1/models in Voxos.

For Docker-host access, LM Studio’s network serving guide uses:

lms server start --bind 0.0.0.0

Enable Require Authentication in LM Studio server settings, create an API token, enter it in the Voxos provider card, and restrict the port before binding beyond loopback.

vLLM: Linux GPU Serving

vLLM is a strong choice for a dedicated Linux GPU host and higher-throughput serving. Follow its frequently updated installation and serving quickstart for the accelerator and Python versions on the host.

For GPT-OSS tool calling, the current vLLM tool-calling guide requires automatic tool choice and the OpenAI parser:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
vllm serve openai/gpt-oss-20b \
  --enable-auto-tool-choice \
  --tool-call-parser openai \
  --api-key replace-with-a-long-random-token

The default base URL is http://localhost:8000/v1. If Voxos runs in Docker, add --host 0.0.0.0, use http://host.docker.internal:8000/v1, enter the same API key in Voxos, and firewall the port. vLLM also publishes current official Docker images for NVIDIA, AMD ROCm, and Intel XPU deployments.

llama.cpp: Lightweight Manual Serving

llama.cpp provides prebuilt packages and binaries for Linux, macOS, and Windows, plus multiple CPU and GPU backends. Its server reference documents the OpenAI-compatible Chat Completions and model-list endpoints.

Start a model with a stable API alias and tool-aware Jinja processing:

llama-server \
  -hf bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M \
  --alias qwen2.5-7b-instruct \
  --jinja \
  --port 8080

On Windows, use llama-server.exe with the same arguments. Configure http://localhost:8080/v1 and qwen2.5-7b-instruct in host-mode Voxos. The official function-calling guide explains compatible templates and model-specific overrides; tool reliability depends on choosing the correct template.

Verify Before Selecting The Provider

First list the models from the endpoint that the Voxos backend can reach:

curl http://host.docker.internal:11434/v1/models

Then test a streamed tool call against the chosen runtime. A basic text reply alone does not prove the model can operate a Voxos agent. Confirm that the response emits structured tool_calls, accepts a later tool role result, and returns a final answer.

After saving the provider and Agent defaults, create a new chat and ask for a low-risk built-in read action that is granted to the agent. Review the Chat activity panel for the structured tool call and result. Run the same check with the durable run worker before enabling schedules.

Operational Guidance

• Size model weights, KV cache, and context length to fit available RAM/VRAM.
• Prefer an instruct model explicitly trained for function calling.
• Pin model IDs or artifacts for repeatable unattended runs.
• Keep the server and model runtime current, but retest streaming and tools before upgrading production.
• Put remote endpoints behind TLS and bearer authentication.
• Treat prompts, attachments, tool schemas, and tool results as user data; the endpoint receives all content needed for each run.
• Monitor queueing, time to first token, context exhaustion, empty responses, malformed tool calls, and worker timeouts.
• Do not assume an endpoint’s /v1/models result proves model capability.

See Troubleshooting for container networking and endpoint policy errors.