llama-server speaks the OpenAI API directly, and GoModel has a
dedicated llamacpp provider type for it: the API key is optional
(llama-server usually runs keyless), and provider-native endpoints such as
/health and /rerank are reachable through passthrough. Do not register
llama.cpp as an ollama provider: that type speaks Ollama’s native API, which
llama.cpp does not implement.
The same provider type fits LM Studio and any other plain OpenAI-compatible
local server.
Start llama-server first. Without --alias, the model ID in /v1/models is
the model file’s path — set an alias so requests can use a clean name:
Configure
The base URL is required and registers the provider — llama-server’s default port (8080) collides with GoModel’s own, so there is no default:These examples assume GoModel runs in Docker and llama-server is on the host
at
localhost:8081 — hence host.docker.internal. If GoModel runs on the
host directly, use http://localhost:8081/v1. Running several llama-server
instances? Register each under a suffixed name:
LLAMACPP_STUDIO_BASE_URL=... creates provider llamacpp-studio.Verify
GET /v1/models returns llama-server’s model IDs prefixed by provider name.
Embeddings
/v1/embeddings works through GoModel as long as llama-server can serve it:
the loaded model must use a pooling type other than none. Dedicated embedding
GGUFs usually declare pooling in their metadata; otherwise pass --pooling mean
(or cls/last). The --embeddings flag is optional — it restricts the
server to embeddings only:
Model metadata
Local GGUFs are not in the upstream model catalog, so GoModel reads what it can from llama-server itself and reports it onGET /v1/models:
- Context window — the per-slot
n_ctxllama-server reports for the model, which is the context it is running with (--ctx-size, divided across--parallelslots) rather than the one the GGUF was trained for. Recent builds report it asmeta.n_ctxin the listing itself; older ones are asked for it via/props. If neither answers — LM Studio, or a proxy that hides/props— the model’s trainedn_ctx_trainis used instead, which is only an upper bound and may exceed what the server will accept. - Modalities — a multimodal server (started with
--mmproj) reportsvision,video, andaudioon/props; the supported ones become model capabilities.
/props describes the one model the server has loaded, so it is only consulted
for a single-entry listing. In router
mode, where one
process serves several models, each model carries its own meta.n_ctx and needs
no /props at all.
Both are defaults, not decisions: anything you declare under the provider’s
model metadata wins — see Model metadata.
Model classification
llama-server reports nothing that separates a chat model from an embedding one, so GoModel classifies its models by ID: names containingembed or matching
well-known embedding families (bge, e5, gte, minilm) are categorized as
embedding models, and names containing rerank as reranking models — namespaced
IDs are checked by their final path segment. For anything that stays
unclassified, declare modes under the provider’s model metadata. Categories
only affect dashboard grouping and failover suggestions; /v1/embeddings routes
to any model the provider serves regardless of category.
Beyond chat and embeddings
- Multimodal input — image and audio input in chat messages works with
multimodal models when llama-server is started with a projector
(
--mmproj, auto-loaded with-hfwhen available). These requests flow through/v1/chat/completionsnormally. - Native endpoints — llama-server’s provider-native routes are reachable
through passthrough at
/p/llamacpp/...: OpenAI-shaped paths (chat/completions,embeddings, …) are served from the/v1base, everything else (/health,/props,/tokenize,/infill, …) from the server root. For a suffixed setup use the instance name, e.g./p/llamacpp-studio/health. - Reranking — llama-server serves reranking (start with a reranker model
and
--rerank --embedding --pooling rank), but GoModel has no rerank endpoint; reach it through passthrough:POST /p/llamacpp/rerank(or/p/llamacpp/v1/rerank).