Docs/Get started

Models and limits

The chat models you can deploy, their context windows and output caps, and which support tool calls, reasoning and images.

#Chat models

:::models :::

• Context is the most tokens a request can hold, prompt and answer together. Some GPUs run a model with a larger window; the deploy page shows the exact value for each choice.
• Max output caps max_tokens on one request. Leave max_tokens unset on your own instance and the model may use the rest of the context.
• Tool calls means the model returns structured tool_calls (OpenAI) or tool_use blocks (Anthropic). Coding agents need this.
• Reasoning models think before they answer; you can set how much (see below).

#Which model for what

#Finding your model name

The model id is shown on your instance page. You can also list the models your key can reach:

curl "https://osarai.xyz/v1/models" -H "Authorization: Bearer $OSARAI_API_KEY"

{
  "object": "list",
  "data": [
    {
      "id": "qwen3.8-27b",
      "object": "model",
      "owned_by": "osarai",
      "display_name": "qwen3.8-27b",
      "max_model_len": 131072,
      "ready": true,
      "instances": [{ "id": "a1b2c3d4e5", "ready": true, "max_model_len": 131072 }]
    }
  ]
}

#Reasoning effort

Reasoning models think first, and the thinking is returned separately from the answer (reasoning_content in Chat Completions, thinking blocks in Messages). Set how much they think per request:

{ "model": "qwen3.8-27b", "reasoning_effort": "low", "messages": [ ... ] }

Anthropic clients set it with thinking: {"type": "disabled"} turns it off, and {"type": "enabled", "budget_tokens": N} picks low (under 4,000), medium (under 16,000) or high.

#Other model types

You can also deploy models for embeddings, reranking, speech, transcription, images, video and OCR. Each has its own endpoint: see Embeddings, Rerank and score, Speech, Transcriptions, Images and Videos.