DocsAPI reference

Chat Completions

POST /v1/chat/completions — the OpenAI chat format with streaming, reasoning, tool calls, images and structured output, served by your own instance.

HTTP

POST https://osarai.xyz/v1/chat/completions

The OpenAI Chat Completions format, the one most SDKs and tools speak. Requests go to your own running instance, which serves them with vLLM.

Request

Terminal

curl "https://osarai.xyz/v1/chat/completions" \
  -H "Authorization: Bearer $OSARAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "What is a KV cache?"}
    ],
    "max_tokens": 1024
  }'

Python

from openai import OpenAI
import os

client = OpenAI(base_url="https://osarai.xyz/v1", api_key=os.environ["OSARAI_API_KEY"])
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "What is a KV cache?"},
    ],
    max_tokens=1024,
)
print(reply.choices[0].message.content)

TypeScript

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://osarai.xyz/v1", apiKey: process.env.OSARAI_API_KEY });
const reply = await client.chat.completions.create({
  model: "qwen3.8-27b",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "What is a KV cache?" },
  ],
  max_tokens: 1024,
});
console.log(reply.choices[0].message.content);

Parameters

Field	Type	Notes
`model`	string	Your instance's model id. With one model running, any name reaches it
`messages`	array	`system`, `user`, `assistant` and `tool` messages. `content` is a string or an array of parts
`max_tokens` / `max_completion_tokens`	integer	Answer length cap, up to the model's max output (Models). Unset, the model may use the rest of the context
`stream`	boolean	Stream the answer as server-sent events
`stream_options.include_usage`	boolean	Add a final chunk with token usage
`temperature`, `top_p`, `top_k`, `min_p`	number	Sampling. Unset, the model's recommended values apply
`presence_penalty`, `frequency_penalty`, `repetition_penalty`	number	Repetition control
`stop`	string or array	Stop sequences
`seed`	integer	Reproducible sampling
`n`	integer	Several answers in one request
`reasoning_effort`	string	`none`, `low`, `medium`, `high` (see below)
`tools`, `tool_choice`, `parallel_tool_calls`		Function calling (see below)
`response_format`	object	`json_object` or `json_schema` (see below)
`logprobs`, `top_logprobs`		Token log probabilities

Other vLLM parameters, such as chat_template_kwargs, are passed on unchanged.

Response

JSON

{
  "id": "chatcmpl-8f1c2e",
  "object": "chat.completion",
  "created": 1793952000,
  "model": "qwen3.8-27b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": "The user asks what a KV cache is…",
        "content": "A KV cache stores the attention keys and values of tokens already processed…"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 28, "completion_tokens": 212, "total_tokens": 240 }
}

finish_reason is stop, length (hit max_tokens) or tool_calls.

Streaming

Set "stream": true to receive chat.completion.chunk events as the model writes. While the model reads a long prompt, the stream carries : keepalive comments so proxies keep the connection open.

Terminal

curl -N "https://osarai.xyz/v1/chat/completions" \
  -H "Authorization: Bearer $OSARAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen3.8-27b", "stream": true, "messages": [{"role": "user", "content": "Count to five."}]}'

Python

stream = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta if chunk.choices else None
    if delta and delta.content:
        print(delta.content, end="", flush=True)

TypeScript

const stream = await client.chat.completions.create({
  model: "qwen3.8-27b",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Output

data: {"id":"chatcmpl-8f1c2e","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"chatcmpl-8f1c2e","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"1, 2"}}]}

data: {"id":"chatcmpl-8f1c2e","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Reasoning

Reasoning models (Qwen3.8, gpt-oss) think before answering. The thinking arrives in reasoning_content (in stream deltas too), separate from content, so you can show or hide it.

JSON

{ "model": "qwen3.8-27b", "reasoning_effort": "none", "messages": [{ "role": "user", "content": "2+2?" }] }

none turns Qwen's thinking off for the fastest answers; low, medium and high think longer. gpt-oss always reasons, so none acts as low. See reasoning effort.

Tool calls

Describe functions in tools; the model answers with tool_calls when it wants one run. Run it yourself and send the result back as a tool message.

Python

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "Weather in Tokyo?"}]
first = client.chat.completions.create(model="qwen3.8-27b", messages=messages, tools=tools)
call = first.choices[0].message.tool_calls[0]

messages.append(first.choices[0].message)
messages.append({"role": "tool", "tool_call_id": call.id, "content": '{"temp_c": 18, "sky": "clear"}'})
final = client.chat.completions.create(model="qwen3.8-27b", messages=messages, tools=tools)
print(final.choices[0].message.content)

tool_choice takes auto (default), required, none or {"type": "function", "function": {"name": "get_weather"}}. Tool calls work on the models marked Tool calls in Models.

Images

Models with image input take image_url parts, as a URL or a base64 data URL:

JSON

{
  "model": "qwen3.8-27b",
  "messages": [{
    "role": "user",
    "content": [
      { "type": "text", "text": "What does this diagram show?" },
      { "type": "image_url", "image_url": { "url": "https://example.com/diagram.png" } }
    ]
  }]
}

The same format sends page images to the OCR model paddleocr-vl-1.6. Request bodies can be up to 64 MiB; for larger images, send URLs.

Structured output

Ask for JSON that matches a schema; vLLM constrains the answer to it:

JSON

{
  "model": "qwen3.8-27b",
  "messages": [{ "role": "user", "content": "Extract: Ana, 31, lives in Lisbon." }],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "person",
      "schema": {
        "type": "object",
        "properties": { "name": { "type": "string" }, "age": { "type": "integer" }, "city": { "type": "string" } },
        "required": ["name", "age", "city"]
      }
    }
  }
}

{"type": "json_object"} asks for any valid JSON.

Choosing an instance

With several instances, model picks the one that serves it. To pin one instance, send its id in X-Instance-Id (Authentication).

Chat Completions

#Request

#Parameters

#Response

#Streaming

#Reasoning

#Tool calls

#Images

#Structured output

#Choosing an instance

Request

Parameters

Response

Streaming

Reasoning

Tool calls

Images

Structured output

Choosing an instance