Skip to content
Osarai
Get started

DocsAPI reference

Messages (Anthropic)

POST /v1/messages and /v1/messages/count_tokens — the Anthropic Messages format with streaming events, thinking blocks and tool use, served by your own open model.

HTTP
POST https://osarai.xyz/v1/messages
POST https://osarai.xyz/v1/messages/count_tokens

Osarai speaks the Anthropic Messages API, so the Anthropic SDKs and Claude Code work against your instance. Requests are translated to your model's chat format and back; the model is still the open model you deployed, not Claude.

Anthropic clients add /v1/messages to the base URL themselves, so give them https://osarai.xyz, without /v1.

Request

Terminal
curl "https://osarai.xyz/v1/messages" \
  -H "x-api-key: $OSARAI_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "qwen3.8-27b",
    "max_tokens": 1024,
    "system": "You are a concise assistant.",
    "messages": [{"role": "user", "content": "What is a KV cache?"}]
  }'

The key works in x-api-key or in Authorization: Bearer. anthropic-version and anthropic-beta headers are accepted and ignored.

Supported fields

FieldNotes
modelYour instance's model id
messagesuser and assistant turns; content as a string or blocks: text, image (base64 or URL), tool_use, tool_result, document (its text)
systemA string or text blocks
max_tokensAnswer length cap
streamStream server-sent events
temperature, top_p, top_k, stop_sequencesSampling and stop sequences
tools, tool_choiceClient tools with input_schema; tool_choice auto, any, tool, none, and disable_parallel_tool_use
thinking{"type": "disabled"}, {"type": "enabled", "budget_tokens": N} or {"type": "adaptive"}
output_config.effortlow, medium, high or max, the same as reasoning_effort

Fields with no counterpart on an open model, such as cache_control, metadata, container and context-management settings, are accepted and ignored, never rejected. Anthropic's server tools (web search, code execution) are dropped from tools, because they run on Anthropic's servers.

Response

JSON
{
  "id": "msg_3f9a1c0e7b2d4a6f8e1c5b90",
  "type": "message",
  "role": "assistant",
  "model": "qwen3.8-27b",
  "content": [
    { "type": "thinking", "thinking": "The user asks what a KV cache is…", "signature": "osarai" },
    { "type": "text", "text": "A KV cache stores the attention keys and values of tokens already processed…" }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": { "input_tokens": 31, "output_tokens": 214 }
}

stop_reason is end_turn, max_tokens, stop_sequence or tool_use.

Streaming

With "stream": true the answer arrives as Anthropic's event sequence:

Output
event: message_start
data: {"type":"message_start","message":{"id":"msg_3f9a1c0e7b2d4a6f8e1c5b90","type":"message","role":"assistant","content":[],"model":"qwen3.8-27b","stop_reason":null,"usage":{"input_tokens":31,"output_tokens":0}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"A KV cache"}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":{"output_tokens":214}}

event: message_stop
data: {"type":"message_stop"}
  • Thinking streams as a thinking block with thinking_delta events, closed by a signature_delta.
  • Tool calls stream as tool_use blocks with input_json_delta events.
  • ping events keep the connection open while the model reads a long prompt.
Python
with client.messages.stream(
    model="qwen3.8-27b",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Count to five."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Tool use

Python
tools = [{
    "name": "get_weather",
    "description": "Current weather for a city",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

messages = [{"role": "user", "content": "Weather in Tokyo?"}]
first = client.messages.create(model="qwen3.8-27b", max_tokens=1024, tools=tools, messages=messages)
call = next(b for b in first.content if b.type == "tool_use")

messages.append({"role": "assistant", "content": first.content})
messages.append({"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": call.id, "content": '{"temp_c": 18, "sky": "clear"}'}
]})
final = client.messages.create(model="qwen3.8-27b", max_tokens=1024, tools=tools, messages=messages)
print(final.content[-1].text)

Thinking

thinkingEffect on the model
{"type": "disabled"}Thinking off (Qwen) or lowest effort (gpt-oss)
{"type": "enabled", "budget_tokens": 2000}Low effort (budget under 4,000)
{"type": "enabled", "budget_tokens": 8000}Medium effort (under 16,000)
{"type": "enabled", "budget_tokens": 32000}High effort
{"type": "adaptive"} or unsetThe model's default

The budget picks an effort level; it is not a hard cap on thinking tokens. signature fields are placeholders: send thinking blocks back unchanged, as Anthropic clients do.

Counting tokens

Terminal
curl "https://osarai.xyz/v1/messages/count_tokens" \
  -H "x-api-key: $OSARAI_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "qwen3.8-27b", "messages": [{"role": "user", "content": "Hello there"}]}'
JSON
{ "input_tokens": 12 }

The count comes from your instance's own tokenizer, with the same chat template as a real request; if the instance cannot answer, you get an estimate of one token per four characters. Counting does not run the model.

Errors

Errors use Anthropic's shape, {"type": "error", "error": {"type": "…", "message": "…"}}. A prompt longer than the context window answers 400 with prompt is too long: N tokens > M maximum, which Claude Code reads to compact the conversation. See Errors and rate limits.