Skip to content
Osarai
Get started

DocsSDKs and frameworks

OpenAI SDK

Use the official OpenAI SDKs for Python and Node.js with your Osarai instance by changing the base URL and key.

The official OpenAI SDKs work with Osarai unchanged: point base_url at https://osarai.xyz/v1 and use your qck_… key.

Install

Terminal
pip install openai

Configure

Both SDKs read two environment variables, so existing code can switch to Osarai without edits:

Terminal
export OPENAI_BASE_URL="https://osarai.xyz/v1"
export OPENAI_API_KEY="$OSARAI_API_KEY"

Or pass them explicitly:

Python
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://osarai.xyz/v1",
    api_key=os.environ["OSARAI_API_KEY"],
    timeout=900,      # long answers can take minutes
    max_retries=3,    # retries 429, 502 and 503 with backoff
)

Chat

Python
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Explain recursion in one paragraph."}],
)
print(reply.choices[0].message.content)

Streaming

Python
stream = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Write a limerick about GPUs."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Reasoning content

The thinking of reasoning models comes back in reasoning_content, a field the SDK types do not declare. Read it like this:

Python
reply = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Is 221 prime?"}],
    reasoning_effort="low",
)
message = reply.choices[0].message
print("thinking:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

Extra parameters

vLLM parameters that OpenAI's API lacks (top_k, min_p, repetition_penalty, chat_template_kwargs) go in extra_body (Python) or straight into the request object (TypeScript, with a cast):

Python
client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Name three rivers."}],
    extra_body={"top_k": 20, "min_p": 0.0},
)

Other endpoints

The same client covers Responses, Embeddings, Images, Speech and Transcriptions; each page has SDK examples.