Skip to content
Osarai
Get started

DocsSDKs and frameworks

LangChain

Use your Osarai GPU in LangChain for Python and JavaScript with ChatOpenAI and OpenAIEmbeddings pointed at your base URL.

LangChain's OpenAI integration works with Osarai: set the base URL, your key and your model id.

Install

Terminal
pip install langchain-openai

Chat model

Python
from langchain_openai import ChatOpenAI
import os

llm = ChatOpenAI(
    model="qwen3.8-27b",
    base_url="https://osarai.xyz/v1",
    api_key=os.environ["OSARAI_API_KEY"],
    timeout=900,
)
print(llm.invoke("Explain recursion in one paragraph.").content)

for chunk in llm.stream("Write a limerick about GPUs."):
    print(chunk.content, end="", flush=True)

Tools

Python
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Current weather for a city."""
    return f"18°C and clear in {city}"

llm_with_tools = llm.bind_tools([get_weather])
reply = llm_with_tools.invoke("What is the weather in Tokyo?")
print(reply.tool_calls)

with_structured_output(MySchema) also works, through tool calls or JSON schema.

Embeddings

Python
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="qwen3-embedding-8b",
    base_url="https://osarai.xyz/v1",
    api_key=os.environ["OSARAI_API_KEY"],
    check_embedding_ctx_length=False,
)
vector = embeddings.embed_query("How do I reset my password?")

check_embedding_ctx_length=False matters: otherwise LangChain splits text with OpenAI's tokenizer and sends token ids, which other models do not understand.

Anthropic format

langchain-anthropic works too: ChatAnthropic(model="qwen3.8-27b", base_url="https://osarai.xyz", api_key=…).