Skip to content
Osarai
Get started

DocsSDKs and frameworks

Vercel AI SDK

Use your Osarai GPU from the Vercel AI SDK with the OpenAI-compatible provider, for text, streaming, tools and embeddings.

The AI SDK reaches Osarai through its OpenAI-compatible provider.

Install

Terminal
npm install ai @ai-sdk/openai-compatible zod

Create the provider

lib/osarai.ts
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

export const osarai = createOpenAICompatible({
  name: "osarai",
  baseURL: "https://osarai.xyz/v1",
  apiKey: process.env.OSARAI_API_KEY,
  includeUsage: true,
});

Generate text

TypeScript
import { generateText } from "ai";
import { osarai } from "./lib/osarai";

const { text, reasoningText } = await generateText({
  model: osarai("qwen3.8-27b"),
  prompt: "Explain recursion in one paragraph.",
});
console.log(text);

Stream

TypeScript
import { streamText } from "ai";
import { osarai } from "./lib/osarai";

const result = streamText({
  model: osarai("qwen3.8-27b"),
  prompt: "Write a limerick about GPUs.",
});
for await (const part of result.textStream) process.stdout.write(part);

In a Next.js route handler, return result.toUIMessageStreamResponse() to stream to useChat on the client.

Tools

TypeScript
import { generateText, stepCountIs, tool } from "ai";
import { z } from "zod";
import { osarai } from "./lib/osarai";

const { text } = await generateText({
  model: osarai("qwen3.8-27b"),
  tools: {
    getWeather: tool({
      description: "Current weather for a city",
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, tempC: 18, sky: "clear" }),
    }),
  },
  stopWhen: stepCountIs(3),
  prompt: "What is the weather in Tokyo?",
});
console.log(text);

Embeddings

TypeScript
import { embedMany } from "ai";
import { osarai } from "./lib/osarai";

const { embeddings } = await embedMany({
  model: osarai.textEmbeddingModel("qwen3-embedding-8b"),
  values: ["A cat sleeps on the sofa.", "A kitten naps on the couch."],
});

Anthropic format

To use the AI SDK's Anthropic provider instead, point it at the Messages API:

TypeScript
import { createAnthropic } from "@ai-sdk/anthropic";

const anthropic = createAnthropic({ baseURL: "https://osarai.xyz/v1", apiKey: process.env.OSARAI_API_KEY });
const model = anthropic("qwen3.8-27b");

The Anthropic provider's base URL includes /v1, unlike the Anthropic SDKs.