Web SDK

npm install @yoob/avatar

The SDK renders with WebGPU and runs its models in workers, so it is a browser-only package. Desktop Chrome, Edge and Safari 26 are supported; call YoobAvatar.isSupported() before you build a UI around it.

This is the only step that uses your API key, and it must happen on your server. Pass the response straight through to the browser — every field in it is short-lived and scoped to one session.

// POST /yoob-session on your backend
const user = await requireSignedInUser(req);          // your auth
const response = await fetch("https://la.yoob.com/api/v1/avatar/sessions", {
  method: "POST",
  headers: {
    authorization: `Bearer ${process.env.YOOB_API_KEY}`,
    "content-type": "application/json",
  },
  body: JSON.stringify({
    characters: ["astrid"],   // the characters this session may show
    mode: "full",                    // "full" = avatar + voice, "avatar" = video only
    end_user_id: user.id,            // optional, shows up in your usage
  }),
});

return response.json();  // hand this to the browser unchanged

Never call this from a browser. A request carrying Origin is refused with 403 browser_key_use — an API key in page source is public the moment it ships.

import { YoobAvatar } from "@yoob/avatar";

const support = await YoobAvatar.isSupported();
if (!support.supported) throw new Error(support.reason);

const avatar = new YoobAvatar({
  container: document.getElementById("avatar"),
  character: "astrid",
  // Called now, and again whenever a session expires.
  getCredentials: () =>
    fetch("/yoob-session", { method: "POST" }).then((r) => r.json()),
  fit: "cover",                       // "contain" letterboxes instead
  onProgress: ({ fraction }) => console.log(`${Math.round(fraction * 100)}%`),
  onPhase: (phase) => console.log(phase),
  onError: (error) => console.error(error.message),
});

await avatar.prepare();

prepare() downloads the character, verifies every chunk and warms the models. The poster appears first, then an idle loop, then the live renderer — so there is something on screen within a second or two even though the full download is tens of megabytes. Chunks are content-addressed and cached, so a second visit is close to instant.

Call avatar.destroy() when you are finished. It settles the final slice of usage; a tab that closes without it is reaped automatically and is not billed for the silence.

YoobConversation connects the character to a realtime voice model and plays the audio through it, so the lips stay in sync. Your server mints a short-lived secret; the browser never sees your provider key.

import { YoobConversation } from "@yoob/avatar";

const conversation = new YoobConversation(avatar, {
  getClientSecret: () =>
    fetch("/openai-secret", { method: "POST" })
      .then((r) => r.json())
      .then((b) => b.value),
  instructions: "You are a warm, concise assistant.",
  greet: true,
  onUserTranscript: (text) => { /* captions */ },
  onAssistantTranscript: (text) => { /* captions */ },
});

await conversation.start();   // asks for the microphone
conversation.stop();

If you already have audio from elsewhere, skip the conversation and feed the character directly with avatar.speak(pcm) or avatar.attachAudioTrack(track) for a LiveKit track.

GPT-Live cannot be reached from a page on its own: it authenticates with an Authorization header, and a browser's WebSocket cannot set one. It takes an SDP offer instead, so your server does the handshake with your OpenAI key and hands back only the answer. The audio then travels between the browser and OpenAI directly — neither your server nor ours is in the media path.

import { YoobWebRTCConversation } from "@yoob/avatar";

const conversation = new YoobWebRTCConversation(avatar, {
  getAnswer: (offer) =>
    fetch("/gpt-live-answer", {
      method: "POST",
      headers: { "content-type": "application/json" },
      body: JSON.stringify({ sdp: offer }),
    }).then((r) => r.json()),            // -> { sdp }
  onUserTranscript: (text) => { /* captions */ },
  onAssistantTranscript: (text) => { /* captions */ },
});

await conversation.start();   // asks for the microphone
await conversation.stop();

On your server, the offer is forwarded and only the answer comes back. Leave audio.format out: with WebRTC the codec is negotiated in SDP, and sending one is refused.

// POST /gpt-live-answer  { sdp }  ->  { sdp }
const upstream = await fetch("https://api.openai.com/v1/live/sessions", {
  method: "POST",
  headers: {
    authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    "content-type": "application/json",
  },
  body: JSON.stringify({
    session: {
      model: "gpt-live-1",
      instructions: "You are a warm, concise assistant.",
      audio: { output: { voice: "marin" } },
    },
    transport: { type: "webrtc", sdp },
  }),
});
const { transport } = await upstream.json();   // 201
res.json({ sdp: transport.sdp });              // the session id stays here

Check the offer before forwarding it: cap it at about 40 kB and refuse anything with a video section. The model is full-duplex and takes its own turns, so there is no barge-in to handle and no greet to send.

Creating a session bills 15 seconds of voice up front, credited back once it starts, so open one when the user asks to talk rather than on page load. Avatar minutes are metered as usual from the moment the connection is live.

The SDK sends a heartbeat every 15 seconds from the moment prepare() starts, and each one bills the slice since the last. A gap longer than twice the interval is capped, so a sleeping laptop is not charged for the nap, and a session with no heartbeat for a minute is closed at its last beat.

Two rates, chosen when the session opens and fixed for its life — a price change never re-prices a running session:

mode defaults to full. Whether a session used voice cannot be detected afterwards — it runs between the browser and your voice provider, not through us — so the cheaper rate has to be asked for. Watch the spend on the Usage page.

Keys beginning yoob_test_ open free sandbox sessions — five minutes each, an hour a day per workspace — so you can build without spending. They appear in Usage marked SANDBOX and cost nothing. Create one on the API Key page.

Mid-session, a heartbeat can answer stop: true with a reason. Only out-of-credits is final; anything else means open a fresh session.

connect-src 'self' https://la.yoob.com https://cdn.yoob.com
script-src  'self' 'wasm-unsafe-eval'
worker-src  'self'
img-src     'self' blob:
media-src   'self' blob:

The character files come from the CDN and the heartbeats from this API, so both hosts have to be reachable. The renderer compiles WebAssembly, which is what wasm-unsafe-eval is for.