Speech to text

Turn speech into text: whole recordings up to an hour, or live audio as it is spoken. The API takes the same requests as OpenAI's transcription endpoint, so the OpenAI SDKs work by changing the base URL.

Models

yoob-asr-2 votes between five speech recognition models and keeps the words they agree on; long audio is cut at its quiet points into pieces of at most 30 seconds. yoob-asr-1 is the default when you send no model.

Transcribe a file

Send a multipart upload with the file in file, as the OpenAI SDKs do, or the raw audio as the request body with its Content-Type and the options in the query string. Any common format works: WAV, MP3, M4A, MP4, FLAC, Ogg, WebM. Files can be up to 25 MB.

import os
from openai import OpenAI

client = OpenAI(base_url="https://tts-api.yoob.com/v1", api_key=os.environ["YOOB_API_KEY"])
with open("call.mp3", "rb") as audio:
    transcript = client.audio.transcriptions.create(model="yoob-asr-2", file=audio)
print(transcript.text)

Response formats

A yoob-asr-2 verbose_json response, shortened:

{
  "task": "transcribe",
  "language": "english",
  "duration": 1834.2,
  "text": "Thanks for calling. How can I help you today? ...",
  "segments": [
    { "id": 0, "start": 0.0, "end": 3.12, "text": "Thanks for calling. How can I help you today?" },
    { "id": 1, "start": 3.84, "end": 9.4, "text": "Hi, yes, I ordered a jacket last week and it still hasn't arrived." },
    ...
  ],
  "model": "yoob-asr-2"
}

Realtime

Live audio uses yoob-asr-1: open the transcription WebSocket and send microphone audio as it is captured: raw 16 kHz, 16-bit little-endian mono PCM, in binary frames of at most a second each (40 ms frames work well). The transcript fills in while the speaker talks.

This runs on your server (Node.js with the ws package), where the key is safe. A browser can't set the header and shouldn't hold your key: send its microphone audio to your server and relay it from there.

import WebSocket from "ws";

const ws = new WebSocket("wss://tts-api.yoob.com/v1/audio/transcriptions/stream", {
  headers: { authorization: `Bearer ${process.env.YOOB_API_KEY}` },
});
ws.binaryType = "arraybuffer";
mic.on("frame", (pcm16k) => ws.send(pcm16k));          // Int16Array, 16 kHz mono
setInterval(() => ws.send(JSON.stringify({ type: "ping", t: Date.now() })), 2000);

ws.onmessage = (e) => {
  const m = JSON.parse(e.data);
  if (m.type === "partial") show(m.utt, m.caption);      // the whole utterance so far: replace, don't append
  if (m.type === "final") lock(m.utt, m.text);
};

// when the speaker is done
ws.send(JSON.stringify({ type: "stop" }));               // the last words arrive, then "done", then the socket closes

• An utterance ends by itself after about 0.7 s of quiet. Send {"type": "end"} to end it sooner, for example when your own turn detection says the speaker is done.
• Send a ping every few seconds; the reply is a pong with your t.
• Any other message, or a frame that is odd-sized or over a second long, ends the session with an error.

Realtime events

All times are seconds of audio since the session began.

Limits and price

• yoob-asr-2: files up to 60 minutes. yoob-asr-1: 60 seconds a request or a realtime session.
• Files up to 25 MB a request.
• Requests run on shared capacity; when it is full you get 429 with busy, retry in a moment. Your plan also sets how many requests you can run at once.
• Billed per second of audio: $0.055 an hour for files, $0.10 an hour live.

Next steps