Name: Whisper Diarization API: Transcription with Speaker Labels
Brand: magicapi
SKU: api-magicapi-fast-audio-transcription-speaker-diarization-thomasmol-whisper-diarization
Availability: InStock
Author: API Market

Shashank Agarwal

# 🎤 Transcription with Speaker Labels API (thomasmol Whisper Diarization)

## Overview
Whisper Diarization combines Whisper Large V3 Turbo for transcription with pyannote 4.0 community-1 for speaker diarization.

Send an audio file and get back the transcript split into segments. Each segment has a speaker label, start and end times in seconds, the text and word-level timestamps. You can set the number of speakers and the language, or leave both empty to detect them, and translate the speech into English.

## Features
- **Speaker labels**: every segment and word carries a speaker label such as `SPEAKER_00`.
- **Timestamps**: start and end times for each segment and each word, plus word probabilities.
- **Three ways to send audio**: a file URL in `file`, a direct audio URL in `file_url`, or a Base64-encoded file in `file_string`.
- **Speaker count**: set `num_speakers` (1 to 50) or leave it empty to detect it.
- **Language**: set a language code such as `en`, or leave it empty to detect it.
- **Translation**: `translate` turns the speech into English text.
- **Vocabulary prompt**: list names, acronyms and loanwords in `prompt` to help spell them correctly.

## Use Cases
1. **Podcast and interview transcripts**: text split by speaker, ready for show notes or captions.
2. **Meeting notes**: who said what, with timestamps to jump back into the recording.
3. **Call analysis**: separate agent and customer speech in support or sales calls.
4. **Subtitles**: word timestamps for captions in the original language or in English.

## Visual Examples
### Example Input
[Listen to the example input audio](https://blog.api.market/wp-content/uploads/2026/10/whisper-diarization-input-api-market.mp3)

The example sends about 90 seconds of a two-person interview with `num_speakers` `2`, `language` `en` and the vocabulary prompt "LLama, AI, Meta.".

```json
{
  "file": "https://blog.api.market/wp-content/uploads/2026/10/whisper-diarization-input-api-market.mp3",
  "prompt": "LLama, AI, Meta.",
  "file_url": "",
  "language": "en",
  "translate": false,
  "num_speakers": 2
}
```

### Example Output
The model detected 2 speakers and returned 16 segments:

| Speaker | Start (s) | End (s) | Text |
| --- | --- | --- | --- |
| SPEAKER_01 | 2.94 | 4.48 | Let me ask you about AI. |
| SPEAKER_01 | 4.74 | 11.66 | It seems like this year for the entirety of the human civilization is an interesting year for the development of artificial intelligence. |
| SPEAKER_01 | 12.22 | 14.24 | A lot of interesting stuff is happening. |
| SPEAKER_01 | 15.22 | 17.06 | So Meta is a big part of that. |
| SPEAKER_01 | 18.32 | 21.92 | Meta has developed LLAMA, which is a 65 billion parameter model. |
| SPEAKER_01 | 24.14 | 28.22 | There's a lot of interesting questions they can ask here, one of which has to do with open source. |
| SPEAKER_01 | 28.22 | 38.88 | But first, can you tell the story of developing of this model and making the complicated decision of how to release it? |
| SPEAKER_00 | 39.66 | 40.16 | Yeah, sure. |
| SPEAKER_00 | 40.44 | 49.54 | I think you're right, first of all, that in the last year, there have been a bunch of advances on scaling up these large transformer models. |
| SPEAKER_00 | 49.7 | 52.32 | So there's the language equivalent of it with large language models. |
| SPEAKER_00 | 52.66 | 57.84 | There's sort of the image generation equivalent with these large diffusion models. |
| SPEAKER_00 | 59 | 61.36 | There's a lot of fundamental research that's gone into this. |
| SPEAKER_00 | 61.54 | 73.51 | And Meta has taken the approach of being quite open and academic in our development of AI. |
| SPEAKER_00 | 73.97 | 78.35 | Part of this is we want to have the best people in the world researching this. |
| SPEAKER_00 | 78.53 | 82.79 | And a lot of the best people want to know that they're going to be able to share their work. |
| SPEAKER_00 | 82.79 | 89.97 | So that's part of the deal that we have is that we can get – if you're one of the top AI researchers – |

Each segment also includes `duration`, `avg_logprob` and a `words` list with per-word timestamps, probability and speaker.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `file_string` | string | No | none |  | Base64-encoded audio file (one of file_string, file_url or file) |
| `file_url` | string | No | none |  | A direct audio file URL (one of file_string, file_url or file) |
| `file` | string (URL) | No | none |  | Audio file URL (one of file_string, file_url or file) |
| `num_speakers` | integer | No | none | `1` to `50` | Number of speakers. Leave empty to detect it. |
| `translate` | boolean | No | `false` |  | Translate the speech into English. |
| `language` | string | No | none |  | Language of the spoken words as a language code like en. Leave empty to detect the language. |
| `prompt` | string | No | none |  | Vocabulary: names, acronyms and loanwords in a list. Use punctuation for best accuracy. |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.056 per API unit.
- **1 API unit = 1 request with up to 51 seconds of processing time.** A request that takes longer is billed one unit per started 51-second block (for example, 80 seconds of processing is 2 units, $0.112).
- Longer recordings take longer to process, so very long files can use more than one unit.
- **Example cost**: the example's 90 seconds of audio took 5.77 seconds of processing: 1 API unit = **$0.056**.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/fast-audio-transcription-speaker-diarization-thomasmol-whisper-diarization`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/fast-audio-transcription-speaker-diarization-thomasmol-whisper-diarization/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "744c4f2bffae674f82e79ed46f9cc54796d4903b3e20e68353e05a93eb10a55c",
    "input": {
      "file": "https://blog.api.market/wp-content/uploads/2026/10/whisper-diarization-input-api-market.mp3",
      "prompt": "LLama, AI, Meta.",
      "file_url": "",
      "language": "en",
      "translate": false,
      "num_speakers": 2
    }
  }'
```

Response (201 Created):
```json
{
  "id": "yf4nptxs69rmy0d1c6v8ay69v4",
  "version": "744c4f2bffae674f82e79ed46f9cc54796d4903b3e20e68353e05a93eb10a55c",
  "status": "starting",
  "created_at": "2026-10-22T03:45:27.599Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/fast-audio-transcription-speaker-diarization-thomasmol-whisper-diarization/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "yf4nptxs69rmy0d1c6v8ay69v4",
  "status": "succeeded",
  "output": {
    "language": "en",
    "num_speakers": 2,
    "segments": [
      {
        "avg_logprob": -0.16484375014901162,
        "duration": 1.5400000000000005,
        "end": 4.48,
        "speaker": "SPEAKER_01",
        "start": 2.94,
        "text": "Let me ask you about AI.",
        "words": [
          {
            "end": 3.12,
            "probability": 0.8173828125,
            "speaker": "SPEAKER_01",
            "start": 2.94,
            "word": " Let"
          },
          {
            "end": 3.24,
            "probability": 0.99951171875,
            "speaker": "SPEAKER_01",
            "start": 3.12,
            "word": " me"
          },
          {
            "end": 3.74,
            "probability": 0.99951171875,
            "speaker": "SPEAKER_01",
            "start": 3.24,
            "word": " ask"
          },
          {
            "end": 3.86,
            "probability": 0.99951171875,
            "speaker": "SPEAKER_01",
            "start": 3.74,
            "word": " you"
          },
          {
            "end": 4.12,
            "probability": 1,
            "speaker": "SPEAKER_01",
            "start": 3.86,
            "word": " about"
          },
          {
            "end": 4.48,
            "probability": 0.99365234375,
            "speaker": "SPEAKER_01",
            "start": 4.12,
            "word": " AI."
          }
        ]
      }
    ]
  },
  "error": null,
  "created_at": "2026-10-22T03:45:27.599Z",
  "started_at": "2026-10-22T03:45:27.608Z",
  "completed_at": "2026-10-22T03:45:33.379Z"
}
```

`output` is an object with `language`, `num_speakers` and `segments`. The response above is shortened to the first of the 16 segments. Each segment has `start`, `end`, `duration`, `speaker`, `text`, `avg_logprob` and a `words` list; each word has `word`, `start`, `end`, `probability` and `speaker`.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example no audio in `file`, `file_url` or `file_string`, or a `num_speakers` outside 1 to 50.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.
## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/fast-audio-transcription-speaker-diarization-thomasmol-whisper-diarization";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "744c4f2bffae674f82e79ed46f9cc54796d4903b3e20e68353e05a93eb10a55c", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
    "file": "https://blog.api.market/wp-content/uploads/2026/10/whisper-diarization-input-api-market.mp3",
    "prompt": "LLama, AI, Meta.",
    "file_url": "",
    "language": "en",
    "translate": false,
    "num_speakers": 2
  }).then((output) => console.log(JSON.stringify(output, null, 2)));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/fast-audio-transcription-speaker-diarization-thomasmol-whisper-diarization"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "744c4f2bffae674f82e79ed46f9cc54796d4903b3e20e68353e05a93eb10a55c", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "file": "https://blog.api.market/wp-content/uploads/2026/10/whisper-diarization-input-api-market.mp3",
    "prompt": "LLama, AI, Meta.",
    "file_url": "",
    "language": "en",
    "translate": False,
    "num_speakers": 2
})
print(output)
```

## FAQ
- **How do I send my audio?**
  - Use one of three fields: `file` (a file URL), `file_url` (a direct audio URL) or `file_string` (the file as Base64).
- **Do I have to know how many people speak?**
  - No. Leave `num_speakers` empty to detect it, or set it (1 to 50) when you know the count.
- **Can I get an English transcript of non-English audio?**
  - Yes. Set `translate` to `true`.
- **How do I improve the spelling of names and terms?**
  - List them in `prompt`, for example "LLama, AI, Meta." as in the example.
- **How much does a request cost?**
  - $0.056 for each started 51 seconds of processing. The 90-second example took 5.77 seconds and cost $0.056.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.

## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello@magicapi.com.


Whisper diarization API: transcribe audio with Whisper Large V3 Turbo and label who spoke when, with segment and word timestamps. Pay as you go.