Name: WhisperX API: Speech to Text with Speaker Diarization
Brand: magicapi
SKU: api-magicapi-fast-speech-recognition-api-awerks-whisperx
Availability: InStock
Author: API Market

Shashank Agarwal

# 🎙️ Speech-to-Text API with Word Timestamps and Speaker Labels (awerks WhisperX)

## Overview
WhisperX by awerks transcribes speech from an audio file and can return word-level timestamps and speaker labels. Its developer describes it as fast automatic speech recognition (70x realtime with large-v2) with word-level timestamps and speaker diarization.

Send an audio file as `audio_file` (or a direct link as `audio_url`). Each request returns one string: by default a JSON document with timed segments, and with `only_text` set to `true` just the text.

## Features
- **Transcription or translation**: `task` is `transcribe` (default) or `translate`.
- **Word-level timing**: set `align_output` to `true` to get start and end times and a confidence score for every word.
- **Speaker labels**: set `diarize` to `true` to tag each segment and word with a speaker such as `SPEAKER_00`.
- **Language hint**: set `language` to the spoken language to reduce hallucinations, or leave it empty to detect it automatically.
- **Two ways to send audio**: upload a file in `audio_file`, or pass a direct link in `audio_url` together with its `file_extension`.
- **Text only**: set `only_text` to `true` to skip the segment metadata.

## Use Cases
1. **Subtitles**: segment start and end times map directly to caption cues.
2. **Meeting and interview notes**: speaker labels separate who said what.
3. **Searchable audio archives**: word timestamps let a search hit jump to the exact moment in a recording.
4. **Podcast and video transcripts**: turn spoken content into text for show notes and indexing.

## Visual Examples
### Example Input
Audio: [Listen to the example input](https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav) (read English sentences; speech ends at 48.4 seconds)

Settings: `task` `transcribe`, `align_output` `true`, `diarize` `true`, `only_text` `false`, `batch_size` `32`, `file_extension` `.wav`, `debug` `false`.

### Example Output
`output` is a JSON string. Shortened (one segment and one word shown):
```json
{
  "segments": [
    {
      "start": 2.697,
      "end": 6.779,
      "text": " The little tales they tell are false.",
      "words": [
        { "word": "The", "start": 2.697, "end": 2.797, "score": 0.908, "speaker": "SPEAKER_00" }
      ],
      "speaker": "SPEAKER_00"
    }
  ],
  "word_segments": [
    { "word": "The", "start": 2.697, "end": 2.797, "score": 0.908, "speaker": "SPEAKER_00" }
  ],
  "language": "en"
}
```
The full result has 11 segments, from "The little tales they tell are false." to "The beetle droned in the hot June sun.", all from `SPEAKER_00`.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `audio_file` | string (URL) | No | none |  | Audio file (input option 1) |
| `audio_url` | string | No | none |  | Direct audio URL (input option 2) |
| `file_extension` | string | No | `.wav` |  | Extension of the audio file, used when `audio_url` is set |
| `batch_size` | integer | No | `32` |  | Parallelization of input audio transcription |
| `task` | string | No | `transcribe` | `transcribe`, `translate` | Task to run |
| `language` | string | No | none |  | Original language of the audio (reduces hallucinations). Leave empty to detect automatically |
| `only_text` | boolean | No | `false` |  | Return only the text; otherwise segment metadata is returned as well |
| `align_output` | boolean | No | `false` |  | Use if you need word-level timing and not just batched transcription |
| `diarize` | boolean | No | `false` |  | Add speaker labels to the result |
| `debug` | boolean | No | `false` |  | Debugging output |

Send either `audio_file` or `audio_url`.

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0083 per API unit.
- **1 API unit = 1 request with up to 5 seconds of processing.** A request that takes longer is billed one more unit for each started 5-second block (for example, 12 seconds of processing is 3 units, $0.0249).
- Longer audio, `align_output` and `diarize` add processing time and can add units.
- The example request took 3.1 seconds of processing, so it was billed 1 unit ($0.0083).
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/fast-speech-recognition-api-awerks-whisperx`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/fast-speech-recognition-api-awerks-whisperx/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "8546c72072508102e7c5c3ca73ed05f48ccb910248717db0db1326df90d09ba5",
    "input": {
      "task": "transcribe",
      "debug": false,
      "diarize": true,
      "only_text": false,
      "audio_file": "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
      "batch_size": 32,
      "align_output": true,
      "file_extension": ".wav"
    }
  }'
```

Response (201 Created):
```json
{
  "id": "7dwchkqr6nrna0d1d8db6c3sxm",
  "version": "8546c72072508102e7c5c3ca73ed05f48ccb910248717db0db1326df90d09ba5",
  "status": "starting",
  "created_at": "2026-10-23T18:51:55.163Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/fast-speech-recognition-api-awerks-whisperx/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK, `output` shortened):
```json
{
  "id": "7dwchkqr6nrna0d1d8db6c3sxm",
  "status": "succeeded",
  "output": "{\"segments\": [{\"start\": 2.697, \"end\": 6.779, \"text\": \" The little tales they tell are false.\", \"words\": [{\"word\": \"The\", \"start\": 2.697, \"end\": 2.797, \"score\": 0.908, \"speaker\": \"SPEAKER_00\"}], \"speaker\": \"SPEAKER_00\"}], \"word_segments\": [{\"word\": \"The\", \"start\": 2.697, \"end\": 2.797, \"score\": 0.908, \"speaker\": \"SPEAKER_00\"}], \"language\": \"en\"}",
  "error": null,
  "created_at": "2026-10-23T18:51:55.163Z",
  "started_at": "2026-10-23T18:51:56.163Z",
  "completed_at": "2026-10-23T18:52:00.163Z"
}
```

`output` is a single string. With `only_text` `false` it holds JSON with `segments` (each with `start`, `end`, `text`, and `words` and `speaker` when alignment and diarization are on), `word_segments` and the detected `language`; parse it before use. With `only_text` `true` it holds the transcript text only.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `task` that is not `transcribe` or `translate`.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why (for example, an audio link that cannot be downloaded). Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/fast-speech-recognition-api-awerks-whisperx";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "8546c72072508102e7c5c3ca73ed05f48ccb910248717db0db1326df90d09ba5", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  task: "transcribe",
  debug: false,
  diarize: true,
  only_text: false,
  audio_file: "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
  batch_size: 32,
  align_output: true,
  file_extension: ".wav"
}).then((output) => {
  const result = JSON.parse(output);
  for (const segment of result.segments) {
    console.log(`[${segment.start} - ${segment.end}] ${segment.speaker}: ${segment.text.trim()}`);
  }
});
```

#### Python
```python
import json
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/fast-speech-recognition-api-awerks-whisperx"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "8546c72072508102e7c5c3ca73ed05f48ccb910248717db0db1326df90d09ba5", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "task": "transcribe",
    "debug": False,
    "diarize": True,
    "only_text": False,
    "audio_file": "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
    "batch_size": 32,
    "align_output": True,
    "file_extension": ".wav",
})
for segment in json.loads(output)["segments"]:
    print(f"[{segment['start']} - {segment['end']}] {segment.get('speaker')}: {segment['text'].strip()}")
```

## FAQ
- **How do I get word-level timestamps?**
  - Set `align_output` to `true`. Each segment then has a `words` list with `start`, `end` and `score` for every word.
- **How do I know who is speaking?**
  - Set `diarize` to `true`. Segments and words get a `speaker` label such as `SPEAKER_00`.
- **Can I send a link instead of uploading a file?**
  - Yes. Put a direct link in `audio_url` and set `file_extension` to match the file (default `.wav`).
- **Why is `output` a string and not an object?**
  - The model returns its result as one string. With `only_text` `false` it is JSON, so parse it with `JSON.parse` or `json.loads`. With `only_text` `true` it is plain text.
- **How much does a long recording cost?**
  - Each request costs $0.0083 per started 5 seconds of processing. Longer audio takes longer to process, so it uses more units.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Transcribe audio with WhisperX by awerks: timed segments, word-level timestamps and speaker diarization in one request. Pay as you go, no monthly fee.