Name: WhisperX API: Speech to Text with Word Timestamps
Brand: magicapi
SKU: api-magicapi-accelerated-transcription-word-timestamps-diarization-victor-upmeet-whisperx
Availability: InStock
Author: API Market

Shashank Agarwal

# 🎙️ Speech-to-Text Transcription API (victor-upmeet WhisperX)

## Overview
WhisperX is a speech recognition model published by victor-upmeet. Its developer describes it as accelerated transcription, word-level timestamps and diarization with whisperX large-v3.

Send an `audio_file` URL. The model detects the language (or uses the `language` you set), transcribes the speech or translates it to English, and returns the detected language with a list of timed segments. Set `align_output` for word-level timestamps and `diarization` for speaker labels; diarization needs your own Hugging Face access token.

## Features
- **Transcribe or translate**: `task` `transcribe` (default) or `translate` (to English).
- **Language detection**: leave `language` unset to detect it; `language_detection_min_prob` and `language_detection_max_tries` control the detection.
- **Word-level timestamps**: set `align_output` to `true`.
- **Speaker diarization**: set `diarization` to `true`, pass `huggingface_access_token`, and optionally `min_speakers` and `max_speakers`.
- **Voice activity detection**: tune `vad_onset` (default 0.5) and `vad_offset` (default 0.363).
- **Initial prompt**: `initial_prompt` gives the first window context, such as names or terms.

## Use Cases
1. **Meeting and interview transcripts**: timed text, with speaker labels when diarization is on.
2. **Subtitles**: segment and word timings for captions.
3. **Translation to English**: English text from audio in other languages.
4. **Searchable audio archives**: index podcasts, calls and recordings by their text.

## Visual Examples
### Example Input
A 49-second English recording of read sentences, transcribed without alignment or diarization.

[Listen to the example input](https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav)

```json
{
  "debug": false,
  "vad_onset": 0.5,
  "audio_file": "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
  "batch_size": 64,
  "vad_offset": 0.363,
  "diarization": false,
  "temperature": 0,
  "align_output": false
}
```

### Example Output
```json
{
  "detected_language": "en",
  "segments": [
    {
      "avg_logprob": -0.02524347235895183,
      "end": 30.811,
      "start": 2.579,
      "text": " The little tales they tell are false. The door was barred, locked and bolted as well. Ripe pears are fit for a queen's table. A big wet stain was on the round carpet. The kite dipped and swayed but stayed aloft. The pleasant hours fly by much too soon. The room was crowded with a mild wob."
    },
    {
      "avg_logprob": -0.02082519605755806,
      "end": 48.597,
      "start": 33.022,
      "text": " The room was crowded with a wild mob. This strong arm shall shield your honour. She blushed when he gave her a white orchid. The beetle droned in the hot June sun."
    }
  ]
}
```

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `audio_file` | string (URL) | Yes |  |  | Audio file URL |
| `language` | string | No | none |  | ISO code of the language spoken in the audio. Leave unset to detect the language |
| `language_detection_min_prob` | number | No | `0` |  | If language is not specified, the language is detected recursively on different parts of the file until it reaches the given probability |
| `language_detection_max_tries` | integer | No | `5` |  | If language is not specified, the language is detected following language_detection_min_prob, but detection stops after the given number of retries. If max retries is reached, the most probable language is kept. |
| `task` | string | No | `transcribe` | `transcribe`, `translate` | Task to perform on the audio file. Options are: transcribe, translate (English only) |
| `initial_prompt` | string | No | none |  | Optional text to provide as a prompt for the first window |
| `batch_size` | integer | No | `64` |  | Parallelization of input audio transcription |
| `temperature` | number | No | `0` |  | Temperature to use for sampling |
| `vad_onset` | number | No | `0.5` |  | VAD onset |
| `vad_offset` | number | No | `0.363` |  | VAD offset |
| `align_output` | boolean | No | `false` |  | Aligns whisper output to get accurate word-level timestamps |
| `diarization` | boolean | No | `false` |  | Assign speaker ID labels |
| `huggingface_access_token` | string | No | none |  | Your Hugging Face access token (read), required to enable diarization. The account must have accepted the user agreements of the speaker diarization models. |
| `min_speakers` | integer | No | none |  | Minimum number of speakers if diarization is activated (leave blank if unknown) |
| `max_speakers` | integer | No | none |  | Maximum number of speakers if diarization is activated (leave blank if unknown) |
| `user_agent` | string | No | none |  | Override the User-Agent used to download the audio file. Useful when the host blocks the default value. |
| `debug` | boolean | No | `false` |  | Print out compute/inference times and memory usage information |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.059 per API unit.
- **1 API unit = 1 request with up to 37 seconds of processing.** A request that takes longer uses one unit per started 37-second block (for example, 80 seconds of processing uses 3 units, $0.177).
- Processing time grows with the length of the audio and with `align_output` and `diarization`.
- **Example cost**: the 49-second example file took 2.9 seconds of processing, which is 1 unit, $0.059.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/accelerated-transcription-word-timestamps-diarization-victor-upmeet-whisperx`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/accelerated-transcription-word-timestamps-diarization-victor-upmeet-whisperx/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "655845d6190ef70573c669245f245892cd039df4b880a1e3a65852c09252f5cc",
    "input": {
      "debug": false,
      "vad_onset": 0.5,
      "audio_file": "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
      "batch_size": 64,
      "vad_offset": 0.363,
      "diarization": false,
      "temperature": 0,
      "align_output": false
    }
  }'
```

Response (201 Created):
```json
{
  "id": "bpkvq0xrxhrne0d1c6pac9swxr",
  "version": "655845d6190ef70573c669245f245892cd039df4b880a1e3a65852c09252f5cc",
  "status": "starting",
  "created_at": "2026-10-22T03:34:32.116Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/accelerated-transcription-word-timestamps-diarization-victor-upmeet-whisperx/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "bpkvq0xrxhrne0d1c6pac9swxr",
  "status": "succeeded",
  "output": {
    "detected_language": "en",
    "segments": [
      {
        "avg_logprob": -0.02524347235895183,
        "end": 30.811,
        "start": 2.579,
        "text": " The little tales they tell are false. The door was barred, locked and bolted as well. Ripe pears are fit for a queen's table. A big wet stain was on the round carpet. The kite dipped and swayed but stayed aloft. The pleasant hours fly by much too soon. The room was crowded with a mild wob."
      },
      {
        "avg_logprob": -0.02082519605755806,
        "end": 48.597,
        "start": 33.022,
        "text": " The room was crowded with a wild mob. This strong arm shall shield your honour. She blushed when he gave her a white orchid. The beetle droned in the hot June sun."
      }
    ]
  },
  "error": null,
  "created_at": "2026-10-22T03:34:32.116Z",
  "started_at": "2026-10-22T03:34:33.116Z",
  "completed_at": "2026-10-22T03:34:36.116Z"
}
```

`output` is an object with `detected_language` (the language code) and `segments`, a list of segments with `start` and `end` times in seconds, `text` and `avg_logprob`. With `align_output` the segments carry word-level timestamps; with `diarization` they carry speaker labels.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `audio_file` or a `task` that is not `transcribe` or `translate`.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.
## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/accelerated-transcription-word-timestamps-diarization-victor-upmeet-whisperx";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "655845d6190ef70573c669245f245892cd039df4b880a1e3a65852c09252f5cc", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "debug": false,
  "vad_onset": 0.5,
  "audio_file": "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
  "batch_size": 64,
  "vad_offset": 0.363,
  "diarization": false,
  "temperature": 0,
  "align_output": false
}).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/accelerated-transcription-word-timestamps-diarization-victor-upmeet-whisperx"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "655845d6190ef70573c669245f245892cd039df4b880a1e3a65852c09252f5cc", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print(create_and_wait_for_prediction({
    "debug": False,
    "vad_onset": 0.5,
    "audio_file": "https://blog.api.market/wp-content/uploads/2026/10/whisperx-input-api-market.wav",
    "batch_size": 64,
    "vad_offset": 0.363,
    "diarization": False,
    "temperature": 0,
    "align_output": False
}))
```

## FAQ
- **How do I get word-level timestamps?**
  - Set `align_output` to `true`.
- **How do I label speakers?**
  - Set `diarization` to `true` and pass your Hugging Face read token in `huggingface_access_token`. The token's account must have accepted the user agreements of the speaker diarization models. Set `min_speakers` and `max_speakers` if you know the speaker count.
- **Should I set the language?**
  - If you know it, yes: pass its ISO code in `language` to skip detection. Otherwise leave it unset.
- **Can it translate?**
  - Yes, to English only. Set `task` to `translate`.
- **What does a long file cost?**
  - One unit ($0.059) covers up to 37 seconds of processing; each further started 37-second block adds one unit. The 49-second example finished in under 3 seconds.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.

## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello@magicapi.com.


WhisperX speech to text API: transcribe or translate audio with whisperX large-v3, word-level timestamps and speaker labels. Pay as you go, no subscription.